docs(ai): add test-placement skill for the plugin testing strategy (RHIDP-13501)

[gustavolira]

Summary

Adds a test-placement Claude Code skill (.claude/skills/test-placement/): when a developer asks "where should I test this?", the assistant gathers the context and proposes the right repo (rhdh-plugins / rhdh-plugin-export-overlays / rhdh), the right test layer (L1 unit → L4b cluster e2e), the location, and how to scaffold the test.

It encodes the per-repository testing responsibility split consolidated on epic RHIDP-13501: source bugs belong here (unit/component tests next to the plugin, dev app, export-dynamic check) · artifact bugs → overlays native smoke (rhdh-plugin-export-overlays#2714) · integration/render bugs → the rhdh cluster-free harness (rhdh#5005) · platform bugs → cluster e2e. Guiding rule: the cheapest environment that catches the bug wins.

Sibling PRs add the same skill to rhdh (rulesync-generated for Claude/Cursor/OpenCode) and rhdh-plugin-export-overlays.

Related

Epic RHIDP-13501 · rhdh#5045 · rhdh#5044 (layer matrix) · rhdh#5005 · rhdh-plugin-export-overlays#2714

🤖 Generated with Claude Code

Tooling note (review follow-up)

This lands for Claude Code only (.claude/skills/) on purpose: unlike rhdh, this repo has no rulesync setup, and hand-maintaining parallel .cursor copies invites drift. If the team wants the same guidance for Cursor/OpenCode users here, the right move is adopting rulesync as rhdh did (source in .rulesync/, generated copies committed, CI sync check) — happy to do that as a follow-up.

[rhdh-qodo-merge]

PR Summary by Qodo

docs(ai): add test-placement skill for plugin test strategy

📝 Documentation 🕐 Less than 10 minutes

AI Description

• Add a new Claude Code skill to advise where plugin-related changes should be tested.
• Document repo ownership boundaries (plugins vs overlays vs rhdh) and a cost-based test-layer
 ladder.
• Provide a concrete output format for recommending placement, scaffolding steps, and rationale.

Diagram

graph TD
  Dev(("Developer")) --> Claude["Claude Code assistant"] --> Skill["test-placement skill"] --> Rules["Placement rules"]
  Rules --> Plugins["rhdh-plugins tests"] & Overlays["overlays smoke"] & Rhdh["rhdh e2e"]

Loading

High-Level Assessment

The following are alternative approaches to this PR:

1. Move guidance into CONTRIBUTING/testing docs

• ➕ Single source of truth for humans reviewing/testing without Claude tooling
• ➕ Easier to discover via repo docs search/browsing
• ➖ Not directly actionable in-editor when developers ask the question
• ➖ Doesn’t enforce/standardize the assistant’s response format

2. Implement as a script/checklist command (CLI) instead of a skill doc

• ➕ Could ask structured questions and output a template consistently
• ➕ Potential to auto-link to repo paths and commands
• ➖ More maintenance and distribution overhead than a markdown-backed skill
• ➖ Harder to iterate quickly vs updating a skill document

Recommendation: Keeping this as a Claude skill is the best fit: it’s lightweight (markdown-only), available at the point of use (when developers ask), and encodes a consistent repo/layer decision framework without introducing new tooling to maintain. If discoverability becomes an issue, consider additionally linking to this skill from existing contributor/testing docs.

Files changed (1) +112 / -0

Documentation (1) +112 / -0

SKILL.mdAdd test-placement Claude skill with repo/layer decision framework +112/-0

Add test-placement Claude skill with repo/layer decision framework

• Introduces a new Claude Code skill documenting how to choose the correct repository and test layer (L1–L4b) for changes in the RHDH dynamic-plugin ecosystem. Includes a decision table for repo ownership boundaries and practical scaffolding guidance per target repo/harness.

.claude/skills/test-placement/SKILL.md

[rhdh-qodo-merge]

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 📜 Skill insights (0)

Context used

✅ Compliance rules (platform): 11 rules

1. Unscoped cross-repo paths ✓ Resolved 🐞 Bug ⚙ Maintainability

Description

The new skill references several repo-relative file paths/commands (e.g., e2e-tests/...,
docs/e2e-tests/..., rhdh/docs/...) without consistently stating which repository they are
relative to, which can lead the assistant (or a developer) to try to open/run them from the current
repo and fail.

Code

.claude/skills/test-placement/SKILL.md[R56-84]

+The full per-spec classification of the RHDH e2e suite lives in `rhdh/docs/e2e-tests/layer-migration-matrix.md` (RHIDP-15076) — consult it before adding or migrating an e2e spec.
+
+## Step 4 — How to create the test (per placement)
+
+### `rhdh-plugins` (or other source repo) — plugin correctness
+
+- **Where:** `workspaces/<workspace>/plugins/<plugin>/src/**` next to the code, following that workspace's existing test setup (Jest/RTL; MSW for API mocks).
+- **How:** copy the pattern of a neighboring `*.test.ts(x)`; run with the workspace's `yarn test`. Manual verification via the workspace dev app (`yarn start`).
+- Also validate packaging when the plugin's build config changed: `npx @red-hat-developer-hub/cli plugin export` must produce `dist-dynamic/` (+ `dist-scalprum/` for frontend).
+- **Cannot do here:** validate the *published* OCI artifact, or integration with RHDH's app shell/shared deps.
+
+### `rhdh-plugin-export-overlays` — the published artifact
+
+- **Where:** the native smoke harness (`smoke-tests-native/`, overlays PR #2714). Per-plugin: install from OCI with the real `install-dynamic-plugins` CLI and boot backend plugins via `startTestBackend`. Workspace mode: `yarn smoke --workspace <name>` boots every `oci://` artifact of the workspace together.
+- New workspaces usually need **no new test code** — the harness discovers artifacts from `workspaces/<name>/metadata/*.yaml`.
+- **Known gaps (don't fight them):** catalog-extending backend modules don't boot in a minimal `startTestBackend` (upstream `catalog-backend` registration issue — stay on Docker smoke until fixed); plugins whose `dynamicArtifact` is a local `./dynamic-plugins/dist/...` path ship inside the RHDH image and have no OCI artifact to validate here.
+- **Never add UI-render tests here** — this repo has no app to render into. Delegate rendering to the `rhdh` cluster-free harness.
+- `workspaces/*/e2e-tests` (cluster, Playwright + `e2e-test-utils`) exists for plugins whose value *is* cluster integration (topology, tekton, argocd…). Don't add one for UI-only behavior.
+
+### `rhdh` — the real app
+
+**L4a cluster-free harness** (`e2e-tests/playwright.legacy-local.config.ts`, docs at `docs/e2e-tests/local-e2e-harness.md`) — the only cheap place a frontend dynamic plugin can be *rendered*. To enable a spec/test:
+
+1. Plugin not yet installed by the harness? Add its OCI entry to `e2e-tests/local-harness/dynamic-plugins.yaml` (tags on `ghcr.io/redhat-developer/rhdh-plugin-export-overlays/<package>`). If its mount-point config is not in the repo's static `app-config.dynamic-plugins.yaml`, attach the plugin's **canonical `pluginConfig`** (source of truth: `rhdh-plugins` → `workspaces/<ws>/plugins/<plugin>/app-config.dynamic.yaml`) — the harness loads the generated `dynamic-plugins-root/app-config.dynamic-plugins.yaml` last, exactly like the production container.
+2. Config that only exists in CI config maps (`.ci/pipelines/resources/config_map/*`)? Mirror just the needed keys into `app-config.local-e2e.yaml` (objects deep-merge; arrays replace).
+3. Tag the test `{ tag: "@cluster-free" }` and add its spec file to the config's `testMatch` allowlist.
+4. Repopulate + validate: `./e2e-tests/local-harness/populate.sh`, then `yarn --cwd e2e-tests e2e:legacy-local` (CI does this in ~4 min; skopeo is Linux/CI-only).
+
+**L4b cluster e2e** (`e2e-tests/playwright/e2e/**`) — only when the subject *is* cluster/platform behavior or a real external service. Requirements: `component` annotation in `beforeAll` (see `ci-e2e-testing` rule), correct config map choice (RBAC vs non-RBAC), project registration in `e2e-tests/playwright/projects.json` if a new project is needed.

Relevance

⭐⭐⭐ High

Team often fixes ambiguous/incorrect doc paths (accepted PRs #1057, #1407); likely to scope
cross-repo paths.

PR-#1057
PR-#1407

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

The skill text cites rhdh/docs/... and e2e-tests/... as bare paths while the skill itself is
added to this repo, which makes the path resolution ambiguous for readers/tools operating in the
current checkout.

.claude/skills/test-placement/SKILL.md[56-84]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The skill text includes repo-relative paths and commands that are intended to be executed/read in other repositories (not necessarily the repo where the skill is installed). Without explicit scoping, the assistant may attempt to open these paths in the current repo, causing dead-ends and unnecessary follow-up questions.

## Issue Context
This skill lives in `.claude/skills/test-placement/SKILL.md`, but it references `rhdh`-repo-specific locations like `e2e-tests/...` and `docs/e2e-tests/...`.

## Fix Focus Areas
- .claude/skills/test-placement/SKILL.md[56-84]

## Suggested change
- Prefix referenced paths/commands with the owning repo name, e.g.:
 - "In the `rhdh` repo: `docs/e2e-tests/local-e2e-harness.md`"
 - "In the `rhdh` repo: run `yarn --cwd e2e-tests e2e:legacy-local`"
- For key docs, consider using full GitHub links (or explicitly say “in the rhdh repo”) so the assistant doesn’t treat them as local files.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud