If FULLSEND_OUTPUT_DIR is set, read and follow .fullsend/AGENTS.md
before doing any work.
This file provides guidance to Agentic AI coding tools when working with code in this repository.
This is the rhdh-plugin-export-overlays repository — a metadata and automation hub for managing dynamic plugins for Red Hat Developer Hub (RHDH). It does NOT contain plugin source code. Instead, it references upstream plugin repositories and defines how to package them as OCI container images for RHDH.
Key versions are tracked in versions.json ( Backstage version, Node version, redhat-developer-hub CLI version).
The repo has two distinct metadata systems that serve different purposes:
-
Workspaces (
workspaces/*/) — Define how to build dynamic plugin OCI images. Each workspace maps to an upstream source repo (monorepo or standalone). Key files:source.json— Source repo URL, git ref, and Backstage versionplugins-list.yaml— Which plugins to export, with optional CLI argsmetadata/*.yaml—kind: Packageentities describing each built artifact (version, OCI reference,appConfigExamples)plugins/<plugin>/overlay/— Files that replace/add source files during packagingpatches/*.patch— Unified diffs applied to the workspace source before build
-
Catalog entities (
catalog-entities/extensions/) — Define how plugins appear in the RHDH Extensions UI. These are separate from workspace metadata:plugins/*.yaml—kind: Pluginentities with descriptions, icons, categories, highlights (user-facing display)collections/*.yaml— Groupings (featured, recommended, cicd, openshift, redhat)plugins/all.yaml— Index file; every plugin YAML must be listed here
Package entities live in workspaces/*/metadata/, not in catalog-entities/extensions/. They are merged at build time into the plugin catalog index image.
main— Development branch for next RHDH release. All new workspaces go here.release-x.y— Long-running release branches. Only receive updates to existing workspaces, never new workspaces.
Plugins fall into three support levels, tracked in text files at the repo root:
rhdh-supported-packages.txt— Red Hat supported (GA or TP heading to GA)rhdh-community-packages.txt— Community supported
Note: the community plugin sweep (community-plugin-sweep.yaml) selects packages from
spec.support in workspaces/*/metadata/*.yaml, not from these files — the metadata is
what the build publishes from. The two currently disagree (41 workspaces carry a
community package; the txt file names 20), so do not treat either as authoritative for
the other's purpose.
Auto-discovery covers three npm scopes (defined in plugins-regexps):
@backstage-community/(github.com/backstage/community-plugins)@red-hat-developer-hub/(github.com/redhat-developer/rhdh-plugins)@roadiehq/(github.com/RoadieHQ/roadie-backstage-plugins)
There is no local build system — all building, testing, and publishing happens via GitHub Actions.
On a PR, comment:
/publish— Build and publish test OCI images (taggedpr_<number>__<version>)/smoketest— Run smoke tests against last published artifacts (requires prior/publish)/override-backstage— Override workspace Backstage compatibility version (createsbackstage.jsonand rewrites metadata OCI tags)/update-versions— Copyversions.jsonfrom the PR base branch onto the PR branch (release-line alignment)/update-commit— Re-run automatic plugin repo ref discovery for the single touched workspace and update the PR/testor/test e2e-tests— Run e2e tests. Only relevant for PRs that modify workspaces containing ane2e-tests/directory (e.g., thebackstageworkspace)
| Workflow | Trigger | Purpose |
|---|---|---|
update-plugins-repo-refs.yaml |
Daily + manual | Auto-generates PRs for plugin version updates |
publish-workspace-plugins.yaml |
Push to release branches | Publishes final OCI images |
pr-actions.yaml |
PR comments | Handles /publish, /smoketest, /override-backstage, /update-versions, and /update-commit commands |
run-workspace-smoke-tests.yaml |
After publish | Verifies plugins load in RHDH container |
community-plugin-sweep.yaml |
Daily + manual | Load-tests every spec.support: community package with the Docker-free smoke-tests-native/ harness |
check-backstage-compatibility.yaml |
Push + PRs | Gates release branch creation on compatibility |
sync-user-guide-to-wiki.yaml |
Weekly + manual | Syncs user-guide/ to GitHub Wiki with placeholder injection |
# Update plugin refs (e.g., for RBAC plugins on main)
gh workflow run update-plugins-repo-refs.yaml -f regexps="@backstage-community/plugin-rbac" -f single-branch=main
# Sync docs to wiki (dry run)
gh workflow run sync-user-guide-to-wiki.yaml -f dry_run=trueAfter cloning, enable the pre-commit hook to run E2E code quality checks (ESLint, Prettier, TypeScript) locally before pushing:
git config core.hooksPath .githooksThe hook only triggers when workspaces/*/e2e-tests/** files are staged — zero overhead otherwise. It uses the same shared script (scripts/e2e-code-quality.sh) as the CI workflow, so checks are always in sync. See .githooks/README.md for details on combining with existing hooks.
- Create
workspaces/<name>/source.json:{"repo":"https://github.com/org/repo","repo-ref":"<tag-or-sha>","repo-flat":false,"repo-backstage-version":"1.45.1"}repo-flat:trueif plugins are at repo root,falseif inside a workspace subdirectory
- Create
workspaces/<name>/plugins-list.yamllisting plugin paths - Create
workspaces/<name>/metadata/<package-name>.yamlfor each plugin (kind: Package)
- Overlay (
plugins/<plugin>/overlay/): Replaces or adds entire files during packaging. Used for plugin-specific changes. - Patch (
patches/*.patch): Applies line-by-line changes to workspace source before build. Used for workspace-wide fixes. Numbered prefix controls application order (e.g.,1-fix-something.patch).
When a patch in patches/*.patch bumps a dependency across a major version (e.g., 2.x to 3.x), the change carries higher risk than a minor or patch-level bump — even when the bump comes through an intermediate dependency. Major versions introduce documented breaking API changes that can cause runtime failures in exported plugins.
Why this matters for this repo: Patches modify yarn.lock to force dependency versions across the upstream workspace. A major version bump in a patched dependency could break any exported plugin whose code (or transitive dependencies) relies on the old API. Unlike upstream repos where CI catches breakage, this repo's patches bypass the upstream test suite — so compatibility must be verified during review.
Review criteria for major version bumps:
When reviewing a PR where a patch bumps a dependency across a major version:
- Detect the major version change. Compare the old and new version numbers in the patch. A change where the first numeric component increases (e.g.,
2.2.1to3.0.8) is a major bump with potential breaking changes. - Check the dependency's changelog or migration guide for breaking API changes. Common breakages include: removed or renamed functions, changed return types, removed configuration options, and altered default behavior.
- Assess whether exported plugins use the affected APIs. Check
plugins-list.yamlto identify which plugins are exported. Since this repo does not contain plugin source code, use therepoURL andrepo-reffrom the workspace'ssource.jsonto browse or clone the upstream repository at the pinned ref. Search the upstream plugin source for imports and usage of the bumped dependency's changed APIs. If the dependency is only in the scaffold/dev tree (see "Reviewing CVE / Dependency Patches" if present), the risk is lower — but still flag it if the patch forces the version across the entireyarn.lock. - Flag the risk for human verification when breaking changes exist and the dependency is in the shipped tree. Do not approve major version bump patches with only generic version-consistency checks. Explicitly acknowledge the major version change and either assess breaking API risk or request human review.
- Check wrapper and type packages. When a dependency crosses a major version, related packages often need compatible bumps — for example,
@types/*type definition packages (e.g.,@types/js-cookie2.x to 3.x) and wrapper libraries (e.g.,react-useupdating its peer dependency range). Verify that all related packages in the patch are version-compatible with each other.
Do not approve major version bump patches without explicit risk assessment. A patch that passes build does not guarantee runtime compatibility — the old API may still be called at runtime by plugin code that was not exercised during the build.
Leverage CI verification. If the workspace has E2E tests (e2e-tests/ directory), they exercise the plugin through basic acceptance criteria and can catch runtime breakage from major version bumps — use the /test PR command to run them. Smoke tests (/smoketest PR command) attempt to load all workspace plugins as a basic build consistency check and should be considered a minimum verification step. Neither replaces a manual review of breaking API changes, but passing E2E and smoke tests increases confidence that exported plugins are compatible with the updated dependency.
Uses kind: Plugin with schema: https://raw.githubusercontent.com/redhat-developer/rhdh-plugins/refs/heads/main/workspaces/extensions/json-schema/plugins.json
Key fields: spec.categories, spec.highlights, spec.icon (base64 SVG), spec.packages (links to Package entities), spec.description (markdown, no images).
After creating/editing, add the file to plugins/all.yaml.
Uses kind: Package. Key fields: spec.packageName, spec.dynamicArtifact (OCI reference), spec.version, spec.backstage.role (frontend-plugin/backend-plugin), spec.support (community/production/tech-preview), spec.appConfigExamples.
E2E tests live in workspaces/<name>/e2e-tests/ and use @red-hat-developer-hub/e2e-test-utils — a shared package that handles RHDH deployment, Playwright fixtures, helpers, and plugin configuration. For the latest and most complete documentation, see: https://github.com/redhat-developer/rhdh-e2e-test-utils/tree/main/docs
workspaces/<plugin>/
├── metadata/ # Plugin metadata (Package CRD) — consumed by deploy()
│ └── backstage-*.yaml # spec.dynamicArtifact, spec.appConfigExamples
└── e2e-tests/
├── package.json # Dependencies (e2e-test-utils, @playwright/test)
├── playwright.config.ts # Extends base config, defines project(s)
├── .env # Local env vars (optional)
└── tests/
├── config/ # All files optional — auto-generated from metadata
│ ├── app-config-rhdh.yaml
│ ├── rhdh-secrets.yaml
│ └── dynamic-plugins.yaml
└── specs/
└── <plugin>.spec.ts # Test specification
Each Playwright project creates a separate Kubernetes namespace (project name = namespace name). The test framework:
- Global setup (once per run) — checks binaries (
oc,kubectl,helm), detects cluster domain, deploys Keycloak - Worker fixture (once per worker) — creates
RHDHDeployment(projectName), sets CWD to the workspace'se2e-tests/directory - Test execution —
beforeAllconfigures + deploys RHDH,beforeEachhandles login, tests useuiHelper/pagefor assertions - Teardown (CI only) — per-project namespace deletion via a custom Playwright reporter as soon as all tests in that project finish
import { test, expect } from "@red-hat-developer-hub/e2e-test-utils/test";
import { $ } from "@red-hat-developer-hub/e2e-test-utils/utils";
test.describe("My Plugin", () => {
test.beforeAll(async ({ rhdh }) => {
await rhdh.configure({ auth: "keycloak" });
// Optional: deploy external services before RHDH
await $`bash scripts/setup.sh ${rhdh.deploymentConfig.namespace}`;
await rhdh.deploy();
});
test.beforeEach(async ({ loginHelper }) => {
await loginHelper.loginAsKeycloakUser();
});
test("verify feature", async ({ uiHelper }) => {
await uiHelper.openSidebar("My Plugin");
await uiHelper.verifyHeading("Expected Title");
});
});Playwright's beforeAll runs once per worker, not once per test run. When a test fails, Playwright kills the worker and creates a new one for remaining tests — causing beforeAll to run again. rhdh.deploy() has built-in protection (it skips if the deployment already exists), but other expensive operations in your beforeAll do not.
When to use: Wrap beforeAll in test.runOnce() when you have pre-deployment setup (external services, scripts, env var extraction) that shouldn't repeat on worker restart.
When NOT needed: If your beforeAll only calls rhdh.configure() + rhdh.deploy(), you don't need runOnce — deploy() is already protected internally.
test.beforeAll(async ({ rhdh }) => {
await test.runOnce("tech-radar-setup", async () => {
await rhdh.configure({ auth: "keycloak" });
// Expensive: deploys an external service to the cluster
await $`bash ${setupScript} ${rhdh.deploymentConfig.namespace}`;
// Extract a route URL and set as env var for RHDH config
process.env.DATA_URL = await rhdh.k8sClient.getRouteLocation(
rhdh.deploymentConfig.namespace, "data-provider"
);
await rhdh.deploy(); // safe to nest — has its own internal runOnce
});
});How env vars survive worker restarts: Environment variables set inside runOnce (like process.env.DATA_URL above) are set in the worker process. When the worker restarts, runOnce skips the callback entirely — so the env var is never set in the new worker. This is fine because:
rhdh.deploy()already ran and setRHDH_BASE_URL(which the fixture re-reads from the route)- Secrets were already applied to the cluster as ConfigMaps/Secrets during the first run
- The env vars were only needed during the deployment phase, not during test execution
If a test does need an env var that was set inside runOnce, extract it from the cluster in a separate step outside runOnce:
test.beforeAll(async ({ rhdh }) => {
await test.runOnce("my-setup", async () => {
await rhdh.configure({ auth: "keycloak" });
await $`bash deploy-service.sh ${rhdh.deploymentConfig.namespace}`;
await rhdh.deploy();
});
// Always runs — even after worker restart. Re-derives the value from cluster state.
process.env.MY_SERVICE_URL = await rhdh.k8sClient.getRouteLocation(
rhdh.deploymentConfig.namespace, "my-service"
);
});Key rules:
- The
key(first argument) must be globally unique across all spec files and projects. Prefix with workspace name:"tech-radar-setup","argocd-deploy". - Nesting is safe —
deploy()usesrunOnceinternally, wrapping it in an outerrunOnceis harmless. - Uses file-based flags in
/tmp/scoped to the Playwright runner process. Flags reset automatically between test runs.
rhdh.deploy() performs these steps:
- Merges config files — package defaults + auth config (keycloak/guest) + your
tests/config/overrides (deep merge, later wins) - Processes dynamic plugins — auto-generates from
metadata/*.yamlif nodynamic-plugins.yamlexists; injects metadata configs; resolves OCI URLs based on mode - Applies to cluster — creates ConfigMaps (app-config, dynamic-plugins) and Secrets (with
envsubstfor env var substitution) - Installs RHDH — via Helm chart or Operator based on
INSTALLATION_METHOD - Waits for readiness — two-phase: pod
Ready=True(with early failure detection for CrashLoopBackOff, ImagePullBackOff) + HTTP health check against the route - Sets
RHDH_BASE_URL— so Playwright navigates to the correct URL
Helm upgrades perform a scale-down-and-restart to avoid MigrationLocked errors; fresh installs skip this.
Plugin metadata at workspaces/*/metadata/*.yaml is actively consumed during deployment. The system operates in three modes:
| Mode | Detection | Plugin Packages | Config Injection |
|---|---|---|---|
| PR check | GIT_PR_NUMBER set |
PR-built OCI images (pr_{number}__{version}) |
Yes — metadata appConfigExamples injected |
| Nightly | E2E_NIGHTLY_MODE=true |
Released OCI refs from spec.dynamicArtifact |
No — uses whatever config is in the file |
| Local dev | Neither set | Local paths as-is (bundled in container) | Yes — metadata appConfigExamples injected |
Priority: GIT_PR_NUMBER (forces PR mode) > E2E_NIGHTLY_MODE > JOB_NAME containing periodic-
PR mode requires:
GIT_PR_NUMBERset (CI exports it automatically; locally:export GIT_PR_NUMBER=1845)- OCI images published via
/publishPR comment before running tests source.jsonandplugins-list.yamlin the workspace root (used to fetch plugin versions from the source repo and build OCI URLs likeoci://ghcr.io/.../plugin-name:pr_1845__1.2.3)
Without GIT_PR_NUMBER (local dev): Plugins use local paths (./dynamic-plugins/dist/...) bundled inside the RHDH container image. Metadata configs are still injected from spec.appConfigExamples. This is the default mode for yarn test from a workspace.
Nightly mode uses released OCI refs directly from each plugin's spec.dynamicArtifact in metadata (e.g., oci://ghcr.io/.../plugin:bs_1.45.3__1.13.0). No config injection — plugins use their baked-in defaults. Enabled via E2E_NIGHTLY_MODE=true or when JOB_NAME contains periodic-.
When no dynamic-plugins.yaml exists, ALL metadata files are read to auto-generate the complete plugin configuration. This is the recommended approach — most workspaces don't need a dynamic-plugins.yaml.
All files in tests/config/ are optional — only create them when you need to override defaults:
app-config-rhdh.yaml— RHDH app configuration (plugin settings, backend config)rhdh-secrets.yaml— Kubernetes Secret manifest for injecting env vars into RHDHdynamic-plugins.yaml— Plugin overrides (usually NOT needed — auto-generated from metadata)value_file.yaml— Helm chart value overridessubscription.yaml— Operator subscription overrides
Environment variables in RHDH config: To use an env var in app-config-rhdh.yaml, it must first be defined in rhdh-secrets.yaml. The flow is:
Environment (CI Vault / .env) rhdh-secrets.yaml app-config-rhdh.yaml
MY_TOKEN=abc123 → MY_TOKEN: $MY_TOKEN → token: ${MY_TOKEN}
(envsubst replaces $VAR) (references the K8s Secret)
envsubst runs only on rhdh-secrets.yaml. Other config files reference the Secret values with ${VAR} syntax — they are not substituted directly.
Config file paths are resolved from test.info().project.testDir (Playwright-provided absolute path), NOT from process.cwd(). This enables the same test code to work from both:
- Workspace level:
cd workspaces/tech-radar/e2e-tests && yarn test - Repo root:
./run-e2e.sh -w tech-radar
The worker fixture also does process.chdir(e2eRoot) as a complementary safety net for shell scripts and fs calls.
In CI (CI=true), namespaces are automatically deleted by a custom Playwright reporter — not afterAll hooks or worker fixture cleanup. This design is intentional:
afterAllhook: Fires when a worker dies. When a test fails and Playwright restarts the worker for retries, the old worker'safterAlldeletes the namespace before the retry can use it.- Worker fixture teardown: Same problem — runs on worker exit, not on suite completion.
globalTeardown: Runs after all tests but has no visibility into which projects ran or which namespaces were created.
The reporter runs in the main Playwright process (survives worker restarts), tracks per-project test completion including retries, and deletes each project's namespace as soon as its last test finishes.
| Fixture | Scope | Purpose |
|---|---|---|
rhdh |
worker | RHDHDeployment instance — configure(), deploy(), k8sClient, deploymentConfig |
uiHelper |
test | UI interactions — verifyHeading(), openSidebar(), clickButton(), verifyRowsInTable() |
loginHelper |
test | Authentication — loginAsKeycloakUser(), loginAsGuest() |
baseURL |
test | Auto-set to the deployed RHDH URL |
One project per spec file. Each Playwright project creates a separate RHDH deployment in its own namespace. A workspace should have one project targeting one spec file. This keeps deployments isolated and avoids namespace conflicts. If a workspace needs multiple test configurations (different auth, different plugins), use multiple projects with testMatch:
// playwright.config.ts
export default defineConfig({
projects: [
{ name: "my-plugin", testMatch: "my-plugin.spec.ts" },
{ name: "my-plugin-guest", testMatch: "my-plugin-guest.spec.ts" },
],
});Don't create config files unless needed. The package auto-generates plugin config from metadata. Most workspaces work with zero config files.
run-e2e.sh runs E2E tests from ALL workspaces (or a subset) in a single Playwright process from the repo root. Used by CI nightly jobs and for cross-workspace validation.
./run-e2e.sh # All workspaces
./run-e2e.sh -w tech-radar # Single workspace
./run-e2e.sh -w backstage -w keycloak # Multiple workspaces
./run-e2e.sh --list # Dry run — list discovered projects
./run-e2e.sh --workers=4 # Control parallelismWhy a single root Playwright instead of per-workspace parallel?
- Workers auto-balance across all projects (no idle resources when workspaces differ in size)
- Keycloak
globalSetupruns once (no race conditions) - Single HTML report with traces/screenshots/videos (no merge step)
- Standard Playwright CLI works (
--project,--grep,--shard) - Yarn resolutions validates dependency upgrades across all workspaces in one run
Why yarn workspaces is required (not optional): Playwright errors out if @playwright/test is loaded from more than one file path in a process. With separate node_modules per workspace, each resolves from a different path. Yarn workspaces hoists to a single root node_modules.
Nothing is committed — package.json, playwright.config.ts, .yarnrc.yml are generated at runtime.
From a workspace (development):
cd workspaces/tech-radar/e2e-tests
cp .env.sample .env # if exists, fill in secrets
yarn install
yarn test # or: npx playwright test
yarn test --headed # watch in browser
yarn test --ui # Playwright UI mode
yarn report # open last HTML reportFrom repo root (CI / cross-workspace):
./run-e2e.sh -w tech-radar
# or with local e2e-test-utils build:
E2E_TEST_UTILS_PATH=/path/to/rhdh-e2e-test-utils ./run-e2e.sh -w tech-radarTest locally with PR-built OCI images:
export GIT_PR_NUMBER=1845
yarn test # uses OCI images published by that PR's /publish commandNamespaces are NOT auto-deleted locally. The teardown reporter only runs when CI=true. After local test runs, namespaces persist on the cluster. Clean up manually:
oc delete project <namespace>Keycloak is reused automatically. If Keycloak is already deployed from a previous run, global setup detects it and skips re-deployment. Use SKIP_KEYCLOAK_DEPLOYMENT=true only if you don't want Keycloak at all (e.g., guest-auth tests).
| Variable | Purpose | Default |
|---|---|---|
RHDH_VERSION |
RHDH version to deploy | "next" |
INSTALLATION_METHOD |
"helm" or "operator" |
"helm" |
GIT_PR_NUMBER |
PR number — enables PR mode with PR-built OCI images | - |
E2E_NIGHTLY_MODE |
"true" or "1" — enables nightly mode with released OCI refs |
- |
JOB_NAME |
CI job name; periodic- prefix triggers nightly mode |
- |
SKIP_KEYCLOAK_DEPLOYMENT |
Skip Keycloak in global setup | - |
CI |
Enables forbidOnly, teardown reporter, namespace cleanup |
- |
E2E_TEST_UTILS_PATH |
Local e2e-test-utils build path (takes precedence over version) | - |
E2E_TEST_UTILS_VERSION |
Pin e2e-test-utils npm version | latest (nightly) |
CATALOG_INDEX_IMAGE |
Override the catalog index image in the RHDH chart | - |
| Job | When | Workspaces | OCI Images |
|---|---|---|---|
PR check (e2e-ocp-helm) |
PR with e2e changes | Changed workspace only | PR-built (pr_ tags) |
Nightly (e2e-ocp-helm-nightly) |
Daily cron / manual | All workspaces | Released (metadata refs) |
Trigger nightly manually: comment /test e2e-ocp-helm-nightly on a PR.
Two Claude Code skills are available at .claude/skills/ for investigating E2E failures:
e2e-failure-analysis— structured workflow: artifact download, diagnostics, trace correlation, cluster log search, and config comparisonplaywright-trace— Playwright trace CLI for inspecting trace ZIP files (actions, DOM snapshots, requests, console, errors)
When fixing E2E test failures from [fullsend] E2E: issues:
workspaces/<workspace>/e2e-tests/— any file under the e2e-tests directory
- Plugin source code (
workspaces/*/plugins/) - CI configuration (
.github/) - Repository config (
CLAUDE.md,CODEOWNERS,.fullsend/)
When the issue says fix_category: product_bug, add test.skip instead
of fixing the test:
test.skip(!!process.env.E2E_NIGHTLY_MODE, "<root cause summary>");
After changes, run from the workspace's e2e-tests directory:
npx tsc --noEmit
npx eslint <changed-files>
npx prettier --check <changed-files>
README.md— Repo overview, PR workflow, testing proceduresuser-guide/— 6-part contributor guide (getting started, export tools, ownership, metadata sync, versions, patches) plus catalog index pipeline docsuser-guide/troubleshooting-catalog-index.md— Troubleshooting content embedded byrenderCatalogStatus.pyinto each generated status page. Anchor slugs must stay in sync withREASON_ANCHORSin the renderercatalog-entities/extensions/README.md— Extensions catalog metadata format- GitHub Wiki — Auto-synced from
user-guide/with dynamic content injection ({{AUTO:*}}placeholders replaced fromversions.json) - E2E test utils docs — https://github.com/redhat-developer/rhdh-e2e-test-utils/tree/main/docs — latest API docs, changelogs, tutorials, and configuration reference for
@red-hat-developer-hub/e2e-test-utils