feat: Docker-free in-process smoke harness for backend dynamic plugins#2714
feat: Docker-free in-process smoke harness for backend dynamic plugins#2714gustavolira wants to merge 4 commits into
Conversation
|
Skipping CI for Draft Pull Request. |
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
|
/review |
|
/agentic-review |
|
/publish |
|
PR action ( |
github-actions
Bot
added
the
non-workspace-changes
gustavolira
changed the title
gustavolira
force-pushed
the
poc/native-smoke-harness
branch
from
c41bdb4 to
588b296
Compare
There was a problem hiding this comment.
In my opinion, this is a step in the good direction, I am just worried that any potential gain atm is negated by the lack of coverage of the frontend and catalog plugins. If I still need to run the old smoke tests on the workspace to test these plugins, what good is it for to have the backend tested faster?
Edit: did not notice we had pure backend workspaces, it makes sense to run it in that case. It could also catch backend issues faster.
gustavolira
force-pushed
the
poc/native-smoke-harness
branch
from
b83702e to
c279415
Compare
|
Heads-up that the frontend / UI-render half this PR scopes out ("backend-only; UI render is the NFS / app-next path") now has a proven, Docker-free path in redhat-developer/rhdh#5005. That PR is basically the frontend complement to this one. It boots the RHDH backend + the legacy app ( Two things from it that are directly relevant here:
Honest gap for doing the same in this repo: the frontend harness needs the full RHDH app to boot, which the overlays repo doesn't have locally — so a "frontend smoke" here wouldn't be a trivial port of this backend harness. But the approach and the OCI-for-frontend validation are proven if we want to add the frontend counterpart down the line. Together they'd cover the whole Docker-free/cluster-free smoke: backend loads (this PR) + frontend renders (#5005). |
Adds smoke-tests-native/: validates RHDH dynamic backend plugins in-process via the published install-dynamic-plugins CLI + startTestBackend — no Docker, no cluster (~20x faster than the per-workspace docker run smoke for that scope). Runs in a dedicated native-smoke.yaml workflow (pull_request + workflow_dispatch). Loader, module-resolution and plugin-config are ported from RHDH PR #4967. Scope: backend non-catalog plugins. Catalog-extending modules and frontend stay on the Docker smoke (catalog wiring + NFS are follow-ups). Additive — removes nothing. Tickets: RHIDP-15075, RHIDP-15076, RHIDP-13530 (epic RHIDP-13501). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
….json - Warn when backend plugins are filtered by KNOWN_FAILURES, and when nothing at all was validated (avoids a silent pass on a known-failure-only workspace); include the skipped list in results.json. [native-smoke.ts] - Move the CLI version check + temp-dir setup inside the try, so a failure there still writes a results.json (status: error) instead of exiting with no report. - Validate the --dynamic-plugins file exists (not just that the arg was passed). - Warn instead of silently skipping a plugin dir with a malformed package.json. [loader.ts] - Clarify that plugin-config KNOWN_FAILURES/configOverrides are for multi-plugin/ workspace runs and kept in TS (type-checked, in sync with the RHDH port) rather than package.json. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… small cleanups - bump esbuild 0.24.2 -> 0.28.1 (clears GHSA-67mh-4wv8-2f99 advisory range) and typescript 6.0.2 -> 6.0.3; engines.yarn ">=4" to match packageManager yarn@4 - workflow: single source of truth for the default plugin ref (env only, input default removed); corepack before setup-node + yarn cache keyed on smoke-tests-native/yarn.lock - native-smoke: shared BackendStartResult type; writeErrorReport helper also covers usage errors (results.json now written on exit 2 too); drop unused env param from run(); single-pass KNOWN_FAILURES partition - loader: normalize "./dist/…" package mains; type-guard filter instead of "as string[]"; hoist repeated role check Verified on Node 24: tsc clean, esbuild build ok, end-to-end smoke run status: pass (backend loaded 1/1, startTestBackend booted), usage-error path writes results.json (status: error) and exits 2. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
gustavolira
force-pushed
the
poc/native-smoke-harness
branch
from
c279415 to
ef2ee44
Compare
…(S3776) - resolveOutPath(): the --out CLI argument is resolved and constrained to the working directory before any write, so a faulty argument can never write outside it; rejected paths exit 2 with a clear message - extract partitionKnownFailures() from main(), bringing its cognitive complexity back under the threshold Verified on Node 24: tsc clean, build ok, happy path status: pass, "../escape.json" rejected (exit 2), usage error still writes results.json. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
3 participants
What
A Docker-free, in-process smoke harness (
smoke-tests-native/) that validates RHDHdynamic backend plugins: it installs them with the published
@red-hat-developer-hub/cli-module-install-dynamic-pluginsCLI, then boots them withstartTestBackend(@backstage/backend-test-utils) — no Docker container, no cluster.About 20x faster than the per-workspace
docker run rhdhsmoke-test for that scope.The loader / module-resolution / plugin-config are ported from RHDH's already-green
nightly test (
plugin-dynamic-loading.spec.ts, redhat-developer/rhdh#4967).How it runs in CI
A dedicated, separate workflow
.github/workflows/native-smoke.yaml— independent of theDocker-based
run-workspace-smoke-tests.yaml:pull_request(pathssmoke-tests-native/**): validates the harness on every change here.workflow_dispatch: run against any plugin ref on demand.It installs skopeo, builds with esbuild, runs
yarn smoke, and uploadsresults.json.There is no
docker runanywhere. The PR's run is green — the harness installed theplugin and booted it in-process (
status: pass).Scope (honest boundary)
minimal harness — tracked as follow-up) and frontend plugins (backend-only; UI render is
the NFS / app-next path). Those stay on the Docker smoke for now.
This is additive — it removes no existing Docker smoke-test. Migrating covered workspaces
off Docker is a separate, parity-gated step.
Verification
smokecheck.tsc --noEmitclean;yarn install --immutablepasses; harness run =passon a good plugin,erroron a bad ref.Tickets: RHIDP-15075, RHIDP-15076, RHIDP-13530 (epic RHIDP-13501).
🤖 Generated with Claude Code