Motivation
Current tests run package scripts in Docker against fakes (findmnt/nsenter/systemctl/tuned-adm/...). That validates script logic but never exercises the real path: the operator scheduling the package, running the lifecycle stages, and the SkyhookCustomResource reaching Complete. We want automated evidence that packages actually function through the operator, captured as CI artifacts.
Requirements (broad strokes)
A new tests/e2e/ harness that, per in-scope package:
- installs the NodeWright operator via its Helm chart (pinned),
- makes the dev image available on the cluster,
- applies an example SCR,
- waits for status
Complete within a timeout,
- uploads SCR status as evidence, optionally pod logs
Promote the inline README example SCRs into runnable example CR files per package.
Image delivery (works on fork PRs)
The harness does not push to the org registry. It builds the package image locally and makes it available in the cluster somehow. Possibly by running a registry in the cluster. This runs entirely on the runner, so fork PRs, which get a read-only token and cannot push to ghcr.io/nvidia/... (see #90), can still produce full e2e evidence with no secrets and no org package creation. Publishing stays a separate concern.
Handling interrupts
Consider that many packages have workflows that require an interrupt to fire in order to fully test. This will need consideration as to either be deferred or somehow supported. Often interrupts are reboots but support could be staged to non-reboots (noop, service) and others.
Handling node prerequisites
Packages will likely need some fixtures to be setup in order to work. For example bind-mount expects a directory/fs to be mountable.
Package tiering
The following should at least be covered by the initial enhancement: shellscript, copy-fail, bind-mount. Packages that are more complicated like nvidia-setup/nvidia-tuned could be deferred.
Non-goals
Replacing the unit/fakes tests; GPU/hardware fidelity; publishing images from fork PRs.
Motivation
Current tests run package scripts in Docker against fakes (
findmnt/nsenter/systemctl/tuned-adm/...). That validates script logic but never exercises the real path: the operator scheduling the package, running the lifecycle stages, and the SkyhookCustomResource reachingComplete. We want automated evidence that packages actually function through the operator, captured as CI artifacts.Requirements (broad strokes)
A new
tests/e2e/harness that, per in-scope package:Completewithin a timeout,Promote the inline README example SCRs into runnable example CR files per package.
Image delivery (works on fork PRs)
The harness does not push to the org registry. It builds the package image locally and makes it available in the cluster somehow. Possibly by running a registry in the cluster. This runs entirely on the runner, so fork PRs, which get a read-only token and cannot push to
ghcr.io/nvidia/...(see #90), can still produce full e2e evidence with no secrets and no org package creation. Publishing stays a separate concern.Handling interrupts
Consider that many packages have workflows that require an interrupt to fire in order to fully test. This will need consideration as to either be deferred or somehow supported. Often interrupts are reboots but support could be staged to non-reboots (noop, service) and others.
Handling node prerequisites
Packages will likely need some fixtures to be setup in order to work. For example bind-mount expects a directory/fs to be mountable.
Package tiering
The following should at least be covered by the initial enhancement:
shellscript,copy-fail,bind-mount. Packages that are more complicated like nvidia-setup/nvidia-tuned could be deferred.Non-goals
Replacing the unit/fakes tests; GPU/hardware fidelity; publishing images from fork PRs.