test(e2e): add workload manifest build flow

Signed-off-by: Evan Lezar <elezar@nvidia.com>

Author: elezar

e2e/gpu/README.md

@@ -3,12 +3,13 @@
 
 # GPU workload images
 
-This directory defines GPU workload images used by OpenShell GPU e2e tests.
+This directory defines workload test images currently used by the OpenShell GPU
+e2e suite.
 
 The image definitions live here first so the OpenShell e2e harness can iterate
-against a concrete contract. The long-term image ownership should move to
-`NVIDIA/OpenShell-Community`; OpenShell should then keep the contract, local
-build task, and tests that consume published image refs.
+against a concrete workload manifest. The long-term image ownership should move
+to `NVIDIA/OpenShell-Community`; OpenShell should then keep the manifest
+contract, local build task, and tests that consume published image refs.
 
 ## Contract
 
@@ -22,13 +23,21 @@ Each workload image must:
 - Print `OPENSHELL_GPU_WORKLOAD_SUCCESS` only when validation succeeds.
 - Print `OPENSHELL_GPU_WORKLOAD_FAILURE` and exit non-zero when validation
   fails.
-- Be usable as an OpenShell sandbox image with `openshell sandbox create
-  --from <image>`.
+- Be usable as an OpenShell sandbox image when OpenShell invokes
+  `/usr/local/bin/openshell-gpu-workload` explicitly.
 
 OpenShell sandbox creation replaces the image entrypoint with the supervisor and
 does not run the OCI image `CMD`. E2e tests that use these images through
 OpenShell should run `/usr/local/bin/openshell-gpu-workload` explicitly.
 
+The test harness is manifest-driven. Each workload entry carries:
+
+- `name`
+- `image`
+- `command`
+- `expect`
+- `requirements`
+
 ## Images
 
 | Source directory | Image name | Purpose |
@@ -42,14 +51,14 @@ OpenShell should run `/usr/local/bin/openshell-gpu-workload` explicitly.
 Build all workload images:
 
 ```shell
-mise run e2e:gpu:images:build
+mise run e2e:workloads:build
 ```
 
 Build a subset by source directory name:
 
 ```shell
 OPENSHELL_GPU_WORKLOAD_IMAGES=smoke-pass,smoke-fail \
-mise run e2e:gpu:images:build
+mise run e2e:workloads:build
 ```
 
 The build task uses `tasks/scripts/container-engine.sh`. Set
@@ -65,12 +74,26 @@ The task writes the latest build refs to:
 e2e/gpu/images/.build/latest.env
 ```
 
-Use it in later commands:
+The task also writes the local workload manifest used by the Rust e2e runner:
+
+```text
+e2e/gpu/images/.build/workloads.yaml
+```
+
+That local manifest is created by `mise run e2e:workloads:build`. It contains
+the full image reference, command, expected outcome, and requirements for each
+selected workload.
+
+Use the env file in later commands:
 
 ```shell
 source e2e/gpu/images/.build/latest.env
 ```
 
+That env file exports `OPENSHELL_E2E_WORKLOAD_MANIFEST` pointing at the local
+manifest. The per-image refs remain available as a convenience for direct
+container-engine validation.
+
 ## Direct Validation
 
 Validate smoke pass:
@@ -101,13 +124,72 @@ where Podman CDI is configured.
 Direct container-engine validation catches image, CDI, CUDA, and host GPU setup
 issues before OpenShell sandbox behavior is involved.
 
+## Manifest-Driven Validation
+
+The Rust GPU validation target is:
+
+```shell
+cargo test --manifest-path e2e/rust/Cargo.toml --features e2e-docker-gpu --test gpu -- --nocapture
+```
+
+The workload validation path reads:
+
+```text
+OPENSHELL_E2E_WORKLOAD_MANIFEST
+```
+
+When that variable is unset, the runner uses the default local manifest path:
+
+```text
+e2e/gpu/images/.build/workloads.yaml
+```
+
+If neither path exists, the workload validation test prints a clear skip
+message telling you to run:
+
+```shell
+mise run e2e:workloads:build
+```
+
+or to set `OPENSHELL_E2E_WORKLOAD_MANIFEST` to an external manifest.
+
+Each manifest entry supplies the sandbox image and command. OpenShell runs:
+
+```text
+/usr/local/bin/openshell-gpu-workload
+```
+
+through `openshell sandbox create --gpu --from <image> -- <command>`. The test
+runner iterates all GPU-tagged workload entries and enforces each entry's
+declared expectation:
+
+- `expect: pass` requires `OPENSHELL_GPU_WORKLOAD_SUCCESS`
+- `expect: fail` requires `OPENSHELL_GPU_WORKLOAD_FAILURE`
+
+The current local manifest includes three workloads:
+
+- `smoke-pass` expected to pass
+- `smoke-fail` expected to fail
+- `cuda-basic` expected to pass
+
+## External Manifests
+
+External workload catalogs can use the same schema. Point the runner at one
+with:
+
+```shell
+export OPENSHELL_E2E_WORKLOAD_MANIFEST=/abs/path/to/workloads.yaml
+```
+
+That lets partner-supplied or published workload manifests use the same test
+runner without introducing per-workload env vars.
 ## Publish Guidance
 
 Published tests should reference immutable image refs:
 
 ```shell
-OPENSHELL_E2E_GPU_CUDA_WORKLOAD_IMAGE=ghcr.io/nvidia/openshell-community/sandboxes/gpu-workload-cuda-basic@sha256:<digest>
+OPENSHELL_E2E_WORKLOAD_MANIFEST=/abs/path/to/published-workloads.yaml
 ```
 
-Mutable tags are acceptable for local iteration. CI should use a digest or an
-immutable release tag once the images are published from OpenShell-Community.
+The published manifest should use immutable image refs such as digests or
+release tags once the images are published from OpenShell-Community.

e2e/gpu/images/cuda-basic/README.md

@@ -20,7 +20,16 @@ pass. On failure it prints `OPENSHELL_GPU_WORKLOAD_FAILURE` and exits non-zero.
 Build it with:
 
 ```shell
-OPENSHELL_GPU_WORKLOAD_IMAGES=cuda-basic mise run e2e:gpu:images:build
+mise run e2e:workloads:build
+```
+
+That command also refreshes the local workload manifest at
+`e2e/gpu/images/.build/workloads.yaml`.
+
+To build only this workload locally, set:
+
+```shell
+OPENSHELL_GPU_WORKLOAD_IMAGES=cuda-basic mise run e2e:workloads:build
 ```
 
 Run it directly with Docker CDI:

e2e/gpu/images/smoke-fail/README.md

@@ -11,7 +11,16 @@ The workload does not perform GPU-specific work. It prints
 Build it with:
 
 ```shell
-OPENSHELL_GPU_WORKLOAD_IMAGES=smoke-fail mise run e2e:gpu:images:build
+mise run e2e:workloads:build
+```
+
+That command also refreshes the local workload manifest at
+`e2e/gpu/images/.build/workloads.yaml`.
+
+To build only this workload locally, set:
+
+```shell
+OPENSHELL_GPU_WORKLOAD_IMAGES=smoke-fail mise run e2e:workloads:build
 ```
 
 Run it directly:

e2e/gpu/images/smoke-pass/README.md

@@ -12,7 +12,16 @@ The workload does not perform GPU-specific work. It prints
 Build it with:
 
 ```shell
-OPENSHELL_GPU_WORKLOAD_IMAGES=smoke-pass mise run e2e:gpu:images:build
+mise run e2e:workloads:build
+```
+
+That command also refreshes the local workload manifest at
+`e2e/gpu/images/.build/workloads.yaml`.
+
+To build only this workload locally, set:
+
+```shell
+OPENSHELL_GPU_WORKLOAD_IMAGES=smoke-pass mise run e2e:workloads:build
 ```
 
 Run it directly:

tasks/scripts/e2e-gpu-build-images.sh

@@ -28,6 +28,16 @@ write_env_var() {
   printf 'export %s=%s\n' "${name}" "$(shell_quote "${value}")"
 }
 
+yaml_quote() {
+  local value=$1
+  value=${value//\\/\\\\}
+  value=${value//\"/\\\"}
+  value=${value//$'\n'/\\n}
+  value=${value//$'\r'/\\r}
+  value=${value//$'\t'/\\t}
+  printf '"%s"' "${value}"
+}
+
 available_image_dirs() {
   local dockerfile
   local preferred
@@ -69,6 +79,17 @@ image_env_var() {
   esac
 }
 
+image_expectation() {
+  case "$1" in
+    smoke-fail) echo "fail" ;;
+    smoke-pass|cuda-basic) echo "pass" ;;
+    *)
+      echo "unsupported GPU workload image source directory: $1" >&2
+      exit 1
+      ;;
+  esac
+}
+
 mapfile -t available < <(available_image_dirs)
 if [[ ${#available[@]} -eq 0 ]]; then
   echo "No GPU workload image Dockerfiles found under ${IMAGES_ROOT}" >&2
@@ -151,21 +172,44 @@ done
 
 mkdir -p "${BUILD_DIR}"
 latest_env="${BUILD_DIR}/latest.env"
+manifest_path="${BUILD_DIR}/workloads.yaml"
 {
-  echo "# Generated by mise run e2e:gpu:images:build"
+  echo "# Generated by mise run e2e:workloads:build"
   echo "# Source this file to use the most recently built GPU workload images."
   write_env_var OPENSHELL_GPU_WORKLOAD_IMAGE_TAG "${image_tag}"
   write_env_var OPENSHELL_GPU_WORKLOAD_IMAGE_SOURCE_PATH "${IMAGES_ROOT}"
   write_env_var OPENSHELL_GPU_WORKLOAD_IMAGE_SOURCE_SHA "${source_sha}"
   write_env_var OPENSHELL_GPU_WORKLOAD_IMAGE_SOURCE_DIRTY "${source_dirty}"
   write_env_var OPENSHELL_GPU_WORKLOAD_CONTAINER_ENGINE "${CONTAINER_ENGINE}"
+  write_env_var OPENSHELL_E2E_WORKLOAD_MANIFEST "${manifest_path}"
   for name in "${selected[@]}"; do
     write_env_var "$(image_env_var "${name}")" "${image_refs[${name}]}"
   done
 } > "${latest_env}"
 
+{
+  echo "schema_version: 1"
+  echo "generated_by: $(yaml_quote "mise run e2e:workloads:build")"
+  echo "source:"
+  echo "  path: $(yaml_quote "${IMAGES_ROOT}")"
+  echo "  revision: $(yaml_quote "${source_sha}")"
+  echo "  dirty: ${source_dirty}"
+  echo "  container_engine: $(yaml_quote "${CONTAINER_ENGINE}")"
+  echo "workloads:"
+  for name in "${selected[@]}"; do
+    echo "  - name: $(yaml_quote "${name}")"
+    echo "    image: $(yaml_quote "${image_refs[${name}]}" )"
+    echo "    command:"
+    echo "      - $(yaml_quote "/usr/local/bin/openshell-gpu-workload")"
+    echo "    expect: $(yaml_quote "$(image_expectation "${name}")")"
+    echo "    requirements:"
+    echo "      gpu: true"
+  done
+} > "${manifest_path}"
+
 echo
 echo "Wrote ${latest_env}"
+echo "Wrote ${manifest_path}"
 echo "Built images:"
 for name in "${selected[@]}"; do
   echo "  ${name}: ${image_refs[${name}]}"

tasks/test.toml

@@ -15,8 +15,8 @@ depends = ["e2e:rust", "e2e:python"]
 description = "Run Docker GPU end-to-end tests"
 depends = ["e2e:docker:gpu"]
 
-["e2e:gpu:images:build"]
-description = "Build local GPU workload images for e2e validation"
+["e2e:workloads:build"]
+description = "Build local workload test images and manifest for e2e validation"
 run = "bash tasks/scripts/e2e-gpu-build-images.sh"
 
 ["e2e:k3s:gpu"]