first iteration of NRI tests

docs/nri-spec-discrepancies.md

@@ -0,0 +1,82 @@
+# NRI Spec Discrepancies
+
+This document tracks cases where observed runtime behavior differs from the
+NRI Pod Sandbox Lifecycle specification (containerd/nri#286).
+
+## StopPodSandbox idempotency — sandbox reuse after Stop
+
+**Spec requirement** (from containerd/nri#286): After StopPodSandbox, the
+sandbox is never reused — CreateContainer on a stopped sandbox MUST fail.
+
+**Observed behavior**: containerd allows CreateContainer to succeed on a
+stopped sandbox, returning a valid container ID instead of an error.
+
+**Contract points in this scenario:**
+
+- [x] StopPodSandbox is idempotent (multiple calls succeed without error)
+- [x] StopPodSandbox NRI hook fires at least once
+- [ ] CreateContainer on a stopped sandbox fails ← discrepancy
+
+**Impact**: A plugin that assumes stopped sandboxes are immutable may see
+unexpected container lifecycle events after StopPodSandbox.
+
+**Status**: Open
+
+## Multi-plugin teardown hook delivery after plugin failure
+
+**Spec requirement** (from containerd/nri#286): One plugin's failure on
+StopPodSandbox/RemovePodSandbox MUST NOT prevent delivery of those hooks to
+subsequent plugins.
+
+**Observed behavior**: NRI aborts hook delivery to remaining plugins when one
+plugin returns an error. Plugin 1 (higher index) receives zero
+StopPodSandbox/RemovePodSandbox hooks when plugin 0 (lower index) returns an
+error.
+
+**Contract points in this scenario:**
+
+- [x] Plugin 0 receives StopPodSandbox hook
+- [x] Plugin 0 receives RemovePodSandbox hook
+- [ ] Plugin 1 receives StopPodSandbox hook despite plugin 0's error ← discrepancy
+- [ ] Plugin 1 receives RemovePodSandbox hook despite plugin 0's error ← discrepancy
+- [x] StopPodSandbox CRI call succeeds despite plugin error
+- [x] RemovePodSandbox CRI call succeeds despite plugin error
+- [x] Sandbox is fully removed
+
+**Impact**: A lower-index plugin returning an error on teardown silently
+prevents all higher-index plugins from receiving teardown notifications,
+potentially causing resource leaks in those plugins.
+
+**Status**: Open
+
+## containerd: NRI plugin errors swallowed on StopPodSandbox CRI call
+
+**Spec requirement**: StopPodSandbox and RemovePodSandbox CRI calls MUST
+propagate NRI plugin errors to the CRI caller so the caller is aware of
+the failure.
+
+**Observed behavior** (containerd): When an NRI plugin returns an error on
+StopPodSandbox or RemovePodSandbox, containerd swallows the error and
+returns success to the CRI caller. CRI-O correctly propagates the error.
+
+**Affected runtimes**: containerd (CRI-O handles this correctly)
+
+**Impact**: The CRI caller is unaware that an NRI plugin failed during
+teardown, potentially masking resource leaks or plugin state corruption.
+
+**Status**: Open
+
+## CRI-O: empty container name in NRI CreateContainer event metadata
+
+**Spec requirement**: NRI CreateContainer events should include the container
+name in the event metadata so plugins can identify containers.
+
+**Observed behavior** (CRI-O): The `ContainerName` field in NRI
+CreateContainer events is an empty string.
+
+**Affected runtimes**: CRI-O (containerd populates this correctly)
+
+**Impact**: Plugins that rely on container name for policy decisions cannot
+identify containers during CreateContainer hooks on CRI-O.
+
+**Status**: Open

images/containerd-local-test/entrypoint.sh

@@ -42,16 +42,65 @@ CONTAINERD_PID=$!
 # Wait for containerd to be ready
 wait-for-containerd.sh
 
+# Trigger the default sandbox creation by listing pods, then capture the mount
+# baseline. containerd lazily creates its sandbox shim on the first CRI call,
+# so we need to trigger it before baselining.
+crictl --runtime-endpoint unix:///run/containerd/containerd.sock pods > /dev/null 2>&1 || true
+sleep 1
+MOUNT_BASELINE=$(mount | grep -E "containerd.*(shm|rootfs)" || true)
+
 # Execute the command passed to the container. Capture the exit code without
 # tripping `set -e` so the containerd cleanup below always runs even when the
 # test command fails (mirrors CI's `|| TEST_RC=$?` pattern).
 echo "Executing: $@"
 EXIT_CODE=0
 "$@" || EXIT_CODE=$?
 
+# Post-test leak detection.
+# Leaked sandboxes or orphaned mounts indicate a test bug — every test must
+# clean up its sandboxes and containers. If a leak is detected here, the
+# test run is marked as failed even if all tests passed, because leaked
+# resources cause "Device or resource busy" errors in CI cleanup.
+LEAK_DETECTED=0
+
+echo "Checking for leaked sandboxes..."
+LEAKED_SANDBOXES=$(crictl --runtime-endpoint unix:///run/containerd/containerd.sock pods 2>/dev/null || true)
+SANDBOX_COUNT=0
+if echo "$LEAKED_SANDBOXES" | grep -q "POD ID"; then
+    SANDBOX_COUNT=$(echo "$LEAKED_SANDBOXES" | tail -n +2 | grep -c . || true)
+fi
+
+if [ "$SANDBOX_COUNT" -gt 0 ]; then
+    echo "FAIL: $SANDBOX_COUNT sandbox(es) leaked after tests — test cleanup is broken"
+    echo "$LEAKED_SANDBOXES"
+    LEAK_DETECTED=1
+else
+    echo "No leaked sandboxes."
+fi
+
+echo "Checking for orphaned mounts..."
+CURRENT_MOUNTS=$(mount | grep -E "containerd.*(shm|rootfs)" || true)
+# Compare against baseline to find mounts created during the test run.
+NEW_MOUNTS=$(diff <(echo "$MOUNT_BASELINE") <(echo "$CURRENT_MOUNTS") | grep "^>" | sed 's/^> //' || true)
+if [ -n "$NEW_MOUNTS" ]; then
+    echo "FAIL: new containerd mounts appeared during tests — sandbox teardown is incomplete"
+    echo "$NEW_MOUNTS"
+    echo "Dumping containerd log for mount correlation:"
+    cat /var/log/containerd.log
+    LEAK_DETECTED=1
+else
+    echo "No orphaned mounts."
+fi
+
 # Cleanup containerd
 echo "Cleaning up containerd (PID: $CONTAINERD_PID)..."
 kill $CONTAINERD_PID
 wait $CONTAINERD_PID || true
 
+# Fail the run if tests passed but resources leaked.
+if [ "$EXIT_CODE" -eq 0 ] && [ "$LEAK_DETECTED" -eq 1 ]; then
+    echo "Tests passed but leaked resources detected — marking run as failed."
+    exit 1
+fi
+
 exit $EXIT_CODE