Summary
helm/values-test-full.yaml deliberately omits DependencyTrack with the comment "requires complex bootstrapping, handled separately". The security-tests suite includes a DTrack integration subsuite that correctly hard-fails under RELEASE_GATE=1:
FAIL: skip_suite called with RELEASE_GATE=1
(reason: DEPENDENCY_TRACK_API_KEY and/or DEPENDENCY_TRACK_URL not set;
DTrack integration not exercised)
a skipped suite in release-gate is silent-success; failing the gate
The fail-on-skip behavior is correct -- it catches the "silent-success class" that release-gate exists to prevent.
Why we shouldn't just remove the subsuite
DTrack is not optional surface in the backend. It's a 3,190 LoC integration:
backend/src/services/dependency_track_service.rs -- 2,489 lines
backend/src/api/handlers/dependency_track.rs -- 701 lines
artifact-keeper-iac/charts/artifact-keeper/templates/dtrack-deployment.yaml -- 323-line subchart
- Default
dependencyTrack.enabled: true in charts/artifact-keeper/values.yaml:494
Removing the subsuite would leave the entire DTrack integration with zero release-gate coverage.
Fix: stand up DTrack in the security-tests deploy
Infrastructure already exists in this repo for the clean-install-smoke-with-deps job:
helm/values-smoke-with-deps.yaml:159-167 enables DTrack with right-sized resources (2 CPU / 4 Gi).
- The chart's DTrack subchart includes a bootstrap init container that creates the admin API key.
Work needed:
- values-test-full.yaml -- enable
dependencyTrack.enabled: true with the same resource limits as values-smoke-with-deps.yaml.
- scripts/create-test-namespace.sh -- after install completes, surface the bootstrap-generated API key from the K8s Secret (whatever name the subchart uses) and the DTrack service URL.
- .github/workflows/release-gate.yml (security-tests job, around lines 949-994) -- export
DEPENDENCY_TRACK_URL=http://artifact-keeper-dtrack.${NAMESPACE}.svc.cluster.local:8080 and DEPENDENCY_TRACK_API_KEY=<from secret> into the job env.
- Namespace ResourceQuota -- current ARC quota is 4 CPU / 8 Gi; security stack with DTrack pushes to ~7.5 CPU / 12.4 Gi. Either raise the namespace quota or route this job to the
ak-beefy-runners pool referenced in values-smoke-with-deps.yaml:48.
Effort and risk
- ~half day to wire and test
- Risk: capacity headroom on ARC runners. Mitigation listed under (4).
Acceptance
security-tests DTrack subsuite passes under RELEASE_GATE=1, and the rest of security-tests remains green (no regression from added DTrack startup time).
Summary
helm/values-test-full.yamldeliberately omits DependencyTrack with the comment "requires complex bootstrapping, handled separately". Thesecurity-testssuite includes a DTrack integration subsuite that correctly hard-fails underRELEASE_GATE=1:The fail-on-skip behavior is correct -- it catches the "silent-success class" that release-gate exists to prevent.
Why we shouldn't just remove the subsuite
DTrack is not optional surface in the backend. It's a 3,190 LoC integration:
backend/src/services/dependency_track_service.rs-- 2,489 linesbackend/src/api/handlers/dependency_track.rs-- 701 linesartifact-keeper-iac/charts/artifact-keeper/templates/dtrack-deployment.yaml-- 323-line subchartdependencyTrack.enabled: trueincharts/artifact-keeper/values.yaml:494Removing the subsuite would leave the entire DTrack integration with zero release-gate coverage.
Fix: stand up DTrack in the security-tests deploy
Infrastructure already exists in this repo for the
clean-install-smoke-with-depsjob:helm/values-smoke-with-deps.yaml:159-167enables DTrack with right-sized resources (2 CPU / 4 Gi).Work needed:
dependencyTrack.enabled: truewith the same resource limits asvalues-smoke-with-deps.yaml.DEPENDENCY_TRACK_URL=http://artifact-keeper-dtrack.${NAMESPACE}.svc.cluster.local:8080andDEPENDENCY_TRACK_API_KEY=<from secret>into the job env.ak-beefy-runnerspool referenced invalues-smoke-with-deps.yaml:48.Effort and risk
Acceptance
security-testsDTrack subsuite passes under RELEASE_GATE=1, and the rest ofsecurity-testsremains green (no regression from added DTrack startup time).