Harden C3 readiness and smoke-test guidance for generated runtime scans

[DovOps]

Context

During a local Vigolium experiment against C3 (012-platform-convergence-c3), the generated Kubernetes runtime could start successfully, but the current smoke-test path exposed a few hardening gaps that make scanner/test automation noisy or racy.

The experiment used the generated C3 start script with published images:

TRADERX_SKIP_GENERATE=1 \
TRADERX_USE_PUBLISHED_IMAGES=1 \
TRADERX_PUBLISHED_NAMESPACE=traderx-c3 \
TRADERX_PUBLISHED_TAG=latest \
./scripts/start-state-012-platform-convergence-c3-generated.sh \
  --skip-build \
  --cluster-name traderx-state-012

Observed behavior

• Kubernetes marked deployments available before several Spring Boot services were actually accepting ingress traffic.
• Early smoke-test probes returned 502 Bad Gateway for paths such as:
  • http://localhost:8080/account-service/account/22214
  • http://localhost:8080/order-matcher/health
• The same paths became healthy after the JVM services completed startup.
• On Apple Silicon, the published images ran under qemu-x86_64; Spring services took roughly 96-126 seconds to report application readiness.
• The state smoke later reached the deployment-hardening check and failed because http://localhost:8080/@vite/client returned the Angular SPA fallback (200) instead of being rejected.

Why this matters

For smoke tests, scanner experiments, and future security automation, deployment availability is not a strong enough readiness signal. Automation needs to wait for application-level readiness across the ingress surface before it starts checking behavior or running active scanners.

Without that, tests can fail for transient startup timing instead of real regressions, especially on Apple Silicon where published amd64 images start slowly under emulation.

Proposed hardening work

• Add or standardize a generated runtime readiness gate that waits for application-level readiness before smoke tests run.
• Prefer explicit service readiness endpoints where available, for example:
  • account-service/actuator/health
  • position-service/health/ready and/or actuator health
  • trade-service/actuator/health
  • trade-processor/actuator/health
  • order-matcher/health and/or actuator health
  • reference-data/health
  • UI root and edge /health
• Update C3/state 010-012 start or smoke scripts so kubectl rollout status is not treated as sufficient by itself.
• Add retry/backoff guidance with a longer timeout budget for Apple Silicon when using published amd64 images.
• Document recommended Apple Silicon workflows:
  • expected slower startup when running ghcr.io/finos/traderx-c3:* amd64 images under emulation
  • when to prefer local native image builds vs published images
  • suggested minimum wait/readiness timeout before smoke tests and scanners
• Harden SPA/edge routing so known dev-only paths such as /@vite/client are not served by the deployed Angular SPA fallback.
• Consider adding a scanner-preflight command or doc section that confirms the ingress is warm before running tools such as Vigolium.

Acceptance criteria

• A C3 smoke run waits for application-level readiness and does not fail with transient 502s immediately after deployment rollout.
• Readiness checks cover the services used by the smoke suite, not only Kubernetes deployment availability.
• Apple Silicon guidance exists for C3 generated runtime testing, including published-image emulation caveats and timeout expectations.
• The deployed UI/runtime rejects or otherwise does not serve dev-only paths such as /@vite/client as a successful SPA route.
• The behavior is documented near the generated runtime/smoke-test entrypoints so future scanner experiments have a repeatable preflight.

Experiment notes

A local experiment branch/worktree captured the commands and Vigolium scan notes:

• Worktree: /Users/dovkatz/Dev/traderx-learning/worktrees/traderX-vigolium-c3
• Branch: codex/vigolium-c3-experiment
• Notes file: experiments/vigolium-c3/README.md

Key Vigolium result from the bounded ingress scan:

• 26 HTTP records
• 17 findings after grouping
• Finding themes included exposed actuator/docs/metrics endpoints, missing policy/security headers, nginx path behavior, and ID-like parameters.

This issue is specifically about runtime readiness and hardening surfaced by that experiment, not necessarily triaging every scanner finding.