Skip to content

Add test infrastructure and post-deploy validation stage to CDK pipeline#167

Open
StevenAskwith wants to merge 8 commits intoaws-solutions-library-samples:mainfrom
StevenAskwith:main
Open

Add test infrastructure and post-deploy validation stage to CDK pipeline#167
StevenAskwith wants to merge 8 commits intoaws-solutions-library-samples:mainfrom
StevenAskwith:main

Conversation

@StevenAskwith
Copy link
Contributor

Add test infrastructure and post-deploy validation stage to CDK pipeline

Wires up a complete test pipeline covering CDK, pre-deploy unit tests, and post-deploy live validation.

CDK / pipeline changes

  • Enable cdk-nag across the stack
  • DremWebsiteDistributionDomainName and appsyncId promoted to public readonly CfnOutput on DremAppStack and InfrastructurePipelineStage
  • PostDeployTests CodeBuild step added, sequenced after MainSiteDeployToS3, fetches live schema via appsync:GetIntrospectionSchema and runs npm run test:post-deploy

Test reporting

  • JUnit XML reports wired up for all CodeBuild steps, visible in the CodeBuild test report console
  • CDK test suite name fixed in jest-junit config (was reporting as undefined)

Pre-deploy tests

  • Migrated raceTableConfig and deviceTableConfig tests from Jest to Vitest for consistency
  • vitest/globals added to tsconfig.json types to fix tsc --noEmit failures on vi.* in test files

Post-deploy tests (16 new)

  • GraphQL schema conformance (Layer 1) — validates all .ts and .js queries, mutations and subscriptions against the live AppSync schema
  • Playwright smoke tests — HTTP 200, page title, #root element, env var sanity
  • Dedicated vitest.config.post-deploy.ts to prevent vitest exclude list blocking explicitly named test files

Documentation

  • docs/testing-strategy.md — full description of the test pipeline, what each layer covers, known gaps, and future directions
  • README.md — Testing section added with link to testing strategy

Total pipeline test count: 57

Stage Tests Count
Pre-deploy (CDK) CDK stack assertions 1
Pre-deploy (website) Component/unit tests 40
Post-deploy GraphQL schema conformance 12
Post-deploy Smoke (HTTP 200, title, #root, env var) 4
Total 57

- Uncomment cdk-nag in bin/drem.ts (global AwsSolutionsChecks)
- Add NagSuppressions to CdkPipelineStack for CDK Pipelines-managed
  resources (IAM5, CB4, S1, SNS3)
- Replace placeholder CDK test with real pipeline stack smoke test
- Add vitest + vitest.config.ts to website-leaderboard, wire test script,
  exclude App.test.tsx (needs jsdom/Amplify config, deferred)
- Update Makefile: add test.cdk, test.leaderboard targets, make test
  aggregate all three, fix duplicate manual.deploy.website target
- Add jest-junit reporter to root jest config (junit-cdk.xml)
- Add built-in junit reporter to website and website-leaderboard vitest configs
- Wire partialBuildSpec in pipeline synth step to publish all three report files
- Install jest-junit dev dependency
- Add vitest/globals to website/tsconfig.json types to fix tsc errors
  on vi.fn() in test files during Docker build step
- Add playwright dev dependency to website
- Add smoke.test.ts (Playwright) for post-deploy site render check
- Add test:post-deploy script running conformance + smoke tests with
  JUnit reporter output
- Exclude smoke.test.ts from normal vitest run (needs live URL)
- Promote dremWebsiteUrl and appsyncId to public readonly CfnOutputs
  on DeepracerEventManagerStack
- Add PostDeployTests CodeBuild step to pipeline post-deploy stage:
  fetches schema.graphql, runs conformance + smoke tests, publishes
  junit-post-deploy.xml report; depends on MainSiteDeployToS3
vitest exclude list blocks explicitly named files on CLI.
Add vitest.config.post-deploy.ts with no excludes and point
test:post-deploy script at it.
@StevenAskwith StevenAskwith requested a review from esbjj March 21, 2026 10:45
@StevenAskwith StevenAskwith self-assigned this Mar 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant