infra/ci: archive logs to S3, per-env tfvars, SBOM+cosign signing, Docker layer caching - #1183
Merged
Mystery-CLI merged 5 commits intoAug 26, 2026
Conversation
Add a Kinesis Firehose subscription-filter pipeline that streams aws_cloudwatch_log_group.backend into a new encrypted, versioned S3 bucket, since the log group's flat 30-day retention previously made logs unrecoverable past that window — a problem for incident postmortems, audits, and financial recordkeeping requirements. - infra/log-archival.tf: S3 archive bucket (versioning + AES-256 SSE + public access block), lifecycle policy transitioning objects to Glacier then Glacier Deep Archive before expiring, a Kinesis Firehose delivery stream, and the IAM roles/policies wiring the backend log group's subscription filter to Firehose to S3. - infra/variables.tf: add log_archive_glacier_transition_days, log_archive_deep_archive_transition_days, and log_archive_expiration_days. - infra/outputs.tf: add log_archive_bucket output. - infra/README.md: document the archival pipeline, lifecycle schedule, and the retrieval steps for pulling archived logs during an incident investigation. Closes Ethereal-Future#1006
… and production
Add infra/environments/{staging,production}.tfvars so environment
sizing is defined in one place instead of relying on shared defaults
plus ad-hoc -var flags, which risked staging accidentally running at
production-sized (or vice versa) instance classes and task counts.
- infra/environments/staging.tfvars: environment = "staging" with
lighter-weight db_instance_class (db.t4g.micro), redis_node_type
(cache.t4g.micro), backend_cpu/memory, and backend_desired_count = 1.
- infra/environments/production.tfvars: makes the current
production-sized defaults (db.t4g.small, cache.t4g.small,
backend_desired_count = 2) explicit.
- Neither file commits secret-bearing values — backend_image,
frontend_image, and credentials are still passed via -var/CI
secrets at plan/apply time.
- .github/workflows/terraform-plan.yml: select
environments/production.tfvars for PRs targeting main and
environments/staging.tfvars otherwise, so posted plans reflect
real per-environment sizing.
- infra/README.md: update plan/apply examples to use -var-file, and
add a table summarizing the two environment files.
Closes Ethereal-Future#1005
docker-scan.yml built both images from scratch on every run (plain `docker build`, no cache-from/cache-to), so the multi-stage npm ci layer in both Dockerfiles was invalidated on every PR and the nightly cron scan even when dependencies hadn't changed. - .github/workflows/docker-scan.yml: add docker/setup-buildx-action and replace the raw `docker build` steps with docker/build-push-action (push: false, load: true) using cache-from/cache-to: type=gha (scoped per image) so Trivy still scans the same locally-loaded image tag as before. - backend/Dockerfile, frontend/Dockerfile: add a BuildKit `# syntax=docker/dockerfile:1` pragma and `RUN --mount=type=cache,target=/root/.npm` on the npm ci dependency-install layers so npm's cache persists across builds. Closes Ethereal-Future#1003
docker-scan.yml scanned images for CVEs but produced no component inventory and had no way to cryptographically verify an image's provenance before it's deployed via infra/ecs.tf — a gap for a platform handling financial transactions. - .github/workflows/docker-scan.yml: generate a CycloneDX SBOM for both the backend and frontend images via anchore/sbom-action and upload it as a workflow artifact alongside the existing Trivy SARIF reports. - For trusted (non-fork) events, push a scan-<sha> tagged copy of each successfully-scanned image to GHCR and sign it keylessly with sigstore/cosign-installer using GitHub OIDC — no long-lived signing keys are added as secrets. Fork pull_request events skip the push/sign steps since they can't authenticate to GHCR, but still get SBOM generation and CVE scanning. - Add packages: write and id-token: write permissions needed for the registry push and keyless OIDC signing. - infra/README.md: document how a deployer verifies an image's signature (cosign verify) and inspects its SBOM before running terraform apply, plus a follow-up note that enforcing signature verification at ECS deploy time is out of scope for this change. Closes Ethereal-Future#1004
|
@miracle605 Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits. You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR bundles four infra/CI hardening tasks for the FuTuRe platform:
Archive ECS CloudWatch logs to S3 for long-term retention #1006 — Archive ECS CloudWatch logs to S3 for long-term retention
infra/log-archival.tf: new encrypted, versioned S3 bucket (aws_s3_bucket.log_archive) fed by a Kinesis Firehose delivery stream subscribed toaws_cloudwatch_log_group.backendvia a CloudWatch Logs subscription filter, plus the IAM roles/policies wiring it together.infra/variables.tfvariables.infra/outputs.tf: newlog_archive_bucketoutput.infra/README.md: documents the archival pipeline and how an operator retrieves archived logs for an incident investigation.Add per-environment Terraform variable files for staging and production #1005 — Add per-environment Terraform variable files for staging and production
infra/environments/staging.tfvarsandinfra/environments/production.tfvarscapture environment-appropriate sizing (RDS/Redis instance classes, backend task count/CPU/memory) — no secret values committed..github/workflows/terraform-plan.ymlnow selectsenvironments/production.tfvarsfor PRs targetingmainandenvironments/staging.tfvarsotherwise.infra/README.md: plan/apply examples updated to use-var-file, plus a summary table of the two environment files.Generate SBOMs and sign container images in CI #1004 — Generate SBOMs and sign container images in CI
.github/workflows/docker-scan.yml: generates a CycloneDX SBOM for both images (viaanchore/sbom-action) and uploads it as a workflow artifact.scan-<sha>tagged copy of each successfully-scanned image to GHCR and signs it keylessly viasigstore/cosign-installer+ GitHub OIDC — no long-lived signing keys stored as secrets.infra/README.md: documentscosign verifyusage and SBOM inspection beforeterraform apply, with a noted follow-up to enforce verification at deploy time.Add Docker layer/BuildKit caching to CI image build workflows #1003 — Add Docker layer/BuildKit caching to CI image build workflows
.github/workflows/docker-scan.yml: replaces rawdocker buildwithdocker/build-push-actionusingcache-from/cache-to: type=gha(scoped per image), so Trivy scanning and the new SBOM/signing steps reuse the same cached, locally-loaded image.backend/Dockerfile/frontend/Dockerfile: add a BuildKit# syntax=docker/dockerfile:1pragma andRUN --mount=type=cache,target=/root/.npmon thenpm cilayers so npm's cache persists across builds.Test plan
terraform validate/terraform plan -var-file=environments/staging.tfvarsand-var-file=environments/production.tfvarsboth succeedterraform planshows the new S3 bucket, lifecycle policy, and Firehose/log-subscription resources applying without errordocker-scan.ymlrun uploads SBOM artifacts for both images and (on non-fork events) signs both images, verifiable withcosign verifydocker-scan.ymlrun with no dependency changes shows cache hits and a faster build stepCloses #1003
Closes #1004
Closes #1005
Closes #1006