Releases: navapbc/template-infra
v0.16.0 ⚠️ Requires migration steps
Summary
Remove Pinpoint and use SES directly for email notifications. Better handle
non-us-east-1 project regions.
| Layer | Has changes | Needs migration |
|---|---|---|
| Account | ✅ | |
| Network | ✅ | |
| Build repository | ||
| Database | ||
| Service | ✅ | ✅ |
| CI/CD |
⚠️ Migration notes
AWS Pinpoint has been deprecated and can no longer be used for new projects. Existing projects can continue to use resources that have already been created. See #942 for more background.
Your applications utilizing the notifications setup from the infra template will need code changes to use SES directly.
If you want to test SES alongside your existing Pinpoint setup before fully migrating, you can adopt a gradual approach while pulling updates.
Phase 1: Add SES alongside Pinpoint (Optional)
-
Keep your Pinpoint resources - Don't delete
aws_pinpoint_app.appor
related resources yet -
Add the SES environment variable to your service configuration
infra/<app_name>/service/notifications.tf:notifications_environment_variables = local.notifications_config != null ? { # Existing Pinpoint variables AWS_PINPOINT_APP_ID = module.notifications[0].app_id # New SES variable (can construct inline to avoid output changes) AWS_SES_FROM_EMAIL = local.notifications_config.sender_display_name != null ? "${local.notifications_config.sender_display_name} <${local.notifications_config.sender_email}>" : local.notifications_config.sender_email } : {} -
Update your application code to use the sesv2 client instead of pinpoint,
referencingAWS_SES_FROM_EMAIL -
Test that SES email sending works in your environment
Phase 2: Remove Pinpoint
Once you've confirmed SES works:
- Apply the full migration by merging/pulling the changes from this release
- Run terraform apply to destroy the Pinpoint resources
- This approach minimizes risk by allowing you to validate SES functionality before removing Pinpoint infrastructure.
PR Envs
If any active PR environments exist from before you remove Pinpoint, you'll need
to take some additional steps. After updating your trunk branch with the release
changes, but before apply the account layer changes, ensure all PRs
merge/rebase on the lastest changes.
If you don't do this, you will need to manually clean up the PR environment
after the PR is closed. You can update bin/destroy-pr-environment as follows.
After:
echo "Select Terraform workspace: ${workspace}"
terraform -chdir="infra/${app_name}/service" workspace select "${workspace}"Add:
# TODO: tmp remove pinpoint state stuff so the rest of the removal can proceed
#
# Then go delete the actual resources via Console UI, Pinpoint > All projects > <project> > Settings > General Settings, "Delete project" button in upper right, enter "delete" then click "Delete" button
terraform -chdir="infra/${app_name}/service" state rm 'module.notifications[0].aws_pinpoint_email_channel.app'
terraform -chdir="infra/${app_name}/service" state rm 'module.notifications[0].aws_pinpoint_app.app'Then run the PR environment cleanup script locally. Then manually removed resources as indicated in the comment. Then discard the script changes.
Service
- Migrate notifications off of AWS Pinpoint by @sean-navapbc and @doshitan in #968, #981, and 7b26ec0
Network
- Better handle non-us-east-1 project regions by @sean-navapbc in #980
- This touches a few things across the stack, but mostly impacts the network
layer, ensuring DNS query logging CloudWatch resources remain in us-east-1
despite the default project region.
- This touches a few things across the stack, but mostly impacts the network
CI / CD
- Remove
typefrom custom GitHub action inputs by @doshitan in 01cd538 - Better match only
p-<num>format for orphaned PR environment detection by @doshitan in #994 - Parameterize service infra test code for container image tag by @doshitan in 85472c0
- Bump golang.org/x/crypto from 0.36.0 to 0.45.0 in /infra/test by @dependabot[bot] in #983
Tech debt and maintenance
The template has an AWS account cleanup script for itself, which you may find useful to reference if you are also doing extensive automated testing. Added by @sean-navapbc in #973.
Full Changelog: v0.15.7...v0.16.0
v0.15.7
Summary
Many CI/CD improvements, notably caching of container build layers, which should
result in a sizeable reduction in build time for images. And better
compatibility with automatic minor updates for the DB, by only specifying the
major version of the database.
| Layer | Has changes | Needs migration |
|---|---|---|
| Account | ||
| Network | ||
| Build repository | ||
| Database | ✅ | |
| Service | ||
| CI/CD | ✅ |
Database
- Better handling of DB auto minor updates by @lorenyu in #938 and @doshitan in #962
- If the template specifies a new minor version every time there's a release,
that forces projects to update in when doing a "simple" pull of template
changes. Projects wanting to pin to a particular version will need to
resolve that on template updates anyway, but for projects using auto updates
(the default), by only specifying the major version, less conflicts are
created on template updates and less juggling of state changes are necessary
for the projects, as AWS rolls out the auto updates on different schedules
for different accounts.
- If the template specifies a new minor version every time there's a release,
- Better handling of DB upgrades by @doshitan in #964
CI / CD
- Upgrade anchore-scan action by @lorenyu in #939
- Fix PR environment destroy race condition by @lorenyu in #945
- Use new version of markdown link check by @lorenyu in #955
- Build and deploys
Tech debt and maintenance
- Bump github.com/ulikunitz/xz from 0.5.12 to 0.5.14 in /infra/test by @dependabot[bot] in #953
- Don't eat non-zero exit in AWS
/binscripts by @doshitan in #958 - Docs
New Contributors
- @lisac made their first contribution in #960
- @sean-navapbc made their first contribution in #963
Full Changelog: v0.15.6...v0.15.7.
v0.15.6
Summary
This release brings configurable AWS Web Application Firewall (WAF) support,
redirects HTTP to HTTPS when HTTPS support is enabled in services, begins of
better support for services running multiple containers/sidecars, and notifies
of deploy failures (if the system notification setup has been configured).
| Layer | Has changes | Needs migration |
|---|---|---|
| Account | ||
| Network | ✅ | |
| Build repository | ||
| Database | ||
| Service | ✅ | |
| CI/CD | ✅ |
Migration notes
If you wish to use the WAF work in this release on your services, note the WAF
needs to first be created at the network layer before applying any changes at
the service layer, otherwise the service layer will not be able to find the WAF
to apply. To do this, apply the network changes by running:
make infra-update-network NETWORK_NAME=<NETWORK_NAME>You can then set enable_waf to true in app-config/main.tf and update its
infrastructure.
CI / CD
- Add deploy failure system notification by @lorenyu in #917
- Separate E2E format and typecheck jobs by @rylew1 in #914
- Add emojies to stale environment summaries for easier visual scanning by @lorenyu in 144b543
Network
- AWS Web Application Firewall (WAF) config by @lorenyu and @devin-ai-integration in #924, #926, #928, #933
- Default policy using some AWS managed rule sets for common threats
- See
docs/infra/web-application-firewall.mdfor more details
Service
- Support sidecars in run command by @coilysiren in #918
- Redirect HTTP traffic to HTTPS by @lorenyu in #925
- AWS Web Application Firewall (WAF) support by @lorenyu
- Disabled by default
Tech debt and maintenance
- Doc updates by @coilysiren and @lorenyu in #919 and a4d7eac
- Dependency updates for infrastructure tests by @dependabot and @doshitan in #922
- Add default issue template by @lorenyu in 6282c65
Full Changelog: v0.15.5...v0.15.6
v0.15.5
Summary
Automatically run E2E tests after a deployment and avoid throttling issues when
running commands.
| Layer | Has changes | Needs migration |
|---|---|---|
| Account | ||
| Network | ||
| Build repository | ||
| Database | ||
| Service | ✅ | |
| CI/CD | ✅ | |
| EOF |
Service
- Fix throttle issues when running commands (such as migrations) in projects with some history by @tdooner in #900
CI / CD
- E2E
- Cleanup E2E dependencies and strip
e2e-from npm script names by @lorenyu in #895 - Add autoformatting and typechecking by @lorenyu and @rylew1 in #896, #911
- Support specifiying a specific email username for tests that need it by @lorenyu in #898
- Consolidate ability to run E2E tests on demand into
e2e-tests.ymlby @lorenyu in #899 - Ignore HTTPS errors by default on E2E tests by @lorenyu in #901
- Run E2E tests at end of deployment except in
prodby @lorenyu in #903 and #908
- Cleanup E2E dependencies and strip
- Check for stale infra service test environments by @lorenyu in #909
- Bump golang.org/x/net from 0.34.0 to 0.36.0 in /infra/test by @dependabot in #905
- Allow PR environment workflows to run in parallel for different apps by @lorenyu in a78dc50
Tech debt and maintenance
Full Changelog: v0.15.4...v0.15.5
v0.15.4
Summary
Re-add feature flags, streamline E2E test development, and refactor how some
things are configured.
| Layer | Has changes | Needs migration |
|---|---|---|
| Account | ||
| Network | ||
| Build repository | ||
| Database | ||
| Service | ✅ | |
| CI/CD | ✅ |
Migration notes
For secrets with manage_method="generated", if you want to avoid regenerating
the secrets, you can move the secrets to the new location with:
terraform -chdir=infra/<APP_NAME>/service state mv 'module.secrets["<SECRET_NAME>"].aws_ssm_parameter.secret[0]' 'module.secrets.aws_ssm_parameter.secrets["<SECRET_NAME>"]'
terraform -chdir=infra/<APP_NAME>/service state mv 'module.secrets["<SECRET_NAME>"].random_password.secret[0]' 'module.secrets.random_password.secrets["<SECRET_NAME>"]' Service
- Add SSM-based feature flags by @lorenyu in #873 and #883
- Convert secret module to create multiple secrets by @lorenyu in #877
- Add ability to run task commands in PR environment by @lorenyu in #879
- Add workflow to run E2E tests on demand by @lorenyu in #892 and c0c9cec
Tech debt and maintenance
- Generate infra service test id from timestamp by @lorenyu in #885
- Move monitoring config out of service root module by @lorenyu in #884
- Fix service module public_endpoint output when https enabled by @lorenyu in #893
- Docs
- Fix typo in variable description by @coilysiren in #881
- Fix link to project-config by @lorenyu in #882
- E2E testing refactors
- bin/orphaned-pr-environments: Handle no PR envs and merged PRs better by @doshitan in #889
Full Changelog: v0.15.3...v0.15.4
v0.15.3
Summary
Fix and improve email notifications, support writeable filesystem locations for
services, more flexible service healthchecks, and continue to shuffle some
configuration things around slightly for simpler inferfaces and better
separation of concerns.
| Layer | Has changes | Needs migration |
|---|---|---|
| Account | ||
| Network | ||
| Build repository | ||
| Database | ✅ | |
| Service | ✅ | |
| CI/CD |
Service
- Fix MAIL FROM and add DMARC DNS Records by @coilysiren in #859
- Support ephemeral write mounts for services by @doshitan in #871
- Use service provided healthcheck, with fallbacks by @doshitan in #870
Documentation
- Add comment describing each AWS service by @lorenyu in #866
- Update module dependencies documentation by @lorenyu in #865
- Add ADR for notifications by @lorenyu in #830
Tech debt and maintenance
- Include prefix in service root
service_nameoutput by @doshitan in #869 - Move service config to separate file by @lorenyu in #872
- Nest network and within service & database modules by @coilysiren in #861
- infra/test: Update dependencies by @doshitan in #867
- Replace deprecated
managed_policy_arnswithaws_iam_role_policy_attachmentby @coilysiren in #862
Full Changelog: v0.15.2...v0.15.3
v0.15.2 ⚠️ Requires migration steps
Summary
Fix a race condition in CI/CD, enable services to use the notifications setup directly, update Terraform to 1.10.x series, and a variety of smaller cleanups.
| Layer | Has changes | Needs migration |
|---|---|---|
| Account | ||
| Network | ||
| Build repository | ||
| Database | ||
| Service | ✅ | |
| CI/CD | ✅ |
⚠️ Migration notes
The required Terraform version has changed from 1.8.x to 1.10.x. CI/CD actions should handle this automatically, but you'll need to update your local Terraform.
Minor callouts
- While there are no changes in functionality, the service/network/domain/database Terraform module reorg may require additional attention if you've customized those modules for your project.
- The ADR doc naming convention has shifted to use a date prefix, it's suggested to move any of your project-specific entries to match. At the very least you may need to re-number yours.
- Some test projects non-uniformly encountered a few oddities when running the update, so just be on the lookout for:
- A deleted
infra/modules/database/resources/role_manager.zip, restore with:(or copy fromgit checkout origin/main -- infra/modules/database/resources/role_manager.ziptemplate-infrarepo) - The old
docs/decisions/infra/ADRs being left around, manually remove those with something like:git rm docs/decisions/infra/0*
- A deleted
Service
Notifications
CI/CD
- Fix race condition often causing failures on merges to
main, by @doshitan and @lorenyu in #848 and #849 - bin/lint-markdown: When a link check returns HTTP 429 "Too Many Requests", wait and retry instead of failing by @lorenyu in #858
- bin/lint-markdown: Only run against git-tracked markdown files by @doshitan in #780
- Show app name in PR environment destroyed message by @lorenyu in af9e30a
Documentation
Tech debt and maintenance
- No functional changes, but various Terraform modules have been internally reorganized for a simpler interface and cleaner separation of concerns:
- Use a date instead of incrementing number for ADR file names by @daphnegold in #851
- Upgrade Terraform version from 1.8.3 to 1.10.5 by @daphnegold in #852
- Remove
otherfile.txtandsomefile.txtby @lorenyu in 5a00683
Full Changelog: v0.15.1...v0.15.2
v0.15.1
Summary
Includes a change for notifications functionality that simplifies the code and fixes a conflict under certain project configurations.
| Layer | Has changes | Needs migration |
|---|---|---|
| Account | ||
| Network | ||
| Build repository | ||
| Database | ||
| Service | ||
| CI/CD | ✅ |
What's Changed
- 🔧 Move prefix from env-config to service root by @lorenyu in #832
- 🐞 Notifications: Reuse app domain for domain identity by @lorenyu in #834
Full Changelog: v0.15.0...v0.15.1
v0.15.0 ⚠️ Requires migration steps
Summary
This release switches to using the Platform CLI tool for managing the template. There should be no breaking changes to functionality, but the switch involves a breaking change to how the template is installed/updated that will need manual migration.
| Layer | Has changes | Needs migration |
|---|---|---|
| Account | ||
| Network | ||
| Build repository | ||
| Database | ||
| Service | ||
| CI/CD | ✅ |
⚠️ Migration notes
See https://navapbc.github.io/platform-cli/getting-started/migrating-from-legacy-template/ for guidance on updating to this version using the Platform CLI.
There are a lot of little documentation tweaks to better support multi-app environments. So diff might look large at first glance.
What's Changed
- Support Platform CLI by @lorenyu and @doshitan in #763 includes a fair amount of changes. Notable ones being:
- Broader multi-app support across all functionality, but in particular new multi-app support in:
- PR Environments
- Infra tests/CI
- Move example/demo/minimal application
app/totemplate-only-app/. Places wanting to use it as their example app for testing can copytemplate-only-app/to the appropriate app-named directory.
- Broader multi-app support across all functionality, but in particular new multi-app support in:
Full Changelog: v0.14.3...v0.15.0
v0.14.3
Summary
Improvements around PR environments and consistency naming terraform modules.
| Layer | Has changes | Needs migration |
|---|---|---|
| Account | ||
| Network | ||
| Build repository | ||
| Database | ||
| Service | ||
| CI/CD | ✅ |
What's Changed
- ⚙️ Send notification for failed infra deploy status check by @lorenyu in #823
- 🐞 Fix trigger for PR environment destroy workflow by @lorenyu in #827
- 🐞 Fix permissions for scan orphaned PRs workflow by @lorenyu in #829
- 🔧 Use underscores in Terraform file names instead of dashes by @daphnegold in #828
Template development
- Add template release management documentation by @lorenyu in #825
- Add template technical design principles by @lorenyu in #824
Full Changelog: v0.14.2...v0.14.3