Document runtime configuration for DC and CO #67
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,80 @@ | ||
| # CO (Colorado) — Runtime Configuration | ||
|
|
||
| This document covers every value that the running CO application consumes at runtime. Build-only and infrastructure-provisioning values (AWS credentials, VPC CIDRs, container sizing, etc.) are omitted. | ||
|
|
||
| CO uses OIDC for authentication rather than email-based OTP. The Tofu modules still provision SES resources and inject SMTP credentials into the container, but they are not used for login in this state. | ||
|
|
||
| ## All Runtime Values | ||
|
|
||
| | # | Name | Secret? | Current Value | Description | | ||
| |---|---|---|---|---| | ||
| | | **API Container — Environment Variables** | | | | | ||
| | 1 | `ASPNETCORE_ENVIRONMENT` | No | `dev` | .NET hosting environment | | ||
| | 2 | `DB_HOST` | No | *(RDS endpoint)* | Database hostname (auto-generated by Tofu from RDS) | | ||
| | 3 | `DB_NAME` | No | `SebtPortal` | Database name | | ||
| | 4 | `DB_PORT` | No | `1433` | SQL Server port | | ||
| | 5 | `EmailOtpSenderServiceSettings__SenderEmail` | No | `noreply@dev.co.sebt-portal.codeforamerica.app` | "From" address on emails (provisioned but unused — CO uses OIDC) | | ||
| | 6 | `PluginAssemblyPaths__0` | No | `plugins-co` | Directory containing state plugin DLLs | | ||
| | 7 | `Seeding__EmailPattern` | No | `sebt.co+{0}@codeforamerica.org` | Format string for seed user emails | | ||
| | 8 | `Seeding__Enabled` | No | `true` | Whether database seeding runs on startup | | ||
| | 9 | `SmtpClientSettings__EnableSsl` | No | `true` | Require TLS for SMTP (provisioned but unused — CO uses OIDC) | | ||
| | 10 | `SmtpClientSettings__SmtpPort` | No | `587` | SMTP port (provisioned but unused — CO uses OIDC) | | ||
| | 11 | `SmtpClientSettings__SmtpServer` | No | *(SES SMTP endpoint)* | SMTP server (provisioned but unused — CO uses OIDC) | | ||
| | 12 | `STATE` | No | `co` | State code; selects plugin directory and config overlay | | ||
| | | **API Container — Secrets (from AWS Secrets Manager)** | | | | | ||
| | 13 | `DB_PASSWORD` | Yes | *** | Database password (auto-generated by RDS) | | ||
| | 14 | `DB_USER` | Yes | *** | Database username (auto-generated by RDS) | | ||
| | 15 | `IdentifierHasher__SecretKey` | Yes | *** | Key for hashing PII identifiers | | ||
| | 16 | `JwtSettings__SecretKey` | Yes | *** | JWT signing key | | ||
| | 17 | `SmtpClientSettings__Password` | Yes | *** | SES SMTP password (provisioned but unused — CO uses OIDC) | | ||
| | 18 | `SmtpClientSettings__UserName` | Yes | *** | SES SMTP username (provisioned but unused — CO uses OIDC) | | ||
| | | **Web Container — Environment Variables** | | | | | ||
| | 19 | `BACKEND_URL` | No | *(internal ALB URL)* | Server-side API URL (internal to VPC) | | ||
| | 20 | `NEXT_PUBLIC_API_BASE_URL` | No | *(internal ALB URL)* | API URL exposed to browser JS | | ||
| | 21 | `NEXT_PUBLIC_STATE` | No | `co` | State code exposed to browser JS | | ||
| | 22 | `STATE` | No | `co` | State code | | ||
| | | **Application Defaults (from `appsettings.json`, overridable at runtime)** | | | | | ||
| | 23 | `EnrollmentCheckRateLimitSettings:PermitLimit` | No | `10` | Max enrollment checks per rate limit window | | ||
| | 24 | `EnrollmentCheckRateLimitSettings:WindowMinutes` | No | `1.0` | Enrollment check rate limit window (minutes) | | ||
| | 25 | `FeatureManagement:email_dob_opt_in` | No | `false` | Feature flag (overridable via AWS AppConfig) | | ||
| | 26 | `JwtSettings:Audience` | No | `SEBT.Portal.Web` | JWT audience claim | | ||
| | 27 | `JwtSettings:ExpirationMinutes` | No | `60` | JWT token lifetime (minutes) | | ||
| | 28 | `JwtSettings:Issuer` | No | `SEBT.Portal.Api` | JWT issuer claim | | ||
|
|
||
| ## Where Each Value Is Currently Set | ||
|
|
||
| **Set by OpenTofu in the ECS task definition** (defined in `tofu/modules/sebt_application/main.tf`): `ASPNETCORE_ENVIRONMENT`, `DB_HOST`, `DB_NAME`, `DB_PORT`, `EmailOtpSenderServiceSettings__SenderEmail`, `PluginAssemblyPaths__0`, `Seeding__EmailPattern`, `Seeding__Enabled`, `SmtpClientSettings__EnableSsl`, `SmtpClientSettings__SmtpPort`, `SmtpClientSettings__SmtpServer`, `STATE`. For the Web container: `BACKEND_URL`, `NEXT_PUBLIC_API_BASE_URL`, `NEXT_PUBLIC_STATE`, `STATE`. | ||
|
|
||
| **Injected from AWS Secrets Manager at container start** (referenced in the ECS task definition): `DB_PASSWORD`, `DB_USER`, `IdentifierHasher__SecretKey`, `JwtSettings__SecretKey`, `SmtpClientSettings__Password`, `SmtpClientSettings__UserName`. | ||
|
|
||
| **Baked into the Docker image via `appsettings.json`**: `EnrollmentCheckRateLimitSettings:*`, `FeatureManagement:*`, `JwtSettings:Audience`, `JwtSettings:ExpirationMinutes`, `JwtSettings:Issuer`. | ||
|
|
||
| **Note:** The application code supports a state-specific config overlay file (`appsettings.co.json`), but none currently exists. If created, it would be baked into the Docker image and could override any `appsettings.json` default. | ||
|
|
||
| ## How Runtime Values Reach the Application | ||
|
|
||
| Runtime configuration reaches the application through a three-step process: | ||
|
|
||
| **Step 1: A generic Docker image is built and pushed to ECR.** The image contains the compiled application code and static defaults from `appsettings.json`, but no environment-specific configuration. The same API image is shared across all states. | ||
|
|
||
| **Step 2: OpenTofu creates or updates the ECS task definition.** When GitHub Actions runs `tofu apply`, Tofu writes two blocks into each container's task definition (defined in `tofu/modules/sebt_application/main.tf`): | ||
|
|
||
| - `environment_variables` — plain-text values like `STATE=co`, `DB_HOST`, and `Seeding__Enabled` | ||
| - `environment_secrets` — references to AWS Secrets Manager ARNs for sensitive values like DB credentials and JWT keys | ||
|
|
||
| The values in these blocks come from three sources: | ||
|
|
||
| - **Hardcoded in the Tofu module** — literal values like `DB_NAME=SebtPortal` and `SmtpClientSettings__SmtpPort=587` | ||
| - **Derived from other Tofu module outputs** — for example, `DB_HOST` comes from the RDS module's endpoint, and `SmtpClientSettings__SmtpServer` comes from the SES module | ||
| - **Passed in from GitHub via `TF_VAR_` environment variables** — the deploy workflow sets environment variables like `TF_VAR_sender_email=${{ vars.SENDER_EMAIL }}`. Tofu automatically reads any `TF_VAR_`-prefixed env var and uses it as the value for the matching variable definition in `variables.tf`. This is how GitHub environment variables (like `SENDER_EMAIL` and `DOMAIN`) make their way into the running application. | ||
|
|
||
| **Step 3: ECS injects the values at container launch.** When ECS starts a container, it reads the task definition and injects all environment variables and resolved secrets into the container's environment. For secret references, ECS fetches the actual values from Secrets Manager at this point — the application never interacts with Secrets Manager directly. | ||
|
|
||
| Once the container starts, the .NET application loads configuration providers in this order (later providers override earlier ones): | ||
|
|
||
| 1. `appsettings.json` — static defaults baked into the Docker image (JWT settings, rate limits, feature flags) | ||
| 2. Environment variables — the values injected by ECS from the task definition; these override `appsettings.json` defaults | ||
| 3. AWS AppConfig Agent — added in `Program.cs`; if configured, polls for feature flag overrides every 90 seconds | ||
| 4. `appsettings.{state}.json` — added in `Program.cs`; state-specific overrides with final priority (supported but not currently used for CO) | ||
|
|
||
| The first two are standard .NET configuration providers. The last two are registered explicitly in `Program.cs` after the defaults, which is why they take higher priority. This means an `appsettings.co.json` file, if created, would be the final word on any value it sets — overriding even environment variables. Note: we may remove support for state-specific config files or restructure the provider order to make AWS AppConfig the highest-priority provider. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,85 @@ | ||
| # DC (District of Columbia) — Runtime Configuration | ||
|
|
||
| This document covers every value that the running DC application consumes at runtime. Build-only and infrastructure-provisioning values (AWS credentials, VPC CIDRs, container sizing, etc.) are omitted. | ||
|
|
||
| ## All Runtime Values | ||
|
|
||
| | # | Name | Secret? | Current Value | Description | | ||
| |---|---|---|---|---| | ||
| | | **API Container — Environment Variables** | | | | | ||
| | 1 | `ASPNETCORE_ENVIRONMENT` | No | `dev` | .NET hosting environment | | ||
| | 2 | `DB_HOST` | No | *(RDS endpoint)* | Database hostname (auto-generated by Tofu from RDS) | | ||
| | 3 | `DB_NAME` | No | `SebtPortal` | Database name | | ||
| | 4 | `DB_PORT` | No | `1433` | SQL Server port | | ||
| | 5 | `EmailOtpSenderServiceSettings__SenderEmail` | No | `noreply@dev.dc.sebt-portal.codeforamerica.app` | "From" address on OTP emails | | ||
| | 6 | `PluginAssemblyPaths__0` | No | `plugins-dc` | Directory containing state plugin DLLs | | ||
| | 7 | `Seeding__EmailPattern` | No | `sebt.dc+{0}@codeforamerica.org` | Format string for seed user emails | | ||
| | 8 | `Seeding__Enabled` | No | `true` | Whether database seeding runs on startup | | ||
| | 9 | `SmtpClientSettings__EnableSsl` | No | `true` | Require TLS for SMTP | | ||
| | 10 | `SmtpClientSettings__SmtpPort` | No | `587` | SMTP port | | ||
| | 11 | `SmtpClientSettings__SmtpServer` | No | *(SES SMTP endpoint)* | SMTP server (auto-generated by Tofu from SES) | | ||
| | 12 | `STATE` | No | `dc` | State code; selects plugin directory and config overlay | | ||
| | | **API Container — Secrets (from AWS Secrets Manager)** | | | | | ||
| | 13 | `DB_PASSWORD` | Yes | *** | Database password (auto-generated by RDS) | | ||
| | 14 | `DB_USER` | Yes | *** | Database username (auto-generated by RDS) | | ||
| | 15 | `IdentifierHasher__SecretKey` | Yes | *** | Key for hashing PII identifiers | | ||
| | 16 | `JwtSettings__SecretKey` | Yes | *** | JWT signing key | | ||
| | 17 | `SmtpClientSettings__Password` | Yes | *** | SES SMTP password (auto-generated by Tofu) | | ||
| | 18 | `SmtpClientSettings__UserName` | Yes | *** | SES SMTP username (auto-generated by Tofu) | | ||
| | | **Web Container — Environment Variables** | | | | | ||
| | 19 | `BACKEND_URL` | No | *(internal ALB URL)* | Server-side API URL (internal to VPC) | | ||
| | 20 | `NEXT_PUBLIC_API_BASE_URL` | No | *(internal ALB URL)* | API URL exposed to browser JS | | ||
| | 21 | `NEXT_PUBLIC_STATE` | No | `dc` | State code exposed to browser JS | | ||
| | 22 | `STATE` | No | `dc` | State code | | ||
| | | **Application Defaults (from `appsettings.json`, overridable at runtime)** | | | | | ||
| | 23 | `EmailOtpSenderServiceSettings:ExpiryMinutes` | No | `10` | OTP code expiry (minutes) | | ||
|
Contributor
There was a problem hiding this comment. These email configs are missing |
||
| | 24 | `EmailOtpSenderServiceSettings:Language` | No | `en` | OTP email language | | ||
| | 25 | `EmailOtpSenderServiceSettings:ProgramName` | No | `DC SUN Bucks` | Program name used in email body | | ||
| | 26 | `EmailOtpSenderServiceSettings:SenderName` | No | `DC SUN Bucks` | Display name on OTP emails | | ||
| | 27 | `EmailOtpSenderServiceSettings:Subject` | No | `Your DC SUN Bucks Login Code` | OTP email subject line | | ||
| | 28 | `EnrollmentCheckRateLimitSettings:PermitLimit` | No | `10` | Max enrollment checks per rate limit window | | ||
| | 29 | `EnrollmentCheckRateLimitSettings:WindowMinutes` | No | `1.0` | Enrollment check rate limit window (minutes) | | ||
| | 30 | `FeatureManagement:email_dob_opt_in` | No | `false` | Feature flag (overridable via AWS AppConfig) | | ||
| | 31 | `JwtSettings:Audience` | No | `SEBT.Portal.Web` | JWT audience claim | | ||
| | 32 | `JwtSettings:ExpirationMinutes` | No | `60` | JWT token lifetime (minutes) | | ||
| | 33 | `JwtSettings:Issuer` | No | `SEBT.Portal.Api` | JWT issuer claim | | ||
| | 34 | `OtpRateLimitSettings:PermitLimit` | No | `5` | Max OTP requests per rate limit window | | ||
| | 35 | `OtpRateLimitSettings:WindowMinutes` | No | `1.0` | OTP rate limit window (minutes) | | ||
|
|
||
| ## Where Each Value Is Currently Set | ||
|
|
||
| **Set by OpenTofu in the ECS task definition** (defined in `tofu/modules/sebt_application/main.tf`): `ASPNETCORE_ENVIRONMENT`, `DB_HOST`, `DB_NAME`, `DB_PORT`, `EmailOtpSenderServiceSettings__SenderEmail`, `PluginAssemblyPaths__0`, `Seeding__EmailPattern`, `Seeding__Enabled`, `SmtpClientSettings__EnableSsl`, `SmtpClientSettings__SmtpPort`, `SmtpClientSettings__SmtpServer`, `STATE`. For the Web container: `BACKEND_URL`, `NEXT_PUBLIC_API_BASE_URL`, `NEXT_PUBLIC_STATE`, `STATE`. | ||
|
|
||
| **Injected from AWS Secrets Manager at container start** (referenced in the ECS task definition): `DB_PASSWORD`, `DB_USER`, `IdentifierHasher__SecretKey`, `JwtSettings__SecretKey`, `SmtpClientSettings__Password`, `SmtpClientSettings__UserName`. | ||
|
Contributor
There was a problem hiding this comment. For |
||
|
|
||
| **Baked into the Docker image via `appsettings.json`**: `EmailOtpSenderServiceSettings:*` (except `SenderEmail`, which is overridden by env var), `EnrollmentCheckRateLimitSettings:*`, `FeatureManagement:*`, `JwtSettings:Audience`, `JwtSettings:ExpirationMinutes`, `JwtSettings:Issuer`, `OtpRateLimitSettings:*`. | ||
|
|
||
| **Note:** The application code supports a state-specific config overlay file (`appsettings.dc.json`), but none currently exists. If created, it would be baked into the Docker image and could override any `appsettings.json` default. | ||
|
|
||
| ## How Runtime Values Reach the Application | ||
|
|
||
| Runtime configuration reaches the application through a three-step process: | ||
|
|
||
| **Step 1: A generic Docker image is built and pushed to ECR.** The image contains the compiled application code and static defaults from `appsettings.json`, but no environment-specific configuration. The same API image is shared across all states. | ||
|
|
||
| **Step 2: OpenTofu creates or updates the ECS task definition.** When GitHub Actions runs `tofu apply`, Tofu writes two blocks into each container's task definition (defined in `tofu/modules/sebt_application/main.tf`): | ||
|
|
||
| - `environment_variables` — plain-text values like `STATE=dc`, `DB_HOST`, and `Seeding__Enabled` | ||
| - `environment_secrets` — references to AWS Secrets Manager ARNs for sensitive values like DB credentials and JWT keys | ||
|
|
||
| The values in these blocks come from three sources: | ||
|
|
||
| - **Hardcoded in the Tofu module** — literal values like `DB_NAME=SebtPortal` and `SmtpClientSettings__SmtpPort=587` | ||
| - **Derived from other Tofu module outputs** — for example, `DB_HOST` comes from the RDS module's endpoint, and `SmtpClientSettings__SmtpServer` comes from the SES module | ||
| - **Passed in from GitHub via `TF_VAR_` environment variables** — the deploy workflow sets environment variables like `TF_VAR_sender_email=${{ vars.SENDER_EMAIL }}`. Tofu automatically reads any `TF_VAR_`-prefixed env var and uses it as the value for the matching variable definition in `variables.tf`. This is how GitHub environment variables (like `SENDER_EMAIL` and `DOMAIN`) make their way into the running application. | ||
|
|
||
| **Step 3: ECS injects the values at container launch.** When ECS starts a container, it reads the task definition and injects all environment variables and resolved secrets into the container's environment. For secret references, ECS fetches the actual values from Secrets Manager at this point — the application never interacts with Secrets Manager directly. | ||
|
|
||
| Once the container starts, the .NET application loads configuration providers in this order (later providers override earlier ones): | ||
|
|
||
| 1. `appsettings.json` — static defaults baked into the Docker image (JWT settings, rate limits, email templates, feature flags) | ||
| 2. Environment variables — the values injected by ECS from the task definition; these override `appsettings.json` defaults | ||
|
Contributor
There was a problem hiding this comment. Should this list include |
||
| 3. AWS AppConfig Agent — added in `Program.cs`; if configured, polls for feature flag overrides every 90 seconds | ||
| 4. `appsettings.{state}.json` — added in `Program.cs`; state-specific overrides with final priority (supported but not currently used for DC) | ||
|
|
||
| The first two are standard .NET configuration providers. The last two are registered explicitly in `Program.cs` after the defaults, which is why they take higher priority. This means an `appsettings.dc.json` file, if created, would be the final word on any value it sets — overriding even environment variables. Note: we may remove support for state-specific config files or restructure the provider order to make AWS AppConfig the highest-priority provider. | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I can't find any references to
EnrollmentCheckRateLimitSettings:PermitLimitorEnrollmentCheckRateLimitSettings:WindowMinutesin the appsettings.json or anywhere else in the codebase. This may be hallucinated? Same for DC.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not hallucinated, but from pending, work-in-progress changes to appsettings.json that I didn't stage here. Whoops. I'll get that cleaned up.