The --config-overrides flag cannot be used to configure options for external plugin reporters (e.g., bedrock-summary-reporter). The reporters section of AshConfig is a fixed Pydantic model (ReporterConfigSegment) that only contains fields for built-in reporters. External plugin reporter configs have no corresponding field on this model, so overrides like reporters.bedrock-summary-reporter.options.model_id=... are silently ignored.
Steps to Reproduce
- Run:
ash report --format bedrock-summary-reporter \
--output-dir .ash/ash_output \
--config-overrides 'ash_plugin_modules=["automated_security_helper.plugin_modules.ash_aws_plugins"]' \
--config-overrides 'reporters.bedrock-summary-reporter.options.model_id=us.anthropic.claude-haiku-4-5-20251001-v1:0' \
--log-level VERBOSE
- Observe that the reporter still uses its hardcoded default model (
us.anthropic.claude-3-7-sonnet-20250219-v1:0) instead of the overridden value.
Expected Behavior
--config-overrides should be able to set options for any registered reporter, including those loaded from external plugin modules.
Workaround
Use the ASH_BEDROCK_MODEL_ID environment variable instead:
ASH_BEDROCK_MODEL_ID="us.anthropic.claude-haiku-4-5-20251001-v1:0" ash report --format bedrock-summary-reporter ...
Root Cause
The ReporterConfigSegment Pydantic model in automated_security_helper/config/ash_config.py has statically defined fields for each built-in reporter (csv, html, markdown, etc.). External plugins registered via ash_plugin_modules don't get a corresponding field added to this model, so:
- The YAML config can't define
reporters.bedrock-summary-reporter (it would fail validation)
--config-overrides targeting that path has no effect
Suggested Fix
Options:
-
Dynamic reporter config: Change ReporterConfigSegment to accept arbitrary additional keys (e.g., using model_config = ConfigDict(extra="allow")) so external plugin configs can be defined in the YAML and overridden via CLI.
-
Plugin-specific override namespace: Add a separate config section like plugin_options that accepts arbitrary key-value pairs and is passed to plugins at initialization.
-
Registry-based approach: After loading plugin modules, dynamically register their config schemas into the ReporterConfigSegment model before parsing the config file.
Environment
- ASH version: v3 (installed via
uvx)
- OS: Linux (Amazon Linux)
The
--config-overridesflag cannot be used to configure options for external plugin reporters (e.g.,bedrock-summary-reporter). Thereporterssection ofAshConfigis a fixed Pydantic model (ReporterConfigSegment) that only contains fields for built-in reporters. External plugin reporter configs have no corresponding field on this model, so overrides likereporters.bedrock-summary-reporter.options.model_id=...are silently ignored.Steps to Reproduce
us.anthropic.claude-3-7-sonnet-20250219-v1:0) instead of the overridden value.Expected Behavior
--config-overridesshould be able to set options for any registered reporter, including those loaded from external plugin modules.Workaround
Use the
ASH_BEDROCK_MODEL_IDenvironment variable instead:ASH_BEDROCK_MODEL_ID="us.anthropic.claude-haiku-4-5-20251001-v1:0" ash report --format bedrock-summary-reporter ...Root Cause
The
ReporterConfigSegmentPydantic model inautomated_security_helper/config/ash_config.pyhas statically defined fields for each built-in reporter (csv, html, markdown, etc.). External plugins registered viaash_plugin_modulesdon't get a corresponding field added to this model, so:reporters.bedrock-summary-reporter(it would fail validation)--config-overridestargeting that path has no effectSuggested Fix
Options:
Dynamic reporter config: Change
ReporterConfigSegmentto accept arbitrary additional keys (e.g., usingmodel_config = ConfigDict(extra="allow")) so external plugin configs can be defined in the YAML and overridden via CLI.Plugin-specific override namespace: Add a separate config section like
plugin_optionsthat accepts arbitrary key-value pairs and is passed to plugins at initialization.Registry-based approach: After loading plugin modules, dynamically register their config schemas into the
ReporterConfigSegmentmodel before parsing the config file.Environment
uvx)