Skip to content

[Bug]: aws_bedrockagentcore_agent_runtime — default CloudWatch log group is auto-created outside Terraform, no way to configure retention/KMS/tags/forwarding #47630

@mano1233

Description

@mano1233

Terraform and AWS Provider Version

Terraform v1.9.8
on linux_amd64
+ provider registry.terraform.io/hashicorp/aws v6.27.0

Affected Resource(s) or Data Source(s)

  • aws_bedrockagentcore_agent_runtime

Expected Behavior

Users should be able to declaratively control the configuration of every resource AgentCore creates on their behalf from inside aws_bedrockagentcore_agent_runtime — specifically the default CloudWatch log group at /aws/bedrock-agentcore/runtimes/<agent_runtime_id>-DEFAULT.

Today, aws_bedrockagentcore_agent_runtime implicitly provisions this log group as a side effect of runtime creation, but exposes no way to configure:

  • retention_in_days
  • kms_key_id
  • tags
  • log_group_class
  • Downstream references (subscription filters, metric filters, resource policies) that need the log group name/ARN

Attempting to manage it with a separate aws_cloudwatch_log_group in the same config fails (see Actual Behavior), so these log groups end up silently running on AWS service defaults (no retention = Never expire, no customer KMS, no tags, no Datadog/OpenSearch forwarding) and silently escape Terraform's source of truth.

Actual Behavior

The AgentCore control plane (or the runtime itself on first startup logging) creates /aws/bedrock-agentcore/runtimes/<runtime_name>_<protocol>-<RUNTIME_ID>-DEFAULT the moment the runtime becomes READY. This is before Terraform's graph can reach any sibling aws_cloudwatch_log_group resource that depends on the runtime's agent_runtime_id, so the subsequent CreateLogGroup call fails:

Error: creating CloudWatch Logs Log Group (/aws/bedrock-agentcore/runtimes/<runtime_name>_a2a-<RUNTIME_ID>-DEFAULT):
  ResourceAlreadyExistsException: The specified log group already exists

This leaves users with two bad choices:

  1. Accept service defaults and give up Terraform-managed retention, KMS, tags, and log forwarding on a production observability surface.
  2. Adopt via import {} every time a runtime is (re)created — but the log group name embeds a server-generated agent_runtime_id that isn't known until after create, so the import block has to be added reactively (run, fail, read the ID from the error, add the block, re-run, then remove it). This doesn't work in read-only CI pipelines and doesn't survive runtime replacement.

Neither approach gives the user a stable, declarative way to own the log group's configuration from the start.

Relevant Error/Panic Output

Error: creating CloudWatch Logs Log Group (/aws/bedrock-agentcore/runtimes/metadata_retrieval_agent_stg_a2a-FB8qJdBWZp-DEFAULT): operation error CloudWatch Logs: CreateLogGroup, https response error StatusCode: 400, RequestID: d790b973-2539-47f3-8faf-b5c4ba5bbba3, ResourceAlreadyExistsException: The specified log group already exists

Error: creating CloudWatch Logs Log Group (/aws/bedrock-agentcore/runtimes/metadata_retrieval_agent_stg_mcp-tgz13FGbSH-DEFAULT): operation error CloudWatch Logs: CreateLogGroup, https response error StatusCode: 400, RequestID: e621e8d7-ba60-4c03-a541-1f7d8c65967e, ResourceAlreadyExistsException: The specified log group already exists

Sample Terraform Configuration

Click to expand configuration
terraform {
  required_version = ">= 1.9.0"
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "6.27.0"
    }
  }
}

provider "aws" {
  region = "eu-west-1"
}

resource "aws_iam_role" "runtime" {
  name = "bedrockagentcore-runtime-role"
  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Effect    = "Allow"
      Principal = { Service = "bedrock-agentcore.amazonaws.com" }
      Action    = "sts:AssumeRole"
    }]
  })
}

resource "aws_bedrockagentcore_agent_runtime" "example" {
  agent_runtime_name = "example_runtime_a2a"
  role_arn           = aws_iam_role.runtime.arn

  agent_runtime_artifact {
    container_configuration {
      container_uri = "123456789012.dkr.ecr.eu-west-1.amazonaws.com/example:latest"
    }
  }

  network_configuration {
    network_mode = "PUBLIC"
  }
}

# Intent: manage retention, KMS, tags on the default log group AgentCore creates.
# Today this fails because AgentCore has already created this log group by the
# time Terraform reaches this resource.
resource "aws_cloudwatch_log_group" "agent_runtime" {
  name              = "/aws/bedrock-agentcore/runtimes/${aws_bedrockagentcore_agent_runtime.example.agent_runtime_id}-DEFAULT"
  retention_in_days = 30
  kms_key_id        = aws_kms_key.logs.arn
  tags = {
    Environment = "stg"
  }
}

# Intent: forward runtime logs to a downstream sink. Can't be authored
# declaratively today because the log_group_name above can't be created by
# Terraform.
resource "aws_cloudwatch_log_subscription_filter" "forward" {
  name            = "datadog-forwarder"
  log_group_name  = aws_cloudwatch_log_group.agent_runtime.name
  destination_arn = var.datadog_forwarder_arn
  filter_pattern  = ""
}

Steps to Reproduce

  1. Apply the configuration above against an AWS account that has never hosted this named runtime.
  2. aws_bedrockagentcore_agent_runtime creates successfully; AgentCore implicitly creates the default log group.
  3. Terraform attempts to create aws_cloudwatch_log_group.agent_runtime and fails with ResourceAlreadyExistsException.
  4. Even after working around step 3 with a reactive import {} block, the user has to re-do this dance any time the runtime is replaced (different agent_runtime_id ⇒ different log group name ⇒ different import ID).

Debug Logging

Click to expand log output

GenAI / LLM Assisted Development

Issue drafted with AI assistance (Claude) based on firsthand reproduction. All logs, configs, and version strings are from real runs.

Important Facts and References

Would you like to implement a fix?

No

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugAddresses a defect in current functionality.service/bedrockagentcoreIssues and PRs that pertain to the bedrockagentcore service.service/iamIssues and PRs that pertain to the iam service.service/logsIssues and PRs that pertain to the logs service.

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions