Skip to content

fix(ci): limit e2e tests to cdk mainline#1500

Draft
Hweinstock wants to merge 1 commit into
aws:mainfrom
Hweinstock:fix/e2e-full-cdk-main-only
Draft

fix(ci): limit e2e tests to cdk mainline#1500
Hweinstock wants to merge 1 commit into
aws:mainfrom
Hweinstock:fix/e2e-full-cdk-main-only

Conversation

@Hweinstock

Copy link
Copy Markdown
Contributor

Problem

#1494

We receive intermittent 403s on control plane calls. The clients do not retry, since 403 is treated as non-retryable and the tests fail. Usually retrying fixes the issue.

Looking deeper, we don't see these request in CloudTrail, suggesting they are getting rejected before hitting the service at the edge. Given that we have both added significantly more tests recently, and run it across 12 runners, means we could be hammering those APIs, and get rate limited. The current evidence suggests this is the most likely cause.

Solution

  • remove 6 runners by only testing against main. The primary use case for testing against npm is for knowing if the current CLI requires the newer CDK changes, which isn't very useful since we always release the CDK first. If we determine we still need this functionality, we can consider adding an optional workflow dispatch override for testing against non-main branch, allowing us to test on latest release manually.
  • This cuts are API reqs in half and should significantly reduce the amount of API calls we make, and hopefully reduce the throttling. If this does not reduce the 403s, we can revert this and continue to investigate.

@github-actions github-actions Bot added the size/xs PR size: XS label Jun 9, 2026
@github-actions github-actions Bot added the agentcore-harness-reviewing AgentCore Harness review in progress label Jun 9, 2026
@agentcore-devx-automation agentcore-devx-automation Bot added the claude-security-reviewing Claude Code /security-review in progress label Jun 9, 2026
@agentcore-devx-automation

Copy link
Copy Markdown
Contributor

Claude Security Review: no high-confidence findings. (run)

@agentcore-devx-automation agentcore-devx-automation Bot removed the claude-security-reviewing Claude Code /security-review in progress label Jun 9, 2026
@github-actions

github-actions Bot commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

Package Tarball

aws-agentcore-0.18.0.tgz

How to install

gh release download pr-1500-tarball --repo aws/agentcore-cli --pattern "*.tgz" --dir /tmp/pr-tarball
npm install -g /tmp/pr-tarball/aws-agentcore-0.18.0.tgz

@agentcore-cli-automation agentcore-cli-automation left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Small, surgical change that does exactly what the description says — drops the cdk-source matrix dimension (cutting runners 12 → 6) and unconditionally runs the two steps that were previously gated on matrix.cdk-source == 'main' (Generate GitHub App Token, Build CDK package from main). Verified no stragglers reference matrix.cdk-source after the change, and the step display name is updated consistently.

The trade-off (no longer testing against the published npm CDK) is acknowledged in the description and the rationale (CDK ships before CLI, so the matrix arm provided limited signal) is sound. Easy to revert if 403s persist, and a workflow_dispatch override for an arbitrary CDK branch would be a clean way to bring back manual npm-source coverage if needed.

@github-actions github-actions Bot removed the agentcore-harness-reviewing AgentCore Harness review in progress label Jun 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/xs PR size: XS

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants