Skip to content

feat: instrument telemetry for invoke command#1227

Open
Hweinstock wants to merge 2 commits into
aws:mainfrom
Hweinstock:feat/invoke-telemetry
Open

feat: instrument telemetry for invoke command#1227
Hweinstock wants to merge 2 commits into
aws:mainfrom
Hweinstock:feat/invoke-telemetry

Conversation

@Hweinstock
Copy link
Copy Markdown
Contributor

@Hweinstock Hweinstock commented May 13, 2026

Description

Add telemetry instrumentation to the invoke command for all CLI execution paths using withCommandRunTelemetry.

As with other PRs for instrumenting telemetry, the main changes are refactoring to the error handling to avoid killing the process before telemetry can be emitted.

Related Issue

Closes #

Documentation PR

N/A

Type of Change

  • New feature

Testing

How have you tested the change?

  • I ran npm run test:unit and npm run test:integ
  • I ran npm run typecheck
  • I ran npm run lint

Additional testing:

  • Added telemetry assertions to all existing invoke tests (validation, agent validation, streaming)
  • Added new test cases for --bearer-token (auth_type) and --session-id (has_session_id)
  • 7/7 invoke tests pass with telemetry assertions

Checklist

  • I have read the CONTRIBUTING document
  • I have added any necessary tests that prove my fix is effective or my feature works
  • I have updated the documentation accordingly
  • I have added an appropriate example to the documentation to outline the feature, or no new docs are needed
  • My changes generate no new warnings
  • Any dependent changes have been merged and published

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@github-actions github-actions Bot added size/m PR size: M agentcore-harness-reviewing AgentCore Harness review in progress labels May 13, 2026
@github-actions
Copy link
Copy Markdown
Contributor

Package Tarball

aws-agentcore-0.13.1.tgz

How to install

npm install https://github.com/aws/agentcore-cli/releases/download/pr-1227-tarball/aws-agentcore-0.13.1.tgz

Copy link
Copy Markdown

@agentcore-cli-automation agentcore-cli-automation left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

Reviewed the changes against the telemetry README and the deploy command's pattern. The PR:

  • Correctly wraps both CLI and TUI paths with withCommandRunTelemetry('invoke', ...), including validation/prompt-resolution failures (these now emit exit_reason: 'failure' instead of being skipped).
  • Properly refactors handleInvokeCLI to return InvokeResult so telemetry can be recorded before output/exit, and extracts printInvokeResult to keep the I/O concerns in the action handler.
  • As a side-benefit, switches JSON output to use serializeResult, fixing the latent bug where a failure result's error would have serialized as {} (Error has non-enumerable props).
  • Tests use real subprocess + temp dirs + audit-mode telemetry helper — no excessive mocking.

A few small things I considered and decided weren't worth blocking on:

  • loadInvokeConfig is now called twice on the CLI path (once up-front for the protocol attribute, once inside handleInvokeCLI). Local file reads, fine.
  • TUI mode always records exit_reason: 'success' since the callback returns {success: true} after waitUntilExit. Per-invoke failures inside an interactive session aren't captured, but that's consistent with one command_run metric per command invocation.
  • A bad -H value would throw from parseHeaderFlags before the telemetry wrapper runs and skip the metric, but that matches existing behavior.

@github-actions github-actions Bot removed the agentcore-harness-reviewing AgentCore Harness review in progress label May 13, 2026
@Hweinstock
Copy link
Copy Markdown
Contributor Author

loadInvokeConfig is now called twice on the CLI path (once up-front for the protocol attribute, once inside handleInvokeCLI). Local file reads, fine.

feels worth addressing.

TUI mode always records exit_reason: 'success' since the callback returns {success: true} after waitUntilExit. Per-invoke failures inside an interactive session aren't captured, but that's consistent with one command_run metric per command invocation.

I think this is the behavior we want. We may one day want separate telemetry for the individual invocations.

A bad -H value would throw from parseHeaderFlags before the telemetry wrapper runs and skip the metric, but that matches existing behavior.

worth addressing and making it a validation exception.

@github-actions github-actions Bot added size/m PR size: M and removed size/m PR size: M labels May 13, 2026
…arsing inside telemetry wrapper

- Use ValidationError for validation and prompt resolution failures
- Pass pre-loaded InvokeContext into handleInvokeCLI to avoid reading
  config files twice on the happy path
- Move parseHeaderFlags inside the telemetry wrapper so invalid -H
  values are recorded as failures
@Hweinstock Hweinstock force-pushed the feat/invoke-telemetry branch from 3f8220a to 0de0e68 Compare May 13, 2026 18:17
@github-actions github-actions Bot added size/m PR size: M and removed size/m PR size: M labels May 13, 2026
@Hweinstock Hweinstock marked this pull request as ready for review May 13, 2026 18:25
@Hweinstock Hweinstock requested a review from a team May 13, 2026 18:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/m PR size: M

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants