feat(agent): agent operability overhaul for AI coding assistants - #70
Merged
Conversation
Make pup the best CLI tool for AI agents to interact with Datadog by minimizing round trips, maximizing information density, and providing structured output that agents can parse without guessing. Phase 1 - Agent Detection & Mode Infrastructure: - Expand agent detection to 11+ AI coding assistants (Claude Code, Cursor, Codex, Aider, Cline, Windsurf, GitHub Copilot, Amazon Q, Gemini Code, Sourcegraph Cody, OpenCode) - Add IsAgentMode() and DetectAgentInfo() exported functions - Add DD_AGENT_MODE=1 explicit override env var - Add --agent persistent flag to root command - Add AgentMode field to Config struct - Auto-approve prompts in agent mode to prevent stdin hangs Phase 2 - Agent Help Schema (--hlp): - Add --hlp flag that outputs complete command schema as structured JSON - Schema includes all commands, flags, query syntax, time formats, workflows, best practices, and anti-patterns in a single call - Support subtree schemas: 'pup logs --hlp' returns only logs-related info - Eliminates multi-round-trip help lookups (was: --help per command) Phase 3 - Agent Command Group: - Add 'pup agent schema' command (same as --hlp) - Add 'pup agent schema --compact' for minimal token-efficient output - Add 'pup agent guide' with comprehensive embedded steering document - Add 'pup agent guide <domain>' for domain-specific sections - Add "agent" to reserved command list in alias.go Phase 4 - Structured Output: - Add AgentEnvelope type wrapping responses with metadata (count, truncated, next_action hints, warnings) - Add AgentError type for structured error responses with suggestions - Add formatAndPrint() helper that wraps output in agent mode, passes through in human mode - Migrate monitors list/get, logs search, metrics query to formatAndPrint - Structured errors include status code-specific suggestions (401->re-auth, etc.) Phase 5 - Smart Defaults: - Agent mode: monitors list default limit 500 (vs 200) - Agent mode: logs search default limit 200 (vs 50) - Human mode behavior is completely unchanged Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ookup - formatAPIError now handles *http.Response (field-based StatusCode) in addition to the interface-based StatusCode() method. This enables structured agent errors for all API calls including v1 endpoints. - GetGuideSection now tries uppercase, capitalized, and case-insensitive matching for section headings (e.g., "apm" finds "## APM / Traces"). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When an AI agent runs `pup --help` or `pup logs --help`, it now gets the structured JSON schema automatically instead of cobra's human-oriented help text. This is detected via agent env vars (CLAUDECODE, CURSOR_AGENT, etc.) before cobra processes the args. - `pup --help` in agent mode → full JSON schema (same as --hlp) - `pup logs --help` in agent mode → logs subtree schema - `pup --help` in human mode → unchanged (normal help text) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Every command that outputs API responses now goes through formatAndPrint
instead of calling formatter.FormatOutput directly. This ensures all
commands automatically get agent envelope wrapping in agent mode.
- 135 formatter.FormatOutput calls replaced with formatAndPrint across
38 command files
- Removed unused formatter and fmt imports
- Net reduction: -735 lines (868 deleted, 133 added)
In agent mode, every command now returns structured output:
{"status": "success", "data": ..., "metadata": {...}}
Human mode behavior is completely unchanged.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
In agent mode (auto-detected or FORCE_AGENT_MODE=1), --help now returns structured JSON schema. This eliminates the need for a separate --hlp flag — agents can use the standard --help and get machine-readable output automatically. - Remove --hlp flag, HandleHlpFlag, errHlpHandled sentinel - Rename DD_AGENT_MODE env var to FORCE_AGENT_MODE for testing - Update tests to use FORCE_AGENT_MODE and --help instead of --hlp - Update guide.md and docstrings Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Rewrites LLM_GUIDE.md to document the full agent operability system: agent mode detection, --help JSON schema, pup agent commands, output envelope, structured errors, query syntax, workflows, and architecture reference with file map. Updates ARCHITECTURE.md user agent section to reflect 11 agent detectors and agent mode behavior. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
# Conflicts: # cmd/notebooks.go
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Transforms pup from a human-oriented CLI into a dual-mode CLI that provides first-class programmatic interfaces for AI coding assistants while maintaining full backward compatibility.
Changes
Agent Detection & Mode System
pkg/useragent/useragent.go)--agentflag andFORCE_AGENT_MODE=1env var for explicit agent mode--helpAuto-Detection (Zero Extra Round Trips)--help/-hreturns structured JSON schema instead of human text (cmd/root.go)pup logs --helpreturns only logs commands + query syntax--helpis completely unchangedNew
pup agentCommand Group (cmd/agent.go)pup agent schema— full JSON schema (same as--helpin agent mode)pup agent schema --compact— minimal token-efficient schemapup agent guide— comprehensive steering guide (300+ lines)pup agent guide [domain]— domain-specific guidanceAgent-Friendly Output Enhancements
formatAndPrint()wrapper withAgentEnvelopemetadata (count, truncation, warnings) (pkg/formatter/envelope.go)cmd/root.go:formatAPIError)formatAndPrintSteering Content (
pkg/agenthelp/)Documentation
docs/LLM_GUIDE.mdas comprehensive agent operability referencedocs/ARCHITECTURE.mduser agent section for 11 agents + agent modeTest plan
FORCE_AGENT_MODE=1 pup --helpreturns valid JSON schemaFORCE_AGENT_MODE=1 pup logs --helpreturns subtree schema--helpunchanged (no JSON output)pup agent schema,pup agent schema --compact,pup agent guideall testedgo build ./...clean🤖 Generated with Claude Code