feat(agent-obs): close MCP parity gaps for projects, datasets, and span search - #681

Summary

Closes four LLM Observability parity gaps between pup llm-obs and the LLM Obs MCP toolset in dd-source (domains/ml-observability/shared/libs/mcp/tools). All four reuse endpoints pup already talks to, so no new API surface is introduced.

Found by auditing pup's LLM Obs surface against the 40 tools the MCP registers; pup covered 25 of them.

Changes

• llm-obs projects list — filters + pagination (src/commands/llm_obs.rs:36)
  New --filter-id, --filter-name, --limit, --cursor, mapping to the public v2 filter[id] / filter[name] / page[limit] / page[cursor] params. Closes both project gaps: list_llmobs_projects pagination and get_llmobs_project name→UUID resolution — the MCP tool is itself a zero-or-one-match filter, so --filter-name is the same operation. This matters for agents: every dataset/record command needs a project UUID, and there was previously no name resolution step.
• llm-obs datasets list — filters + pagination (src/commands/llm_obs.rs:126)
  Same four flags, closing list_llmobs_datasets.
• llm-obs datasets records-add — new command (src/commands/llm_obs.rs:265)
  Wraps POST /api/unstable/llm-obs-mcp/v1/dataset/records-add, closing add_llmobs_dataset_records. Preserves the endpoint's two-step safety: previews by default, writes only with --confirm. --create-new-version <bool> controls the version bump. Records come from --file.
• llm-obs spans search — --tags and --apm-trace-id (src/commands/llm_obs.rs:646)
  --tags takes comma-separated key:value and converts to the JSON object map the endpoint expects. parse_tag_filters splits on the first colon only, so values like version:1.2:3 survive intact.
• Read-only guard (src/main.rs:10786)
  Added name.ends_with("-add") to is_write_command_name. The guard is name-based and no existing pattern matched records-add, so without this it would bypass --read-only and be advertised as read-only in the agent schema.
• Docs (docs/COMMANDS.md:69) — added records-add to the llm-obs command inventory.

Testing

13 new tests in src/commands/llm_obs.rs and src/main.rs:

• Strict query-param matchers asserting the new filter/pagination params reach both list endpoints
• PartialJson body matchers for the new span filters and for both records-add modes (preview vs confirmed, including create_new_version: false)
• Negative cases: malformed --tags (envprod, :prod, env:), non-array / empty-array / missing records file, records-add 400
• Unit test for parse_tag_filters covering the colon-in-value case
• Read-only classification test asserting records-add is a write while records and records-full stay read-only

Also smoke-tested against the live API (datadoghq.com):

• Project name filter resolves a single project; dataset name filter resolves a single dataset
• --tags=env:prod,ml_app:docstore returns matching spans, while a bogus tag returns 0 results — confirming the filter is applied server-side rather than silently ignored
• --apm-trace-id returns only that trace's spans
• records-add preview returned a real confirmation_prompt, and --create-new-version=false correctly flipped will_bump_version

cargo fmt --check, cargo clippy --all-targets -- -D warnings, and cargo test llm_obs (99/99) all pass.

Two notes for reviewers:

• The records-add insert branch (--confirm) is covered by unit tests only — I did not write to a real dataset, since the one available belonged to another person's project.
• cargo test (full suite) shows a handful of failures in traces / dbm / monitors / security. These are pre-existing flakes unrelated to this PR: they reproduce on an unmodified tree and all pass in isolation (env-var-lock contention under parallelism).

Not included

launch_llmobs_experiment was deliberately left out. The other experiment-side gap, create_llmobs_experiment, is a pure record-create with no inference and is already covered by llm-obs experiments create.

Still open from the audit, for follow-up: Topic Discovery / Patterns (7 tools), Agent Insights (4 tools), get_llmobs_model_pricing, get_llmobs_bits_session, and the 9 LLM-Obs workflow skills that pup skills can't reach (it targets /api/v2/onboarding/skills, a different backend).

🤖 Generated with Claude Code