feat(llm-obs): add annotation queue label schema and annotation commands

[gsvigruha]

Summary

Wires up four LLM Observability annotation endpoints that already exist in the pinned datadog-api-client-rust SDK but were never exposed by pup. Queue label schemas could previously only be set at creation time via annotation-queues create --file, with no way to read one back or edit it; annotations on queue interactions had no CLI surface at all.

Changes

• Add llm-obs annotation-queues schema get|update (src/commands/llm_obs.rs)
• Add llm-obs annotation-queues annotations upsert|delete (src/commands/llm_obs.rs)
• Register the four operations in UNSTABLE_OPS (src/client.rs:323) — without this the SDK rejects them with UnstableOperationDisabledError
• Update the UNSTABLE_OPS section comment (21→25) and length assertion (186→190) (src/client.rs)
• Document the annotation-queues subtree, which was absent from the llm-obs row entirely (docs/COMMANDS.md:69)

New commands:

pup llm-obs annotation-queues schema get <queue_id>
pup llm-obs annotation-queues schema update <queue_id> --file
pup llm-obs annotation-queues annotations upsert <queue_id> --file
pup llm-obs annotation-queues annotations delete <queue_id> --file

Both annotation endpoints are queue-scoped (they take a queue_id), so they are nested under annotation-queues rather than a top-level llm-obs annotations group.

Testing

9 new tests in src/commands/llm_obs.rs, positive and negative per endpoint:

• Happy path for all four commands against mockito
• API errors: 404 on schema get, 400 on annotations upsert
• Bad input: unreadable file, malformed JSON, and valid JSON missing the required data member

Verified: cargo fmt --check clean, cargo clippy --all-targets -- -D warnings clean, cargo test 1725 passed.

CLI-level smoke test confirms routing reaches the handlers and read_only classification is correct (schema get read-only; the three writes not).

Note on the suite: four tests in dbm, monitors, security, and traces fail under parallel execution on main as well — verified by stashing this work and re-running on a clean tree (1716 passed, same 4 failed). They pass individually, so they are pre-existing env-var races unrelated to this change.

Not included

Monitor annotations (exposed by the data-observability MCP server) are not in the pinned SDK for either datadogV1 or datadogV2, so adding them would require hand-rolled HTTP against non-public endpoints. Left out deliberately.

---

🤖 Generated with Claude Code

[gsvigruha]

Integration-tested against the real API — three bugs found and fixed

The original commit was only verified against mockito. Because mock_any matches any path and any body, and I wrote the fixtures myself, the tests passed while three of the four commands were broken against real responses.

Retested against a live org. The generated client declares annotation_schema, annotations and annotation_ids as non-Option, but the API returns null for each in ordinary cases:

Command	Real-world failure
schema get	Queue with no schema yet returns "annotation_schema": null → invalid type: null, expected a mapping. Most queues have no schema, so this was the common case.
annotations upsert	Per-item failures come back as HTTP 200 with "annotations": null and a populated errors array → invalid type: null, expected a sequence, turning a readable partial-failure report into an opaque serde error.
annotations delete	Same exposure — both annotation_ids and errors are non-Option Vecs.

schema update was unaffected (it echoes back the schema just written, never null).

Fix

All four now go through raw_client, matching the existing precedent in this file — see experiments_update, which uses the raw path for the same class of generated-client mismatch. Request shapes and URL paths are unchanged. The UNSTABLE_OPS additions are reverted, since the SDK methods are no longer called.

Verification

Now confirmed against the real API, not just mocks:

• schema get on a schema-less queue → returns "annotation_schema": null cleanly
• schema get on a populated queue → both label schemas returned
• schema update → 2xx, and a follow-up schema get confirms the schema persisted
• annotations upsert with an unknown interaction → renders the 200 partial-failure report instead of erroring
• annotations delete with an unknown id → renders the per-item error report
• All three writes against a nonexistent queue → distinct, endpoint-specific 404s, confirming correct URL routing

Test scaffolding was a throwaway queue created and then deleted; deletion verified by a 404.

Tests are now 13 (up from 9), including regression tests that pin the null-response shapes, captured from real responses rather than hand-written. cargo fmt --check and cargo clippy --all-targets -- -D warnings clean; 1729 passed.

Separately: a pre-existing bug this surfaced

pup llm-obs annotation-queues list (already on main, not touched by this PR) fails against real data with the same root cause:

Error: failed to list annotation queues: Serde(Error("invalid type: null, expected a sequence"))

Not fixed here to keep this PR scoped — worth its own issue.