Skip to content

Commit 597a7ce

Browse files
committed
docs: address review comments - remove Jira keys, pin to release tag, fix API key placeholder, link to repo file
1 parent 9b6df51 commit 597a7ce

1 file changed

Lines changed: 8 additions & 8 deletions

File tree

agents/claude-code/mlflow-tracing.md

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ We deployed Claude Code as a containerized agent on Red Hat OpenShift AI and wir
44

55
---
66

7-
## RHAIENG-4751 — Inventory OGX Telemetry Hooks and MLflow Integration Points
7+
## Inventory OGX Telemetry Hooks and MLflow Integration Points
88

99
### Summary
1010

@@ -88,13 +88,13 @@ We ran **"build me a tetris game"** against all three backends. All three produc
8888

8989
---
9090

91-
## RHAIENG-4752 & RHAIENG-4753 — Tool Call Traces & Agent Execution Metrics
91+
## Tool Call Traces & Agent Execution Metrics
9292

9393
### Summary
9494

95-
**RHAIENG-4752** — We prototyped tool call tracing using `mlflow autolog claude`. Every tool Claude Code calls (Write, Read, Edit, Bash, AskUserQuestion, etc.) is captured as a span in MLflow with the tool name, input parameters, output/result, and latency. Tested across three backends with a real coding task — Vertex AI produced 15 spans, vLLM and OGX produced 8 each. MLflow integration works end-to-end. The stop-hook fires after the session so there is no latency impact.
95+
**Tool call tracing** — We prototyped tool call tracing using `mlflow autolog claude`. Every tool Claude Code calls (Write, Read, Edit, Bash, AskUserQuestion, etc.) is captured as a span in MLflow with the tool name, input parameters, output/result, and latency. Tested across three backends with a real coding task — Vertex AI produced 15 spans, vLLM and OGX produced 8 each. MLflow integration works end-to-end. The stop-hook fires after the session so there is no latency impact.
9696

97-
**RHAIENG-4753** — On top of the tool call spans, each trace also captures higher-level session metrics: session ID, total duration, input/output token counts, and the full tool call sequence as a waterfall. This answers "what did the agent do and how much did it cost?" for any session. Validated with a complete multi-turn coding task ("build me a tetris game") across all three backends.
97+
**Session-level metrics** — On top of the tool call spans, each trace also captures higher-level session metrics: session ID, total duration, input/output token counts, and the full tool call sequence as a waterfall. This answers "what did the agent do and how much did it cost?" for any session. Validated with a complete multi-turn coding task ("build me a tetris game") across all three backends.
9898

9999
As you can see in the results below.
100100

@@ -170,7 +170,7 @@ Each span captures: tool name, input parameters, output/result, and per-span lat
170170

171171
---
172172

173-
## RHAIENG-4754 — Observability Setup Guide & RHOAI 3.5 Recommendation
173+
## Observability Setup Guide & RHOAI 3.5 Recommendation
174174

175175
### Summary
176176

@@ -180,7 +180,7 @@ MLflow integration works. This guide documents how to hook Claude Code, OGX, and
180180

181181
The following must already be running on the cluster:
182182

183-
- Claude Code container deployed (see [PR #92](https://github.com/red-hat-data-services/agentic-starter-kits/pull/92))
183+
- Claude Code container deployed (see [agents/claude/claude_agent](https://github.com/red-hat-data-services/agentic-starter-kits/tree/main/agents/claude/claude_agent))
184184
- OGX deployed and serving a model
185185
- MLflow instance running via the ODH/RHOAI operator with a workspace matching your namespace
186186

@@ -193,7 +193,7 @@ The ODH build of MLflow uses the Red Hat fork which includes the `kubernetes-nam
193193
```dockerfile
194194
RUN microdnf install -y python3.12 python3.12-pip
195195
RUN python3.12 -m pip install --no-cache-dir \
196-
'mlflow[kubernetes] @ git+https://github.com/red-hat-data-services/mlflow.git@rhoai-3.4'
196+
'mlflow[kubernetes] @ git+https://github.com/red-hat-data-services/mlflow.git@v3.10.1+rhaiv.3'
197197
```
198198

199199
> This fork requirement will go away when RHOAI ships MLflow 3.11, at which point replace with `mlflow[kubernetes]>=3.11`.
@@ -227,7 +227,7 @@ oc adm policy add-role-to-user edit -z default -n <your-namespace>
227227
- name: ANTHROPIC_BASE_URL
228228
value: "https://<your-ogx-route>"
229229
- name: ANTHROPIC_API_KEY
230-
value: "fake" # OGX does not validate API keys for self-hosted models, any non-empty string works
230+
value: "not-needed" # OGX does not validate API keys for self-hosted models, any non-empty string works
231231
- name: ANTHROPIC_CUSTOM_MODEL_OPTION
232232
value: "vllm/<your-model-name>"
233233
```

0 commit comments

Comments
 (0)