The sample dataset contains a release change with slow generations, one provider error, and weaker answer-quality scores. The goal is to connect a symptom to request-level evidence rather than infer a cause from one aggregate chart.
Source: Figure 5 from the scientific paper Preble: Efficient Distributed Prompt Scheduling for LLM Serving.
uv run python -m observability_lab summarize
uv run python -m observability_lab diagnoseCompare releases v1.0.0 and v1.1.0. Identify the first metric that changed,
the stage responsible for latency, and the traces that deserve inspection.
The degraded requests are concentrated in v1.1.0. Retrieval remains fast;
generation dominates the high-latency traces. Quality falls on two responses
and one request fails completely. The evidence supports investigating provider
behavior and release-specific prompt or model changes before scaling the API.