Skip to content

feat: implement and compare LLM tracing options to Grafana. #269#313

Draft
Azamat0315277 wants to merge 20 commits into
developmentfrom
269-ops-setup-llm-tracing-to-grafana-using-mlflow-autologger
Draft

feat: implement and compare LLM tracing options to Grafana. #269#313
Azamat0315277 wants to merge 20 commits into
developmentfrom
269-ops-setup-llm-tracing-to-grafana-using-mlflow-autologger

Conversation

@Azamat0315277
Copy link
Copy Markdown
Contributor

@Azamat0315277 Azamat0315277 commented Apr 20, 2026 •
edited
Loading

Applicable issue

PR #313 was opened to implement MLflow autologging. The branch evolved significantly — MLflow was replaced with OpenInference LangChainInstrumentor

Description of changes

What was changed

Replaces MLflow autologging with OpenInference LangChainInstrumentor for LLM tracing to Grafana, and includes several improvements merged from development.
LLM Tracing (core change)

  • pyproject.toml - added openinference-instrumentation-langchain;
  • statgpt/app/application/application.py - added LangChainInstrumentor().instrument() inside the FastAPI lifespan
  • .ort.yml - added license curations for new OpenInference transitive dependencies.

Why OpenInference over MLflow

  • MLflow's homegrown ContextVar tracking crashes under asyncio.gather . OpenInference uses context.attach()/detach() which asyncio copies correctly across task boundaries.

Configuration

❗ Set OTEL_EXPORTER_OTLP_TRACES_ENDPOINT environment variable for the Grafana OTLP endpoint.
❗ Set OTEL_SPAN_ATTRIBUTE_VALUE_LENGTH_LIMIT to cap span attribute size (gRPC 4 MB limit).

Checklist

By submitting this pull request, I confirm that my contribution is made under the terms of the MIT license.

Status
Draft

feat: add MLflow LangChain autologging with OTLP tracing.
558cae8 
 Add MLflow as a dependency and configure LangChain autologging for LLM tracing. Set up an OpenTelemetry TracerProvider with a BatchSpanProcessor
  backed by OTLPSpanExporter to forward traces to Grafana.
@Azamat0315277 Azamat0315277 requested a review from navalnica April 20, 2026 11:46
@Azamat0315277 Azamat0315277 self-assigned this Apr 20, 2026
@Azamat0315277 Azamat0315277 linked an issue Apr 20, 2026 that may be closed by this pull request
[ops] setup LLM tracing to grafana (using mlflow autologger) #269
Open
@Azamat0315277
Copy link
Copy Markdown
Contributor Author

Azamat0315277 commented Apr 20, 2026 •
edited by statgpt-actions
Loading

/deploy-review

GitHub actions run: 24668573379

@Azamat0315277
Copy link
Copy Markdown
Contributor Author

Azamat0315277 commented Apr 21, 2026 •
edited by statgpt-actions
Loading

/deploy-review

GitHub actions run: 24711348125

refactor: Move mlflow.langchain.autolog() out of module-level import …
7f119b6 
…in app.py and

  into the FastAPI lifespan, delete the
  manual TracerProvider, which was overriding the provider already registered by aidial-sdk via TelemetryConfig.
@Azamat0315277
Copy link
Copy Markdown
Contributor Author

Azamat0315277 commented Apr 21, 2026 •
edited by statgpt-actions
Loading

/deploy-review

GitHub actions run: 24719738549
Environment URL: review-environment | pipeline

fix: for testing purposes, temporarily enabling MLFLOW_TRACING_ENABLE…
a141a2f 
…D=true in the code; will be reverted after testing.
@Azamat0315277
Copy link
Copy Markdown
Contributor Author

Azamat0315277 commented Apr 21, 2026 •
edited by statgpt-actions
Loading

/deploy-review

GitHub actions run: 24723480453

@Azamat0315277
Copy link
Copy Markdown
Contributor Author

Azamat0315277 commented Apr 21, 2026 •
edited by statgpt-actions
Loading

/deploy-review

GitHub actions run: 24727877396

@Azamat0315277
Copy link
Copy Markdown
Contributor Author

Azamat0315277 commented Apr 23, 2026 •
edited by statgpt-actions
Loading

/deploy-review

GitHub actions run: 24835349514

@Azamat0315277
Copy link
Copy Markdown
Contributor Author

Azamat0315277 commented Apr 23, 2026 •
edited by statgpt-actions
Loading

/deploy-review

GitHub actions run: 24840266179

@Azamat0315277
Copy link
Copy Markdown
Contributor Author

Azamat0315277 commented Apr 24, 2026 •
edited by statgpt-actions
Loading

/deploy-review

GitHub actions run: 24881718554
Environment URL: review-environment | pipeline

@Azamat0315277
Copy link
Copy Markdown
Contributor Author

Azamat0315277 commented Apr 27, 2026 •
edited by statgpt-actions
Loading

/deploy-review

GitHub actions run: 24999420361
Environment URL: review-environment | pipeline

@Azamat0315277
Copy link
Copy Markdown
Contributor Author

Azamat0315277 commented Apr 28, 2026 •
edited by statgpt-actions
Loading

/deploy-review

GitHub actions run: 25035536734
Environment URL: review-environment | pipeline

Azamat Kuzdibayev added 2 commits April 28, 2026 11:44
@Azamat0315277
Copy link
Copy Markdown
Contributor Author

Azamat0315277 commented Apr 28, 2026 •
edited by statgpt-actions
Loading

/deploy-review

GitHub actions run: 25038929231
Environment URL: review-environment | pipeline

@Azamat0315277 Azamat0315277 changed the title feat: add MLflow LangChain LLM tracing to Grafana. feat: implement and compare LLM tracing options to Grafana. Apr 29, 2026
@Azamat0315277 Azamat0315277 changed the title feat: implement and compare LLM tracing options to Grafana. feat: implement and compare LLM tracing options to Grafana. #269 Apr 29, 2026
@MykhailoRyzhman
Copy link
Copy Markdown
Collaborator

MykhailoRyzhman commented May 13, 2026 •
edited by ai-dial-actions
Loading

/deploy-review

GitHub actions run: 25815925949

Stage Status
deploy-review Cancelled 🚫
matrix.application Cancelled 🚫

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@navalnica navalnica Awaiting requested review from navalnica

@ypldan ypldan Awaiting requested review from ypldan ypldan will be requested when the pull request is marked ready for review ypldan is a code owner

At least 1 approving review is required to merge this pull request.

Assignees

@Azamat0315277 Azamat0315277

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[ops] setup LLM tracing to grafana (using mlflow autologger)

2 participants

@Azamat0315277 @MykhailoRyzhman