Open-source observability for your LLM application, based on OpenTelemetry.
LLM Tracekit is a set of OpenTelemetry instrumentations that gives you complete observability over your LLM application. Because it uses OpenTelemetry under the hood, it can be connected to your existing observability solutions - Coralogix, Datadog, Honeycomb, and others.
Install the instrumentation for your LLM provider:
pip install llm-tracekit-openai # For OpenAI
pip install llm-tracekit-bedrock # For AWS Bedrock
pip install llm-tracekit-gemini # For Google Gemini
pip install llm-tracekit-google-adk # For Google ADK
pip install llm-tracekit-litellm # For LiteLLM
pip install llm-tracekit-langchain # For LangChain
pip install llm-tracekit-openai-agents # For OpenAI Agents SDKThen instrument your code:
from llm_tracekit.openai import OpenAIInstrumentor, setup_export_to_coralogix
setup_export_to_coralogix(
service_name="my-ai-service",
capture_content=True,
)
OpenAIInstrumentor().instrument()
from openai import OpenAI
client = OpenAI()
response = client.chat.completions.create(
model="gpt-4o-mini",
messages=[{"role": "user", "content": "Hello!"}],
)| Provider | Package | Instrumentor |
|---|---|---|
| OpenAI | llm-tracekit-openai |
OpenAIInstrumentor |
| AWS Bedrock | llm-tracekit-bedrock |
BedrockInstrumentor |
| Google Gemini | llm-tracekit-gemini |
GeminiInstrumentor |
| Framework | Package | Instrumentor |
|---|---|---|
| Google ADK | llm-tracekit-google-adk |
GoogleADKInstrumentor |
| LiteLLM | llm-tracekit-litellm |
LiteLLMInstrumentor |
| LangChain | llm-tracekit-langchain |
LangChainInstrumentor |
| OpenAI Agents SDK | llm-tracekit-openai-agents |
OpenAIAgentsInstrumentor |
from llm_tracekit.openai import setup_export_to_coralogix
setup_export_to_coralogix(
service_name="ai-service",
application_name="ai-application",
subsystem_name="ai-subsystem",
capture_content=True,
)LLM Tracekit also includes Coralogix Guardrails - a client for protecting your LLM applications with content moderation, PII detection, prompt injection detection, and more.
See the Guardrails documentation for details.
For detailed documentation on each instrumentation, see the individual READMEs:
Apache 2.0 - See LICENSE for details.