Skip to content

installation.md

Latest commit

 

History

History
163 lines (108 loc) · 4.01 KB

installation.md

File metadata and controls

163 lines (108 loc) · 4.01 KB

Installation Guide

This guide covers supported ways to run LLMTrace as implemented in this repository. All instructions are based on the current codebase and bundled files.

Options

Method Best For Notes
Install script Quickest install Downloads binary + config in one command
cargo install Rust users Pre-built from crates.io
pip install Python SDK Pre-built from PyPI
Docker (GHCR) Containerized runs Pre-built multi-arch image from GHCR
From source Development Uses local Rust toolchain
Docker Compose Local infra/dev stack Starts ClickHouse/Postgres/Redis and dashboard
Helm chart Kubernetes clusters Chart under deployments/helm/llmtrace

Install Script (Recommended)

Downloads the latest binary for your platform (Linux amd64, macOS arm64) and a starter config:

curl -sS https://raw.githubusercontent.com/epappas/llmtrace/main/scripts/install.sh | bash

Options via environment variables:

LLMTRACE_VERSION=0.1.3 curl -sS .../install.sh | bash   # pin version
LLMTRACE_INSTALL=~/bin curl -sS .../install.sh | bash    # custom install dir
LLMTRACE_NO_CONFIG=1 curl -sS .../install.sh | bash      # skip config download

Cargo Install (Proxy)

Install the proxy binary directly from crates.io:

cargo install llmtrace

# Run with a config file
cp config.example.yaml config.yaml
llmtrace-proxy --config config.yaml

Pip Install (Python SDK)

Install the Python SDK from PyPI (imports as import llmtrace):

pip install llmtracing
import llmtrace

tracer = llmtrace.configure({"enable_security": True})
span = tracer.start_span("chat_completion", "openai", "gpt-4")
span.set_prompt("Hello!")
span.set_response("Hi there!")
print(span.to_dict())

From Source

Prerequisite: Rust 1.93+.

# 1. Clone
git clone https://github.com/epappas/llmtrace
cd llmtrace

# 2. Build (--features ml enables DeBERTa ML detectors)
cargo build --release --bin llmtrace-proxy --features ml

# 3. Run with example config
cp config.example.yaml config.yaml
./target/release/llmtrace-proxy --config config.yaml

Docker (GHCR)

Pre-built multi-arch images (amd64 + arm64) are published to GHCR on every release:

docker pull ghcr.io/epappas/llmtrace-proxy:latest
docker run -p 8080:8080 ghcr.io/epappas/llmtrace-proxy:latest

To customise configuration, mount a config file:

docker run -p 8080:8080 \
  -v $(pwd)/config.yaml:/etc/llmtrace/config.yaml \
  ghcr.io/epappas/llmtrace-proxy:latest

Build Locally

docker build -t llmtrace-proxy .
docker run -p 8080:8080 --env-file .env llmtrace-proxy

Docker Compose (Infra + Dashboard)

compose.yaml starts ClickHouse, PostgreSQL, Redis, and the dashboard UI. It does not run the proxy.

# 1. Copy env file
cp .env.example .env

# 2. Start infra services
docker compose up -d

# 3. Run the proxy separately (example)
cargo run --bin llmtrace-proxy -- --config config.yaml

Notes:

  • The dashboard expects the proxy at http://localhost:8080.
  • Use storage.profile: production with ClickHouse/Postgres/Redis from compose.

Helm (Local Chart)

A Helm chart is included under deployments/helm/llmtrace.

# From repo root
helm install llmtrace ./deployments/helm/llmtrace

Notes:

  • The chart defaults to the GHCR image ghcr.io/epappas/llmtrace-proxy.
  • Override proxy.image.repository and proxy.image.tag in values.yaml if using a custom registry.

Verify the Proxy

# Health check
curl http://localhost:8080/health

# Proxy a test OpenAI-compatible request
curl http://localhost:8080/v1/chat/completions \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "gpt-4", "messages": [{"role": "user", "content": "Hello"}]}'

Next Steps

  • docs/getting-started/quickstart.md
  • docs/getting-started/configuration.md