Skip to content

quickstart.md

Latest commit

 

History

History
132 lines (86 loc) · 3.09 KB

quickstart.md

File metadata and controls

132 lines (86 loc) · 3.09 KB

Quick Start Guide

This guide gets the LLMTrace proxy running and verifies that traces and findings are captured. All steps below map to the current codebase.

Prerequisites

  • OpenAI API key (or another OpenAI-compatible upstream)
  • Either Rust toolchain or Docker

Environment Setup

Set your LLM provider API key before running the proxy:

export OPENAI_API_KEY="sk-..."

The proxy forwards this key to the upstream provider. Without it, proxied requests will fail with 401 Unauthorized.

Option A: One-line Install (Fastest)

curl -sS https://raw.githubusercontent.com/epappas/llmtrace/main/scripts/install.sh | bash

This downloads the latest binary for your platform and a starter config.yaml. Then run:

llmtrace-proxy --config config.yaml

Option B: Cargo Install

cargo install llmtrace
cp config.example.yaml config.yaml
llmtrace-proxy --config config.yaml

Option C: Docker (GHCR)

docker pull ghcr.io/epappas/llmtrace-proxy:latest
docker run -p 8080:8080 ghcr.io/epappas/llmtrace-proxy:latest

Option D: From Source

git clone https://github.com/epappas/llmtrace
cd llmtrace
cargo build --release --bin llmtrace-proxy --features ml
cp config.example.yaml config.yaml
./target/release/llmtrace-proxy --config config.yaml

The --features ml flag enables the DeBERTa ML-based security detectors. Without it, only regex-based detection is available.

Optional: Start Infra + Dashboard (Docker Compose)

compose.yaml starts ClickHouse/Postgres/Redis and the dashboard. Run the proxy separately.

cp .env.example .env
docker compose up -d

# Then run the proxy (from source or Docker)

Test the Proxy

1) Send a test request

curl http://localhost:8080/v1/chat/completions \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "gpt-4", "messages": [{"role": "user", "content": "Hello, LLMTrace!"}]}'

You should receive a normal OpenAI-compatible response.

2) Fetch traces

curl http://localhost:8080/api/v1/traces | jq '.[0]'

3) Fetch security findings

curl http://localhost:8080/api/v1/security/findings | jq

Quick Integration Example (OpenAI SDK)

import openai

client = openai.OpenAI(
    base_url="http://localhost:8080/v1",
    api_key="your-key"
)

response = client.chat.completions.create(
    model="gpt-4",
    messages=[{"role": "user", "content": "Hello"}]
)

Troubleshooting

Symptom Cause Fix
401 Unauthorized from upstream Missing or invalid API key export OPENAI_API_KEY="sk-..."
connection refused on :8080 Proxy not running Check proxy logs, ensure --config config.yaml is passed
No security findings ML features not compiled Rebuild with --features ml (source builds only)
config.yaml not found Missing config file cp config.example.yaml config.yaml

Next Steps