Refresh README with AgentPond positioning and setup

marcusschiesser · marcusschiesser · commit e654c9c5d37b · 2026-06-18T16:41:43.000+08:00
diff --git a/README.md b/README.md
@@ -1,46 +1,93 @@
 <p align="center">
-  <img src="https://raw.githubusercontent.com/marcusschiesser/agentpond/main/docs/assets/agentpond-logo-gpt-image.png" alt="AgentPond - trace analytics for AI agents" width="720">
+  <img src="https://raw.githubusercontent.com/marcusschiesser/agentpond/main/docs/assets/agentpond-logo-gpt-image.png" alt="AgentPond — private trace storage and local analytics for AI agents" width="720">
 </p>
 
-AgentPond is a data pond for AI agent traces and human annotations with a agent-native CLI for local analytics. It accepts Langfuse SDK ingestion and its using the same data format, so you can use it as a drop-in replacement.
+<p align="center">
+  <strong>Store agent traces remotely. Analyze them locally. Keep control of the data.</strong>
+</p>
+
+AgentPond is a lightweight, self-hosted trace backend and CLI for AI agents. It accepts traces from Langfuse SDKs and OTLP, stores raw events in S3-compatible object storage, and syncs them into a local DuckDB cache for fast analysis.
+
+It is designed for AI projects that want to:
+
+- keep production traces and human annotations in their own infrastructure
+- inspect traces through a CLI or SQL
+- let coding agents analyze failures and propose regression tests
+- avoid operating a full observability platform
+
+AgentPond provides Langfuse-compatible ingestion endpoints, so supported Langfuse SDK integrations can send traces to AgentPond by changing their base URL and credentials.
+
+> AgentPond is an alternative ingestion backend, not a replacement for the complete Langfuse product. It does not provide Langfuse's web UI, prompt management, datasets, or evaluation workflows.
+
+## How it works
+
+```mermaid
+flowchart LR
+    A[Agent or application] --> B[Langfuse SDK or OTLP]
+    B --> C[AgentPond ingestion service]
+    C --> D[S3-compatible object storage]
+    D -->|agentpond sync| E[Local DuckDB cache]
+    E --> F[CLI and SQL]
+    F --> G[Developers and coding agents]
+```
+
+S3-compatible object storage is the source of truth. The local DuckDB database is a rebuildable cache optimized for fast analytical queries.
+
+This gives you durable remote storage without requiring an always-on analytical database. Developers and coding agents can sync the latest production data, query it locally, identify recurring failures, and use those findings to improve the agent.
 
-It is for AI projects that want to privately store valuable traces, analyze them automatically with a coding agent, and avoid operating a Kubernetes cluster or sending trace data to a public cloud. You just need a S3-compatible object storage with a serverless ingestion service.
+## Features
 
-Events from the object storage are synced into a local DuckDB cache for fast analytical queries of your production agent data. Coding agents use these queries to generate regression tests and improvements of your agents.
+- Langfuse-compatible ingestion endpoints for SDK and OTLP traces
+- S3-compatible raw event storage
+- Manifest-based discovery and incremental synchronization
+- Local DuckDB cache containing:
+  - `events_raw`
+  - `traces`
+  - `observations`
+  - `scores`
+  - `sessions`
+- CLI commands to create, list, and inspect traces, observations, sessions, and scores
+- Human annotations represented as scores
+- Custom SQL queries against synchronized trace data
 
-## Scope
+## Intentional non-goals
 
-AgentPond focuses on collecting and analysing agent traces:
+AgentPond deliberately does not include:
 
-- Langfuse-compatible ingestion endpoints for SDK traces and OTLP traces
-- S3-compatible raw event storage with manifest-based discovery
-- Local DuckDB cache with `events_raw`, `traces`, `observations`, `scores`, and `sessions`
-- CLI commands to create list/read traces, observations, scores (e.g. annotations), and to run custom SQL
+- **A web UI:** developers and coding agents interact with the CLI and SQL
+- **Always-on databases:** no Postgres, ClickHouse, or Redis
+- **Background processing infrastructure:** no worker queues
+- **Prompt or dataset management:** prompts, test cases, and generated evaluations can remain in your Git repository
+- **Hosted trace storage:** you choose and control the object-storage infrastructure
 
-Compared to Langfuse there's no: 
-- Web UI: Your coding agent talks directly to the CLI
-- Postgres, ClickHouse, Redis or background queues: Simplified infrastructure by just using S3 with one ingestion service
-- Prompts, datasets, evals: Let your coding agent generate and store them in GitHub
+A full observability platform is a better choice when you need shared dashboards, hosted evaluation workflows, or non-technical team access.
 
-## Getting Started 
+AgentPond is intended for projects whose main requirement is private, durable trace storage with programmable local analysis.
+
+## Quick start
 
 ### Prerequisites
 
-Install the AgentPond CLI from npm:
+You need:
+
+- Node.js and npm
+- Docker with Docker Compose
+
+Install the AgentPond CLI:
 
 ```sh
 npm install -g agentpond
 ```
 
-Start MinIO (S3-compatible storage) and the ingestion service:
+Start the local MinIO object store and AgentPond ingestion service:
 
 ```sh
 docker compose up --build
 ```
 
-### Test the CLI
+### Configure the CLI
 
-Configure the CLI for the local MinIO endpoint:
+Configure AgentPond to use the local MinIO instance:
 
 ```sh
 export AGENTPOND_S3_ENDPOINT=http://localhost:9000
@@ -49,7 +96,9 @@ export AWS_ACCESS_KEY_ID=minio
 export AWS_SECRET_ACCESS_KEY=minio123
 ```
 
-> Note: The CLI is directly accessing the remote object storage
+> The CLI accesses S3-compatible object storage directly. The credentials above are for local development only.
+
+### Create and query a trace
 
 Create a test trace directly through the CLI:
 
@@ -63,77 +112,115 @@ agentpond traces create \
   --output '{"answer":"The bank declined the authorization. Please try another card or contact your bank."}'
 ```
 
-Sync S3 data into DuckDB and query it:
+Synchronize the stored events into the local DuckDB cache:
 
 ```sh
 agentpond sync
+```
+
+Inspect the synchronized data:
+
+```sh
 agentpond traces list
 agentpond sql "select id, name, session_id from traces"
 ```
 
-For the full command reference, see [CLI usage](./docs/cli.md).
+For the complete command reference, see [CLI usage](./docs/cli.md).
 
-### Test Ingestion
+## Send traces from a Langfuse SDK
 
-AgentPond's ingestion service is using the same API as Langfuse. To point a Langfuse SDK to the AgentPond service started in the prerequisites, set these environment variables:
+AgentPond implements Langfuse-compatible ingestion endpoints. To send traces to the local service, configure your Langfuse SDK with:
 
 ```sh
 export LANGFUSE_BASE_URL=http://localhost:3030
 export LANGFUSE_PUBLIC_KEY=pk-agentpond
 export LANGFUSE_SECRET_KEY=sk-agentpond
 ```
 
-Then you can use the normal Langfuse SDK setup for your language and framework. 
+You can then use the normal Langfuse SDK integration for your language or framework.
+
+See the [Langfuse SDK overview](https://langfuse.com/docs/observability/sdk/overview) for SDK installation and instrumentation instructions.
+
+### Example projects
+
+The repository contains matching Python and TypeScript examples. Each example emits:
 
-We provide matching Python and TypeScript examples using the Langfuse SDK. Both send one realistic trace with generation cost details and one human annotation score. To keep dependencies simple, the examples do not call an LLM; they directly call the Langfuse SDK with fixture inputs, outputs, usage, and cost data. To run the Python example with `uv`:
+- one trace
+- one generation observation with usage and cost data
+- one human annotation score
+
+The examples use fixture data and do not call an LLM, so no model-provider API key is required.
+
+Run the Python example with `uv`:
 
 ```sh
 uv run --project examples/python python examples/python/send_traces.py
 ```
 
-Run the TypeScript SDK example:
+Run the TypeScript example:
 
 ```sh
 pnpm --dir examples/typescript start
 ```
 
-Each example prints the trace ID it created and the `agentpond` commands to inspect trace, observations, and human annotation score. Call them to analyze the generated trace.
+Each example prints the generated trace ID and the corresponding `agentpond` commands for inspecting its trace, observations, and annotation score.
 
-To add AgentPond to your own project, add the latest Langfuse SDK, the docs are here:
+## Use AgentPond in your project
 
-- [Langfuse SDK overview](https://langfuse.com/docs/observability/sdk/overview)
+Install the latest Langfuse SDK for your language and configure it to point to your AgentPond deployment:
 
-Make sure to set the `LANGFUSE_BASE_URL`, `LANGFUSE_PUBLIC_KEY` and `LANGFUSE_SECRET_KEY` variables in your project to point to a deployment of the [ingestion containers](./docker-compose.yml) in your own infrastructure.
+```sh
+export LANGFUSE_BASE_URL=https://your-agentpond-ingestion.example.com
+export LANGFUSE_PUBLIC_KEY=your-public-key
+export LANGFUSE_SECRET_KEY=your-secret-key
+```
+
+Deploy the [ingestion service](./docker-compose.yml) in your infrastructure and configure it to use your S3-compatible object store.
+
+For production deployments, replace all example credentials, use TLS, and give the ingestion service and CLI only the object-storage permissions they require.
 
 ## Configuration
 
-The ingestion service and CLI read environment variables:
+The ingestion service and CLI use the following environment variables:
 
 ```sh
+# Project
 AGENTPOND_PROJECT_ID=default-project
+
+# Ingestion authentication
 LANGFUSE_PUBLIC_KEY=pk-agentpond
 LANGFUSE_SECRET_KEY=sk-agentpond
+
+# Object storage
 AGENTPOND_S3_BUCKET=agentpond
 AGENTPOND_S3_PREFIX=
 AGENTPOND_S3_ENDPOINT=http://localhost:9000
 AGENTPOND_S3_FORCE_PATH_STYLE=true
+
+# AWS or S3-compatible credentials
 AWS_ACCESS_KEY_ID=minio
 AWS_SECRET_ACCESS_KEY=minio123
-AWS_REGION=us-east-1 # optional, defaults to us-east-1
+AWS_REGION=us-east-1
+
+# Local analytical cache
 AGENTPOND_DB=~/.agentpond/cache.duckdb
 ```
 
-If `--s3-endpoint` and `AGENTPOND_S3_ENDPOINT` are both omitted, AgentPond uses the default AWS S3 endpoint for `AWS_REGION`. For local MinIO, set `AGENTPOND_S3_ENDPOINT` or pass `--s3-endpoint`.
+`AWS_REGION` defaults to `us-east-1`.
+
+When `--s3-endpoint` and `AGENTPOND_S3_ENDPOINT` are both omitted, AgentPond uses the standard AWS S3 endpoint for the configured region.
+
+For local MinIO or another S3-compatible service, set `AGENTPOND_S3_ENDPOINT` or pass `--s3-endpoint` to the CLI.
 
 ## Development
 
-Install workspace dependencies once before running tests, examples, or local development commands:
+Install the workspace dependencies:
 
 ```sh
-pnpm i
+pnpm install
 ```
 
-Instead of `agentpond`, run the CLI from source code:
+Run the CLI directly from the source tree:
 
 ```sh
 pnpm cli --help
