Skip to content

Commit 311352a

Browse files
katmaybkatmayb
committed
Add docs for LangSmith Fetch
1 parent 94aeee4 commit 311352a

File tree

3 files changed

+219
-0
lines changed

3 files changed

+219
-0
lines changed

‎src/docs.json‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -961,6 +961,7 @@
961961
"langsmith/export-traces",
962962
"langsmith/compare-traces",
963963
"langsmith/share-trace",
964+
"langsmith/langsmith-fetch",
964965
"langsmith/platform-logs",
965966
"langsmith/data-export"
966967
]

‎src/langsmith/images/langsmith-fetch-output-single.png‎

209 KB
Loading

‎src/langsmith/langsmith-fetch.mdx‎

Lines changed: 218 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,218 @@
1+
---
2+
title: LangSmith Fetch
3+
---
4+
5+
LangSmith Fetch is a command-line interface (CLI) tool for retrieving trace data ([runs](/langsmith/observability-concepts#runs), [traces](/langsmith/observability-concepts#traces), and [threads](/langsmith/observability-concepts#threads)) from your LangSmith projects. It allows you to use LangSmith’s tracing and debugging features directly in your terminal and development workflows.
6+
7+
You can use LangSmith Fetch for the following use cases:
8+
9+
- Immediate debugging: Fetch the most recent trace of a failed or unexpected agent run with a single command.
10+
- Bulk export for analysis: Export large numbers of traces or entire conversation threads to JSON files for offline analysis, building [evaluation](/langsmith/evaluation-concepts) datasets, or [regression tests](/langsmith/evaluation-types#regression-tests).
11+
- Terminal-based workflows: Integrate trace data into your existing tools; for example, piping output to Unix utilities like jq, or feeding traces into an AI coding assistant for automated analysis.
12+
13+
## Installation
14+
15+
```bash
16+
pip install langsmith-fetch
17+
```
18+
19+
## Setup
20+
21+
Set your [LangSmith API key](/langsmith/create-account-api-key) and project name:
22+
23+
```bash
24+
export LANGSMITH_API_KEY=lsv2_...
25+
export LANGSMITH_PROJECT=your-project-name
26+
```
27+
28+
The CLI will automatically fetch traces or threads in `LANGSMITH_PROJECT`. Replace `your-project-name` with the name of your LangSmith project (if it doesn't exist, it will be created automatically on first use).
29+
30+
`langsmith-fetch` only requires the `LANGSMITH_PROJECT` environment variable. It automatically looks up the project UUID and saves both to `~/.langsmith-cli/config.yaml`.
31+
32+
You can also specify [a project by its UUID](#override-the-configured-tracing-project) via a CLI flag.
33+
34+
### Use a coding agent with `langsmith-fetch`
35+
36+
After you've installed and set up `langsmith-fetch`, use your coding agent to ask questions like the following:
37+
38+
> _Use langsmith-fetch to analyze the last 3 threads from my LangSmith project for potential improvements_
39+
40+
Many agents will use the `langsmith-fetch --help` command to understand how to use the CLI and complete your request.
41+
42+
## Find project and trace IDs
43+
44+
In most cases, you won’t need to find IDs manually (the CLI uses your project name and latest traces by default). However, if you want to fetch a specific item by ID, you can find them in the [LangSmith UI](https://smith.langchain.com):
45+
46+
- **Project UUID**: Each project has a unique ID (UUID). You can find it in the project's URL or by hovering over <Icon icon="link-simple"/> **ID** next to the project's name. This UUID can be used with the `--project-uuid` flag on CLI commands
47+
- **Trace ID**: Every trace (single execution) has an ID. In the **Runs** view, click on a specific run to see its Trace ID (copyable from the trace details panel). You can use `langsmith-fetch trace <trace-id>` to retrieve that exact trace if you have the ID.
48+
49+
## Usage
50+
51+
After installation and setup, you can use the `langsmith-fetch` command to retrieve traces or threads. The general usage is:
52+
53+
```bash
54+
langsmith-fetch COMMAND [ARGUMENTS] [OPTIONS]
55+
```
56+
57+
LangSmith Fetch provides the following commands to fetch either single items or in bulk:
58+
59+
| Command | Fetches | Output location |
60+
| --------------------- | ------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
61+
| `trace <id>` | A specific trace by ID | Prints to stdout (or to a file with `--file`) |
62+
| `thread <id>` | A specific thread by ID | Prints to stdout (or to a file with `--file`) |
63+
| `traces [directory]` | Recent traces from the project (multiple) | Saves each trace as a JSON file in the given directory, or prints to stdout if no directory is provided. **Tip:** Using a directory is recommended for [bulk exports](#fetch-multiple). |
64+
| `threads [directory]` | Recent threads from the project (multiple) | Saves each thread as a JSON file in the given directory, or prints to stdout if no directory is provided. |
65+
66+
<Note>
67+
Traces are fetched chronologically with most recent first.
68+
</Note>
69+
70+
## Options
71+
72+
The commands support additional flags to filter and format the output:
73+
74+
| Option / Flag | Applies to | Description | Default |
75+
| --------------------------- | -------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------- |
76+
| `-n, --limit <int>` | `traces`, `threads` | Maximum number of traces/threads to fetch. Use this to limit how many items you retrieve (e.g., the last 5 traces). | **1** (if not specified) |
77+
| `--last-n-minutes <int>` | `traces`, `threads` | Only fetch items from the last N minutes. This is useful to get recent data (e.g.,`--last-n-minutes 30` for the past half hour). | *(no time filter)* |
78+
| `--since <timestamp>` | `traces`, `threads` | Only fetch items since a specific time. Provide an ISO 8601 timestamp (e.g.,`2025-12-01T00:00:00Z`) to get data after that time. | *(no time filter)* |
79+
| `--project-uuid <uuid>` | `trace, thread, traces, threads` | Manually specify the project by UUID (overrides the `LANGSMITH_PROJECT` env setting). Use this if you want to fetch from a different project without changing your env var. | From env/config |
80+
| `--filename-pattern <text>` | `traces`, `threads` | Pattern for output filenames when saving multiple files. You can use placeholders like `{trace_id}`, `{thread_id}`, `{index}`. | `{trace_id}.json` or `{thread_id}.json` |
81+
| `--format <type>` | **All commands** | Output format: `pretty`, `json`, or `raw`. (Refer to [Output formats](#output-formats) for details.) | `pretty` |
82+
| `--file <path>` | `trace`, `thread` | Save the fetched trace/thread to a file instead of printing it. | *(stdout)* |
83+
| `--include-metadata` | `traces` (bulk fetch) | Include run metadata in the output (such as tokens used, execution time, status, costs). This will add a `"metadata"` section to each trace’s JSON. | *Off by default* |
84+
| `--include-feedback` | `traces` (bulk fetch) | Include any feedback entries attached to the runs. Enabling this will make an extra API call for each trace to fetch feedback data. | *Off by default* |
85+
| `--max-concurrent <int>` | `traces`, `threads` | Maximum concurrent fetch requests. Tune this if you are fetching a large number of items; increasing it may speed up retrieval but 5–10 is recommended to avoid API overload. | **5** |
86+
| `--no-progress` | `traces`, `threads` | Disable the progress bar output. By default a progress indicator is shown when fetching multiple items; use this flag to hide it (useful for non-interactive scripts). | Progress bar on |
87+
88+
## Output formats
89+
90+
The `--format` option controls how the fetched data is displayed:
91+
92+
- `pretty` (default): A human-readable view with rich text formatting for easy inspection in the terminal. This format is great for quick debugging of a single trace or thread.
93+
94+
By default:
95+
96+
```bash
97+
langsmith-fetch trace <trace-id>
98+
```
99+
Explicitly specify the format:
100+
101+
```bash
102+
langsmith-fetch trace <trace-id> --format pretty
103+
```
104+
105+
- `json`: Well-formatted JSON output with syntax highlighting. Use this if you want to examine the raw data structure or pipe it into JSON processing tools.
106+
107+
```bash
108+
langsmith-fetch trace <trace-id> --format json
109+
```
110+
111+
- `raw`: Compact JSON with no extra whitespace. This is useful for piping the output to other programs (e.g., using `jq` or saving directly) without extra formatting.
112+
113+
```bash
114+
langsmith-fetch trace <trace-id> --format raw | jq '.[] | select(.role=="user")'
115+
```
116+
117+
## Examples
118+
119+
### Fetch a trace or thread
120+
121+
You can fetch a single thread or trace with the ID. The command will output to the terminal by default:
122+
123+
```bash
124+
langsmith-fetch trace <trace-id>
125+
```
126+
```bash
127+
langsmith-fetch thread <thread-id>
128+
```
129+
130+
![Output from a single trace fetch in default pretty format](/langsmith/images/langsmith-fetch-output-single.png)
131+
132+
You can optionally redirect the thread or trace data to a file using the `--file` option.
133+
134+
### Fetch multiple
135+
136+
<Note>
137+
For bulk fetches of traces or threads, we recommend specifying a target directory path. Each fetched trace or thread will be saved as a separate JSON file in that folder, making it easy to browse or process them later.
138+
</Note>
139+
140+
You can specify a destination directory for the bulk commands (`traces`/`threads`). For example, the following command will save the 10 most recent traces as JSON files in the `my-traces-data` directory:
141+
142+
```bash
143+
langsmith-fetch traces ./my-traces-data --limit 10
144+
```
145+
```bash
146+
langsmith-fetch threads ./my-thread-data --limit 10
147+
```
148+
149+
If you omit the directory and `--limit`, the tool will output the results of the most recent, single trace to your terminal.
150+
151+
When sending to a directory, files will be named in the following way:
152+
153+
- Default: Files named by trace ID (e.g., `3b0b15fe-1e3a-4aef-afa8-48df15879cfe.json`).
154+
- Custom pattern: Use `--filename-pattern` with placeholders:
155+
- `{trace_id}`: Trace ID (default: `{trace_id}.json`).
156+
- `{index}` or `{idx}`: Sequential number starting from 1.
157+
- Format specs supported: `{index:03d}` for zero-padded numbers.
158+
159+
### Include metadata and feedback
160+
161+
You can include [run metadata](/langsmith/observability-concepts#metadata) and any [feedback](/langsmith/observability-concepts#feedback) associated with the trace:
162+
163+
```bash
164+
langsmith-fetch traces --limit 1 --include-metadata --include-feedback
165+
```
166+
```
167+
RUN METADATA
168+
============================================================
169+
Status: success
170+
Start Time: 2025-12-12T18:05:47.558274
171+
End Time: 2025-12-12T18:05:48.811072
172+
Duration: 1252ms
173+
174+
Token Usage:
175+
Prompt: 15
176+
Completion: 88
177+
Total: 103
178+
179+
Costs:
180+
Total: $0.00014
181+
Prompt: $0.00001
182+
Completion: $0.00013
183+
184+
Custom Metadata:
185+
LANGSMITH_PROJECT: weather-demo
186+
LANGSMITH_TRACING: true
187+
ls_run_depth: 0
188+
189+
Feedback Stats:
190+
correctness: {'n': 1, 'avg': 1.0, 'stdev': 0.0, 'errors': 0, 'show_feedback_arrow': False, 'comments': [''], 'sources':
191+
['{"type":"app","metadata":null,"user_id":"d5ee8d42-a274-4f32-9c35-b765287fe5ec","ls_user_id":"ac375f5f-0da0-44c1-82a2-0ecfd6ecac27"}'],
192+
'session_min_score': 1.0, 'session_max_score': 1.0, 'values': {}, 'contains_thread_feedback': False}
193+
note: {'n': 0, 'avg': None, 'stdev': None, 'errors': 0, 'show_feedback_arrow': False, 'comments': ['The answer should be more specific on rainfall
194+
totals.'], 'sources':
195+
['{"type":"app","metadata":null,"user_id":"d5ee8d42-a274-4f32-9c35-b765287fe5ec","ls_user_id":"ac375f5f-0da0-44c1-82a2-0ecfd6ecac27"}'],
196+
'session_min_score': None, 'session_max_score': None, 'values': {}, 'contains_thread_feedback': False}
197+
...
198+
```
199+
200+
### Override the configured tracing project
201+
202+
To fetch traces from a different project than the one configured with `LANGSMITH_PROJECT`, use the `--project-uuid` option:
203+
204+
```bash
205+
langsmith-fetch traces --project-uuid <project-id> --limit 3
206+
```
207+
208+
Running this command will just fetch traces from that project, it will not modify the LangSmith project already configured in `~/.langsmith-cli/config.yaml`.
209+
210+
### Export to files
211+
212+
You can fetch traces or full threads and export to a file:
213+
214+
```bash
215+
langsmith-fetch threads ./my_threads --since 2025-12-01T00:00:00Z
216+
```
217+
218+
This command retrieves all threads that have occurred since December 1, 2025, saving each conversation as a JSON file under `./my_threads`. This is useful for exporting chat transcripts or building regression tests on multi-turn conversations. You could also use `--limit` with threads to fetch a specific number of recent threads, and `--last-n-minutes` works here as well.

0 commit comments

Comments
 (0)