Skip to content

Commit ea18e4f

Browse files
committed
add cli info [do not merge until mar 4th]
1 parent 9e40a4e commit ea18e4f

File tree

2 files changed

+166
-0
lines changed

2 files changed

+166
-0
lines changed

src/docs.json

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -191,6 +191,7 @@
191191
{
192192
"group": "Additional resources",
193193
"pages": [
194+
"langsmith/langsmith-cli",
194195
"langsmith/polly",
195196
{
196197
"group": "Data management",

src/langsmith/langsmith-cli.mdx

Lines changed: 165 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,165 @@
1+
---
2+
title: LangSmith CLI
3+
sidebarTitle: LangSmith CLI
4+
description: Query and manage LangSmith projects, traces, runs, datasets, evaluators, experiments, and threads from the terminal
5+
---
6+
7+
The LangSmith CLI is a fast, agent-friendly command-line tool for working with your LangSmith data and workflows directly from the terminal. It’s designed for both humans and AI coding agents to list, filter, retrieve, and export data — with predictable JSON output by default and a pretty table mode for humans.
8+
9+
<Callout icon="terminal" color="#504B5F">
10+
Built for agents and scripts: defaults to JSON, supports clean stdout/stderr separation, and offers `--yes` flags for non-interactive use.
11+
</Callout>
12+
13+
## Installation
14+
15+
Choose any method below.
16+
17+
<CodeGroup>
18+
```bash Install script (recommended)
19+
curl -sSL https://raw.githubusercontent.com/langchain-ai/langsmith-cli/main/scripts/install.sh | sh
20+
```
21+
22+
```bash Install to a custom directory
23+
INSTALL_DIR=$HOME/.local/bin \
24+
curl -sSL https://raw.githubusercontent.com/langchain-ai/langsmith-cli/main/scripts/install.sh | sh
25+
```
26+
27+
```bash GitHub Releases
28+
# Download the latest binary for your platform
29+
# https://github.com/langchain-ai/langsmith-cli/releases
30+
```
31+
32+
```bash Go install
33+
go install github.com/langchain-ai/langsmith-cli/cmd/langsmith@latest
34+
```
35+
</CodeGroup>
36+
37+
## Authentication
38+
39+
Set your API key as an environment variable:
40+
41+
```bash
42+
export LANGSMITH_API_KEY="lsv2_..."
43+
```
44+
45+
Optional defaults:
46+
47+
```bash
48+
export LANGSMITH_ENDPOINT="https://api.smith.langchain.com" # self-hosted/hybrid
49+
export LANGSMITH_PROJECT="my-default-project" # default project for queries
50+
```
51+
52+
Or pass them as flags when running commands:
53+
54+
```bash
55+
langsmith --api-key lsv2_... trace list --project my-app
56+
```
57+
58+
## Quickstart
59+
60+
Common tasks to get oriented:
61+
62+
```bash
63+
# List tracing projects (sessions)
64+
langsmith project list
65+
66+
# List recent traces in a project
67+
langsmith trace list --project my-app --limit 5
68+
69+
# Get a specific trace with full detail
70+
langsmith trace get <trace-id> --project my-app --full
71+
72+
# List LLM runs with token counts
73+
langsmith run list --project my-app --run-type llm --include-metadata
74+
75+
# Datasets and experiments
76+
langsmith dataset list
77+
langsmith experiment list --dataset my-eval-set
78+
79+
# Conversation threads in a project
80+
langsmith thread list --project my-chatbot
81+
```
82+
83+
## Output formats
84+
85+
- Default: JSON to stdout for easy piping and scripting
86+
- Pretty tables: `--format pretty` for human-readable tables and trees
87+
- Write to file: `-o <path>`
88+
89+
```bash
90+
langsmith trace list --project my-app # JSON array to stdout
91+
langsmith --format pretty trace list --project my-app # tables/trees
92+
langsmith trace list --project my-app -o traces.json # write JSON to file
93+
```
94+
95+
## Commands overview
96+
97+
The CLI groups functionality by resource. Each command supports filters like `--limit`, `--last-n-minutes`, and more.
98+
99+
<AccordionGroup>
100+
<Accordion title="project — list tracing projects" icon="folders">
101+
102+
```bash
103+
langsmith project list # default limit: 20
104+
langsmith project list --name-contains chatbot
105+
langsmith --format pretty project list
106+
```
107+
</Accordion>
108+
109+
<Accordion title="trace — query and export traces" icon="route">
110+
111+
```bash
112+
langsmith trace list --project my-app --limit 50 --last-n-minutes 60
113+
langsmith trace list --project my-app --error --include-metadata
114+
langsmith trace get <trace-id> --project my-app --full
115+
langsmith trace export ./traces --project my-app --limit 20 --full
116+
```
117+
</Accordion>
118+
119+
<Accordion title="run — query individual runs" icon="list-details">
120+
121+
```bash
122+
langsmith run list --project my-app --run-type llm
123+
langsmith run list --project my-app --run-type tool --name search
124+
langsmith run get <run-id> --full
125+
langsmith run export llm_calls.jsonl --project my-app --run-type llm --full
126+
```
127+
</Accordion>
128+
129+
<Accordion title="thread — query conversation threads" icon="messages">
130+
131+
```bash
132+
langsmith thread list --project my-chatbot --last-n-minutes 120
133+
langsmith thread get <thread-id> --project my-chatbot --full
134+
```
135+
</Accordion>
136+
137+
<Accordion title="dataset — manage evaluation datasets" icon="database">
138+
139+
```bash
140+
langsmith dataset list --name-contains eval
141+
langsmith dataset get my-dataset
142+
langsmith dataset create --name my-eval-set --description "QA pairs for v2"
143+
langsmith dataset export my-dataset ./data.json --limit 500
144+
```
145+
</Accordion>
146+
147+
<Accordion title="evaluator — manage evaluators" icon="scale">
148+
149+
```bash
150+
langsmith evaluator list
151+
langsmith evaluator upload evals.py --name accuracy --function check_accuracy --dataset my-eval-set
152+
langsmith evaluator delete accuracy --yes
153+
```
154+
</Accordion>
155+
156+
<Accordion title="experiment — results and summaries" icon="chart-bar">
157+
158+
```bash
159+
langsmith experiment list --dataset my-eval-set
160+
langsmith experiment get my-experiment-2024-01-15
161+
```
162+
</Accordion>
163+
</AccordionGroup>
164+
165+

0 commit comments

Comments
 (0)