Skip to content

Commit 776721c

Browse files
feat(docs): add comprehensive guide for AI agents using NNsight
- Introduced a new documentation file `llms.md` that provides detailed guidance for AI agents working with the NNsight library. - Included sections on core concepts, quick reference, and various functionalities such as tracing, modifying activations, and remote execution. - Enhanced accessibility of NNsight's features for users by linking to related resources and documentation.
1 parent d4447a4 commit 776721c

File tree

2 files changed

+60
-21
lines changed

2 files changed

+60
-21
lines changed

README.md

Lines changed: 59 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -25,12 +25,70 @@ Originally developed in the [NDIF team](https://ndif.us/) at Northeastern Univer
2525

2626
> 📖 For a deeper technical understanding of nnsight's internals (tracing, interleaving, the Envoy system, etc.), see **[NNsight.md](./NNsight.md)**.
2727
28+
---
29+
2830
## Installation
2931

3032
```bash
3133
pip install nnsight
3234
```
3335

36+
---
37+
38+
## Agents
39+
40+
Inform LLM agents how to use nnsight using one of these methods:
41+
42+
### Skills Repository
43+
44+
**Claude Code**
45+
46+
```bash
47+
# Open Claude Code terminal
48+
claude
49+
50+
# Add the marketplace (one time)
51+
/plugin marketplace add https://github.com/ndif-team/skills.git
52+
53+
# Install all skills
54+
/plugin install nnsight@skills
55+
```
56+
57+
**OpenAI Codex**
58+
59+
```bash
60+
# Open OpenAI Codex terminal
61+
codex
62+
63+
# Install skills
64+
skill-installer install https://github.com/ndif-team/skills.git
65+
```
66+
67+
### Context7 MCP
68+
69+
Alternatively, use [Context7](https://github.com/upstash/context7) to provide up-to-date nnsight documentation directly to your LLM. Add `use context7` to your prompts or configure it in your MCP client:
70+
71+
```json
72+
{
73+
"mcpServers": {
74+
"context7": {
75+
"url": "https://mcp.context7.com/mcp"
76+
}
77+
}
78+
}
79+
```
80+
81+
See the [Context7 README](https://github.com/upstash/context7/blob/master/README.md) for full installation instructions across different IDEs.
82+
83+
### Documentation Files
84+
85+
You can also add our documentation files directly to your agent's context:
86+
87+
- **[llms.md](./llms.md)** — Comprehensive guide for AI agents working with nnsight
88+
- **[NNsight.md](./NNsight.md)** — Deep technical documentation on nnsight's internals
89+
90+
---
91+
3492
## Quick Start
3593

3694
```python
@@ -53,8 +111,6 @@ print(model.tokenizer.decode(output.logits.argmax(dim=-1)[0]))
53111

54112
> **💡 Tip:** Always call `.save()` on values you want to access after the trace exits. Without `.save()`, values are garbage collected. You can also use `nnsight.save(value)` as an alternative.
55113
56-
---
57-
58114
## Accessing Activations
59115

60116
```python
@@ -74,8 +130,6 @@ with model.trace("The Eiffel Tower is in the city of"):
74130

75131
**Note:** GPT-2 transformer layers return tuples where index 0 contains the hidden states.
76132

77-
---
78-
79133
## Modifying Activations
80134

81135
### In-Place Modification
@@ -101,8 +155,6 @@ with model.trace("Hello"):
101155
result = model.transformer.h[-1].mlp.output.save()
102156
```
103157

104-
---
105-
106158
## Batching with Invokers
107159

108160
Process multiple inputs in one forward pass. Each invoke runs its code in a **separate worker thread**:
@@ -143,7 +195,6 @@ with model.trace() as tracer:
143195
out_all = model.lm_head.output[:, -1].save() # Shape: [3, vocab]
144196
```
145197

146-
---
147198

148199
## Multi-Token Generation
149200

@@ -193,7 +244,6 @@ with model.generate("Hello", max_new_tokens=5) as tracer:
193244
> final = model.output.save() # Now works!
194245
> ```
195246
196-
---
197247
198248
## Gradients
199249
@@ -213,7 +263,6 @@ with model.trace("Hello"):
213263
print(grad.shape)
214264
```
215265
216-
---
217266

218267
## Model Editing
219268

@@ -236,7 +285,6 @@ assert not torch.all(out1 == 0)
236285
assert torch.all(out2 == 0)
237286
```
238287

239-
---
240288

241289
## Scanning (Shape Inference)
242290

@@ -249,7 +297,6 @@ with model.scan("Hello"):
249297
print(dim) # 768
250298
```
251299

252-
---
253300

254301
## Caching Activations
255302

@@ -264,7 +311,6 @@ layer0_out = cache['model.transformer.h.0'].output
264311
print(cache.model.transformer.h[0].output[0].shape)
265312
```
266313

267-
---
268314

269315
## Sessions
270316

@@ -280,7 +326,6 @@ with model.session() as session:
280326
hs2 = model.transformer.h[0].output[0].save()
281327
```
282328

283-
---
284329

285330
## Remote Execution (NDIF)
286331

@@ -298,7 +343,6 @@ with model.trace("Hello", remote=True):
298343

299344
Check available models at [nnsight.net/status](https://nnsight.net/status/)
300345

301-
---
302346

303347
## vLLM Integration
304348

@@ -316,7 +360,6 @@ with model.trace("Hello", temperature=0.0, max_tokens=5) as tracer:
316360
logits.append(model.logits.output)
317361
```
318362

319-
---
320363

321364
## NNsight for Any PyTorch Model
322365

@@ -338,8 +381,6 @@ with model.trace(torch.rand(1, 5)):
338381
output = model.output.save()
339382
```
340383

341-
---
342-
343384
## Source Tracing
344385

345386
Access intermediate operations inside a module's forward pass. `.source` rewrites the forward method to hook into all operations:
@@ -356,8 +397,6 @@ with model.trace("Hello"):
356397
attn_out = model.transformer.h[0].attn.source.attention_interface_0.output.save()
357398
```
358399

359-
---
360-
361400
## Ad-hoc Module Application
362401

363402
Apply modules out of their normal execution order:
@@ -454,7 +493,7 @@ For more debugging tips, see the [documentation](https://www.nnsight.net).
454493

455494
- **[Documentation](https://www.nnsight.net)** — Tutorials, guides, and API reference
456495
- **[NNsight.md](./NNsight.md)** — Deep technical documentation on nnsight
457-
- **[CLAUDE.md](./CLAUDE.md)** — Comprehensive guide for AI agents working with nnsight
496+
- **[llms.md](./llms.md)** — Comprehensive guide for AI agents working with nnsight
458497

459498
---
460499

CLAUDE.md renamed to llms.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
# CLAUDE.md - NNsight AI Agent Guide
1+
# llms.md - NNsight AI Agent Guide
22

33
This document provides comprehensive guidance for AI agents working with the `nnsight` library. NNsight enables interpreting and manipulating the internals states of deep learning models through a deferred execution tracing system.
44

0 commit comments

Comments
 (0)