Skip to content

Commit c79239f

Browse files
vigneshnarayanaswamyVignesh Narayanaswamyclaude
authored
v0.6.0: Agent-first protocol — MCP server, REST API, 6 tools
* Add Pydantic I/O schemas for v0.6.0 agent protocol tools Create the tools package with all 16 Pydantic v2 models covering 6 tool contracts (record, query, investigate, trace, changelog, discover) plus 4 shared types (ModelSummary, EventSummary, EventDetail, DependencyNode). These schemas are the single source of truth for MCP, REST API, and CLI. 72 tests covering all fields, defaults, JSON round-trip, and JSON Schema export. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add record tool — register models and record events Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add query tool — search and filter inventory Implements the query tool with structured filters (model_type, owner, status), case-insensitive text search on name/purpose, and pagination. Includes _model_to_summary helper for reuse by the discover tool. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add investigate tool — deep dive on single model Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add trace tool — dependency graph traversal Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add discover tool — bulk ingestion Implements the discover tool for bulk model ingestion from inline data, JSON files, or connectors. Inline and file sources convert dicts to DataNodes and leverage ledger.add() for content-hash dedup and ledger.connect() for auto-linking dependencies. Connector source raises NotImplementedError pending SDK-direct usage. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add changelog tool — cross-model event timeline Implements the changelog tool with time range filtering (since/until), model_name and event_type filters, pagination, and newest-first ordering. Includes 15 tests covering all specified behaviors. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: export all tool functions from tools package Re-exports changelog, discover, investigate, query, record, and trace from model_ledger.tools so consumers can import tool functions directly from the package. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add JSON file backend — human-readable, git-friendly default JsonFileLedgerBackend stores models, snapshots, and tags as indented JSON files in a directory tree. Implements full LedgerBackend protocol including sorted snapshot listing and tag-based resolution. 23 tests. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add REST API (FastAPI) wrapping tool functions Adds a FastAPI app at model_ledger.rest.app with endpoints for all 6 agent protocol tools (record, discover, investigate, query, trace, changelog) plus /overview. ModelNotFoundError maps to HTTP 404. Adds rest-api optional dependency and E402 suppression for tests using pytest.importorskip. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add MCP server with 6 tools and 3 resources FastMCP server wrapping all tool functions (discover, record, investigate, query, trace, changelog) with primitive-typed parameters for MCP compat. Three resources: ledger://overview, ledger://schema, ledger://backends. Includes create_server() factory and main() CLI entry point. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add demo inventory with sample models and events Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add mcp and serve CLI commands Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: export agent protocol tools and bump to v0.6.0 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Update README for v0.6.0 agent protocol Repositions model-ledger as a fundamental model inventory (not MRM-specific). Adds agent-first messaging, MCP/REST docs, tool protocol overview, JSON file backend, and updated install profiles. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Reorganize README for v0.6.0 - Clear section hierarchy with progressive disclosure - Fix architecture diagram (add SDK layer between tools and backends) - Fix self-contained investigate example - Trim agent conversation (3 → 2 exchanges) - Collapse compliance/introspection into brief "Additional Capabilities" - Add positioning line vs MLflow/SageMaker/W&B - Remove "thin tools, thick descriptions" from public README (internal) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Remove outdated design docs These docs reflect the old compliance-first positioning (v0.2-v0.3) and are superseded by the v0.6.0 README. Archived to second brain. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Fix tool output quality: model_type, metadata noise, platform - Ledger.add() now reads model_type/type from DataNode metadata (was only reading node_type, which nobody set) - investigate: skip internal events (depends_on, has_dependent, registered) and internal keys (_content_hash, upstream/downstream hashes) when merging metadata - query: extract platform from discovered payload, not just snapshot.source - demo: add model_type to all DataNode metadata Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Vignesh Narayanaswamy <Vigneshn@squareup.com> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent ef63110 commit c79239f

36 files changed

+4564
-1119
lines changed

README.md

Lines changed: 177 additions & 66 deletions
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,34 @@
11
# model-ledger
22

3-
**The model inventory your regulator actually wants. Auto-discovered, dependency-traced, audit-ready.**
3+
**Know what models you have deployed, where they run, what they depend on, and what changed.**
44

55
[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](LICENSE)
66
[![Python](https://img.shields.io/badge/python-3.10+-blue.svg)](https://python.org)
7+
[![PyPI](https://img.shields.io/pypi/v/model-ledger)](https://pypi.org/project/model-ledger/)
78

89
---
910

10-
model-ledger automatically discovers models, rules, pipelines, and queues across your systems — then builds the dependency graph between them.
11+
model-ledger is a model inventory for any company with deployed models. It discovers models across your platforms, maps the dependency graph, and tracks every change as an immutable event. Unlike model registries tied to a single platform (MLflow, SageMaker, W&B), model-ledger discovers across *all* of them — as one connected graph.
12+
13+
## Quick Start
14+
15+
**Talk to your inventory** — point Claude (or any MCP-compatible agent) at it:
16+
17+
```bash
18+
pip install model-ledger[mcp]
19+
claude mcp add model-ledger -- model-ledger mcp --demo
20+
```
21+
22+
```
23+
You: "what models are in my inventory?"
24+
Claude: "7 models across 5 platforms. fraud_scoring was retrained
25+
and deployed this week. Want me to dig into anything?"
26+
27+
You: "if we deprecate customer_features, what breaks?"
28+
Claude: "3 models consume it directly, 2 more transitively."
29+
```
30+
31+
**Or use the Python SDK:**
1132

1233
```python
1334
from model_ledger import Ledger, DataNode
@@ -34,58 +55,141 @@ graph LR
3455
style C fill:#FF9800,color:#fff,stroke:#F57C00
3556
```
3657

37-
Unlike model registries that track ML models only, model-ledger tracks the *entire model risk ecosystem* — ETL pipelines, heuristic rules, scoring jobs, alert queues, and ML models — as one connected graph with a full audit trail.
38-
3958
## Install
4059

4160
```bash
42-
pip install model-ledger # Core + SQLite backend
43-
pip install model-ledger[snowflake] # + Snowflake backend
44-
pip install model-ledger[rest] # + REST API connector
45-
pip install model-ledger[github] # + GitHub connector
46-
pip install model-ledger[all] # Everything
61+
pip install model-ledger # Core — SDK + tools + CLI
62+
pip install model-ledger[mcp] # + MCP server (for Claude Code / AI agents)
63+
pip install model-ledger[rest-api] # + REST API (for frontends / dashboards)
64+
pip install model-ledger[snowflake] # + Snowflake backend
65+
pip install model-ledger[mcp,rest-api,snowflake] # Everything
4766
```
4867

4968
## How It Works
5069

5170
```mermaid
5271
graph TB
53-
subgraph discover ["1. Discover"]
72+
subgraph consumers ["Consumers"]
73+
direction LR
74+
AGENT["Claude / AI Agents<br/><small>MCP</small>"]
75+
FRONT["Frontends<br/><small>REST API</small>"]
76+
SCRIPT["Scripts / Notebooks<br/><small>Python SDK</small>"]
77+
CLI_C["CLI<br/><small>model-ledger</small>"]
78+
end
79+
80+
subgraph tools ["Agent Protocol — 6 Consolidated Tools"]
81+
direction LR
82+
DISC["discover"] ~~~ REC["record"] ~~~ INV["investigate"]
83+
QRY["query"] ~~~ TRC["trace"] ~~~ CHG["changelog"]
84+
end
85+
86+
subgraph sdk ["Ledger SDK"]
87+
direction LR
88+
REG["register()"] ~~~ RECD["record()"] ~~~ GET["get() / list()"]
89+
HIST["history()"] ~~~ TRAC["trace()"] ~~~ CONN["connect()"]
90+
end
91+
92+
subgraph discover ["Discovery Sources"]
5493
direction LR
5594
DB["SQL databases"] --> F["sql_connector()"]
5695
API["REST APIs"] --> G["rest_connector()"]
5796
GH["GitHub repos"] --> H["github_connector()"]
5897
CUSTOM["Your platform"] --> I["SourceConnector protocol"]
5998
end
6099
61-
subgraph ledger ["2. Build Graph"]
100+
subgraph backends ["Storage — Pluggable Backends"]
62101
direction LR
63-
ADD["ledger.add()"] --> CON["ledger.connect()"]
64-
CON --> |"match output ports<br/>to input ports"| GRAPH["Dependency graph"]
102+
JSON["JSON files<br/><small>default</small>"]
103+
SQLITE["SQLite"]
104+
SNOW["Snowflake"]
105+
PLUG["Plugin<br/><small>Postgres, GitHub, ...</small>"]
65106
end
66107
67-
subgraph query ["3. Query"]
68-
direction LR
69-
TRACE["trace()"] ~~~ UP["upstream()"] ~~~ DOWN["downstream()"] ~~~ INV["inventory_at()"]
70-
end
108+
consumers --> tools
109+
tools --> sdk
110+
sdk --> discover
111+
sdk --> backends
112+
113+
style consumers fill:#F3E5F5,stroke:#7B1FA2,color:#4A148C
114+
style tools fill:#E3F2FD,stroke:#1565C0,color:#0D47A1
115+
style sdk fill:#E1F5FE,stroke:#0277BD,color:#01579B
116+
style discover fill:#E8F5E9,stroke:#2E7D32,color:#1B5E20
117+
style backends fill:#FFF3E0,stroke:#E65100,color:#BF360C
118+
```
119+
120+
Every model is a **DataNode** with typed input and output ports. When an output port name matches an input port name, `connect()` creates the dependency edge automatically. Every mutation is recorded as an immutable **Snapshot** — an append-only event log that gives you full history and point-in-time reconstruction.
121+
122+
## Agent Protocol
123+
124+
Six consolidated tools designed for AI agents ([Anthropic's tool design guidance](https://www.anthropic.com/engineering/writing-tools-for-agents)). Each is a plain Python function with Pydantic I/O — usable via MCP, REST, CLI, or direct import.
125+
126+
| Tool | What it does | Scale |
127+
|------|-------------|-------|
128+
| **discover** | Add models from any source — scan platforms, import files, inline data | Bulk |
129+
| **record** | Register a model or record an event with arbitrary metadata | Single |
130+
| **investigate** | Deep dive — identity, merged metadata, recent events, dependencies | Single |
131+
| **query** | Search and filter the inventory with pagination | Multi |
132+
| **trace** | Dependency graph — upstream, downstream, impact analysis | Graph |
133+
| **changelog** | What changed across the inventory in a time range | Multi |
71134

72-
discover --> ledger --> query
135+
### Using the tools directly
73136

74-
style discover fill:#E3F2FD,stroke:#1565C0,color:#0D47A1
75-
style ledger fill:#E8F5E9,stroke:#2E7D32,color:#1B5E20
76-
style query fill:#FFF3E0,stroke:#E65100,color:#BF360C
137+
```python
138+
from model_ledger import Ledger, record, investigate, query
139+
from model_ledger.tools.schemas import RecordInput, InvestigateInput, QueryInput
140+
from model_ledger.graph.models import DataNode
141+
142+
ledger = Ledger.from_sqlite("./inventory.db")
143+
144+
# Register a model
145+
record(RecordInput(
146+
model_name="fraud_scoring", event="registered",
147+
owner="risk-team", model_type="ml_model",
148+
purpose="Real-time fraud detection",
149+
), ledger)
150+
151+
# Record an event with schema-free payload
152+
record(RecordInput(
153+
model_name="fraud_scoring", event="retrained",
154+
payload={"accuracy": 0.94, "features_added": ["velocity_24h"]},
155+
actor="ml-pipeline",
156+
), ledger)
157+
158+
# Deep dive
159+
result = investigate(InvestigateInput(model_name="fraud_scoring"), ledger)
160+
result.metadata # {"accuracy": 0.94, "features_added": ["velocity_24h"]}
161+
result.total_events # 2
162+
163+
# Search
164+
models = query(QueryInput(text="fraud", model_type="ml_model"), ledger)
165+
models.total # 1
166+
```
167+
168+
### MCP server
169+
170+
```bash
171+
model-ledger mcp # empty inventory
172+
model-ledger mcp --demo # sample data
173+
model-ledger mcp --backend sqlite --path ./inventory.db # SQLite
174+
model-ledger mcp --backend json --path ./my-inventory # JSON files
175+
176+
# Connect to Claude Code (one time)
177+
claude mcp add model-ledger -- model-ledger mcp
77178
```
78179

79-
Every model is a **DataNode** with typed input and output ports. When an output port name matches an input port name, `connect()` creates the dependency edge automatically.
180+
### REST API
80181

81-
Every mutation is recorded as an immutable **Snapshot** — an append-only event log. Nothing is deleted. This gives you a complete audit trail and point-in-time inventory reconstruction for any date.
182+
```bash
183+
model-ledger serve # start on port 8000
184+
model-ledger serve --demo --port 3001 # with sample data
185+
```
186+
187+
Auto-generated OpenAPI docs at `/docs`. Endpoints: `POST /record`, `POST /discover`, `GET /query`, `GET /investigate/{name}`, `GET /trace/{name}`, `GET /changelog`, `GET /overview`.
82188

83189
## Discover Models From Your Systems
84190

85191
### SQL databases
86192

87-
Most discovery is "query a table, map rows to models." The `sql_connector` factory handles this without writing classes:
88-
89193
```python
90194
from model_ledger import Ledger, sql_connector
91195

@@ -146,7 +250,7 @@ pipelines = github_connector(
146250

147251
### Custom connectors
148252

149-
For anything the factories don't cover, implement the `SourceConnector` protocol:
253+
Implement the `SourceConnector` protocol for anything the factories don't cover:
150254

151255
```python
152256
class SageMakerConnector:
@@ -162,93 +266,100 @@ class SageMakerConnector:
162266
]
163267
```
164268

165-
## Persistent Storage
269+
## Storage
270+
271+
Storage-agnostic. Default is JSON files — human-readable, git-friendly, zero config. Upgrade when you need scale.
166272

167273
```python
168274
from model_ledger import Ledger
275+
from model_ledger.backends.json_files import JsonFileLedgerBackend
169276

170-
ledger = Ledger.from_sqlite("./inventory.db") # SQLite — zero infrastructure
171-
ledger = Ledger.from_snowflake(connection, schema="DB.MODEL_LEDGER") # Snowflakeproduction scale
172-
ledger = Ledger() # In-memorytesting
173-
ledger = Ledger(my_custom_backend) # CustomLedgerBackend protocol
277+
ledger = Ledger(JsonFileLedgerBackend("./my-inventory")) # JSON files — default
278+
ledger = Ledger.from_sqlite("./inventory.db") # SQLitezero infrastructure
279+
ledger = Ledger.from_snowflake(connection, schema="DB.MODEL_LEDGER") # Snowflakeproduction
280+
ledger = Ledger() # In-memorytesting
174281
```
175282

176-
## Key Capabilities
283+
JSON file layout — inspect, diff, and version-control your inventory:
284+
285+
```
286+
my-inventory/
287+
├── models/
288+
│ ├── fraud_scoring.json
289+
│ └── churn_predictor.json
290+
├── snapshots/
291+
│ ├── a1b2c3d4.json
292+
│ └── e5f6g7h8.json
293+
└── tags/
294+
└── {model_hash}/
295+
└── v1.json
296+
```
297+
298+
Add community backends via entry points:
299+
300+
```toml
301+
# pyproject.toml
302+
[project.entry-points."model_ledger.backends"]
303+
postgres = "my_package:PostgresBackend"
304+
```
305+
306+
## Additional Capabilities
177307

178308
### Dependency tracing
179309

180310
```python
181311
ledger.trace("fraud_alerts") # Full pipeline path
182312
ledger.upstream("fraud_alerts") # Everything that feeds this
183313
ledger.downstream("segmentation") # Everything that depends on this
184-
ledger.dependencies("fraud_alerts", direction="upstream") # Detailed with relationship info
185314
```
186315

187316
### Shared table disambiguation
188317

189-
When multiple models write to the same table, `DataPort` handles precision matching:
318+
When multiple models write to the same table, `DataPort` schema matching handles precision:
190319

191320
```python
192321
from model_ledger import DataPort, DataNode
193322

194-
# Two models write to the same alert table with different model_name values
195323
DataNode("check_rules", outputs=[DataPort("alerts", model_name="checks")])
196324
DataNode("card_rules", outputs=[DataPort("alerts", model_name="cards")])
197-
198-
# This reader only connects to check_rules — model_name must match
199325
DataNode("check_queue", inputs=[DataPort("alerts", model_name="checks")])
326+
# check_queue connects to check_rules only — model_name must match
200327
```
201328

202329
### Point-in-time inventory
203330

204331
```python
205-
from datetime import datetime
206332
inventory = ledger.inventory_at(datetime(2025, 12, 31))
207333
# Every model that was active on that date
208334
```
209335

210-
### Compliance validation
211-
212-
Built-in profiles for major model risk regulations:
336+
### Compliance validation (plugin)
213337

214-
| Profile | Regulation | Checks |
215-
|---------|-----------|--------|
216-
| `sr_11_7` | US Federal Reserve SR 11-7 | Validator independence, governance docs, validation schedule |
217-
| `eu_ai_act` | EU AI Act (2024/1689) | Risk classification, data governance, human oversight |
218-
| `nist_ai_rmf` | NIST AI RMF 1.0 | GOVERN, MAP, MEASURE, MANAGE functions |
338+
Built-in profiles for SR 11-7, EU AI Act, and NIST AI RMF. Add custom profiles for your organization's policies. See [validation docs](docs/) for details.
219339

220340
### Model introspection
221341

222-
Extract metadata from fitted ML models:
223-
224-
```python
225-
from model_ledger import introspect
226-
227-
result = introspect(fitted_model)
228-
result.algorithm # "XGBClassifier"
229-
result.features # [FeatureInfo(name="velocity_30d", ...), ...]
230-
result.hyperparameters # {"n_estimators": 50, "max_depth": 4}
231-
```
232-
233-
Ships with sklearn, XGBoost, and LightGBM support. Add your own via the `Introspector` protocol.
342+
Extract metadata from fitted sklearn, XGBoost, and LightGBM models. Add custom introspectors via the `Introspector` protocol. See [introspection docs](docs/) for details.
234343

235344
## Design Principles
236345

346+
- **Agents are the primary interface** — the MCP server is the product. SDK and CLI are still first-class, but the agent experience is what we optimize for.
347+
- **Fundamental, not specialized** — model inventory for any company with deployed models. Not tied to a specific regulatory framework or industry.
237348
- **Everything is a DataNode** — ML models, heuristic rules, ETL pipelines, alert queues. One abstraction.
238349
- **The graph builds itself** — declare inputs and outputs. Dependencies follow from port matching.
239-
- **Schema-agnostic metadata**`Snapshot.payload` is `dict[str, Any]`. The framework stores whatever your connectors discover.
240-
- **Append-only audit trail** — every change is an immutable Snapshot. Full history, point-in-time queries.
241-
- **Factory for the 80%, protocol for the 20%** — config-driven factories for common patterns, open protocols for anything custom.
242-
- **Batteries included** — persistence, discovery, graph building, and compliance with zero infrastructure.
350+
- **Schema-free payloads** — record whatever metadata matters. No schema to maintain, no migrations.
351+
- **Change tracking is central** — every mutation is an immutable Snapshot. The inventory is a living event log.
352+
- **Storage-agnostic** — JSON files, SQLite, Snowflake, or bring your own via the `LedgerBackend` protocol.
243353

244354
## For Organizations
245355

246-
model-ledger is designed as a core framework with lightweight organization-specific extensions. The OSS core handles graph building, storage, compliance, and the connector factories. Your internal package provides:
356+
The OSS core handles discovery, graph building, change tracking, storage, and the agent protocol. Your internal package provides:
247357

248-
- **Connector configs** — point `sql_connector()` at your tables, `rest_connector()` at your APIs
358+
- **Connector configs** — point factories at your tables and APIs
249359
- **Custom connectors** — for internal platforms the factories don't cover
250-
- **Authentication** — your database/API credentials and auth wrappers
251-
- **Additional compliance profiles** — OSFI E-23, PRA SS1/23, MAS AIRG, or internal policies
360+
- **Authentication** — your credentials and auth wrappers
361+
- **Custom backends** — Postgres, GitHub repos, or any storage via `LedgerBackend` protocol
362+
- **Compliance profiles** — SR 11-7, EU AI Act, or your own internal policies (plugin-based)
252363

253364
Your internal repo should be thin config and credentials, not reimplemented logic.
254365

0 commit comments

Comments
 (0)