Stop reading the model's press releases. Start reading its activations.
compressed-rIHeHOdw.mp4
Live Demo 🚀 | Audit Report 📊 | Video Demo 🎥
Verbalize leverages Natural Language Autoencoders (NLA), the state-of-the-art interpretability technique published by Anthropic (Transformer Circuits) in May 2026.
While most safety tools monitor Chain-of-Thought (CoT), we know from research that CoT is often performative—the model tells you a story it thinks you want to hear. Verbalize applies this research by reading the mid-layer residual stream directly, translating raw internal activations back into natural language in real-time.
"CoT is what the model says it's thinking. We read what it's actually computing."
See three independent streams converge on one screen:
- Speech Stream: The standard tokens generated by the model (e.g., Qwen-2.5-7B).
- Thought Stream: Real-time NLA decodes of the Layer 20 activations.
- Judge Stream: A Claude Haiku 4.5 agent scoring the "alignment" between speech and thought live.
Our audit/ engine generates adversarial probes to find Fragile Passes: cases where the model's output is compliant, but its internal activations show it was considering a violation.
- Black-box red-teaming misses these.
- Verbalize catches them.
| Metric | Traditional Evals | Verbalize Audit |
|---|---|---|
| Input Signal | Final Text | Internal Activations |
| Gameable? | Yes (Performative) | No (Un-performed) |
| Output Type | Boolean / Score | Readable Natural Language |
ComoFuncionaDemo.mp4
Verbalize intercept the Layer 20 residual stream—the "semantic sweet spot" where the model has formed its intent but hasn't yet collapsed onto a specific token.
- Extract: We pull the hidden state vector from the base model.
- Translate: We inject this vector into a specialized NLA Actor (Natural Language Autoencoder).
- Decode: The NLA Actor translates the vector into a human-readable "thought."
- Judge: A parallel judge model compares the thought to the spoken output and flags discrepancies.
Getting two large models to run concurrently on a single 24GB/48GB GPU with low-latency streaming required deep surgery on the SGLang inference stack, specifically the input_embeds path.
nla/
├── frontend/ # Next.js 15 App Router (Vercel) — The live dashboard
├── backend/ # FastAPI Orchestrator (Railway) — SSE relays & Claude judge
├── gpu/ # SGLang + Qwen + NLA Stack (vast.ai) — Heavy-lifting inference
├── audit/ # Automated compliance auditing & adversarial probe engine
├── demo_data/ # Pre-recorded traces for zero-GPU/offline mode
├── animation/ # Manim scripts for viral 3D architecture visualizations
├── media/ # Rendered videos, diagrams, and project assets
├── scripts/ # Utility scripts for data processing and variant generation
└── tests/ # Pytest suite for backend and inference contracts
You can run the full UI and Backend using pre-recorded scenarios from demo_data/:
# Terminal 1: Backend
PYTHONPATH=. JUDGE_BACKEND=regex ORCHESTRATOR_GPU=mock \
uv run uvicorn backend.app:app --port 8001
# Terminal 2: Frontend
cd frontend && npm install && npm run devSee DEPLOY.md for the full guide on setting up the SGLang GPU node and connecting it to the Railway orchestrator.
- Authors: Juan Andres Quiroga, Ana Paula Tissera, Ignacio Vargas, Joaquin Alderete, and Alexander Bodner.
- NLA Models & Research: Kit Fraser-Taliente · kitft/natural_language_autoencoders
- Built for: Platanus Hack 2026.
License: Apache-2.0.