recursive-lm

Process arbitrarily long contexts by recursively decomposing prompts — based on the Recursive Language Models paper.

What is this?

Traditional LLMs have a fixed context window. When your document exceeds it, you lose information. Recursive LM solves this by treating the prompt as an external environment and letting the model programmatically decompose and recursively process it.

The model gets a REPL interface with three tools:

• read(start, end) — read a character range from the document
• llm_query(query, context) — spawn a recursive sub-call to process a chunk
• FINAL(answer) — return the final answer

This enables divide-and-conquer strategies where the model automatically chunks, summarizes, and synthesizes — processing documents of 1M+ tokens with models that only have 32k-128k context windows.

Install

npm install recursive-lm

Quick Start

import { RecursiveLM } from 'recursive-lm';
import type { LLMProvider, Message } from 'recursive-lm';

// 1. Implement the provider interface for your LLM
class MyProvider implements LLMProvider {
  async generate(messages: Message[]): Promise<string> {
    // Call OpenAI, Anthropic, local model, etc.
    const response = await callYourLLM(messages);
    return response;
  }
}

// 2. Create the RecursiveLM instance
const rlm = new RecursiveLM({
  provider: new MyProvider(),
  maxDepth: 5,       // max recursion depth
  chunkSize: 8000,   // characters per chunk
  maxIterations: 20, // max loop iterations
});

// 3. Query any length document
const answer = await rlm.query(
  'What are the key findings?',
  veryLongDocument   // can be millions of characters
);

API

RecursiveLM

The main class. Wires together Environment, Scaffold, and Sandbox.

new RecursiveLM(config: RLMConfig)

Option	Type	Default	Description
provider	LLMProvider	required	Your LLM adapter
maxDepth	number	5	Max recursion depth for llm_query()
chunkSize	number	8000	Chunk size in characters
maxIterations	number	20	Max scaffold loop iterations
onStep	(e: StepEvent) => void	—	Called on each loop iteration
onRecurse	(e: RecurseEvent) => void	—	Called on recursive sub-calls
onFinal	(e: FinalEvent) => void	—	Called when answer is produced

Methods

• query(question, context) — Process a single document
• queryMultiDoc(question, documents) — Process multiple named documents
• queryWithEnvironment(question, environment) — Use a pre-configured Environment

LLMProvider Interface

interface LLMProvider {
  generate(messages: Message[]): Promise<string>;
}

interface Message {
  role: 'system' | 'user' | 'assistant';
  content: string;
}

Advanced: Direct Component Access

For fine-grained control, use the components directly:

import { Environment, Scaffold, Sandbox } from 'recursive-lm';

const env = new Environment(8000);
env.addDocument('report', longText);

const scaffold = new Scaffold({
  provider: myProvider,
  maxDepth: 5,
  maxIterations: 20,
});

const answer = await scaffold.run('Summarize the report', env);

How It Works

Based on Algorithm 1 from the paper:

1. Initialize: History H ← [system prompt, user query]
2. Loop:
   a. Call LLM(H) → response
   b. If response contains FINAL(answer) → return answer
   c. If response contains ```repl code:
      - Execute code (read, llm_query, etc.)
      - Append results to H
   d. Repeat

The model learns to perform parallel mapping (processing chunks in parallel recursive calls) and hierarchical reduction (combining summaries into higher-level summaries) — automatically adapting its strategy to the query.

License

MIT