chalkboard

Open-source whiteboard-style explainer videos. Prompt → mp4.

chalkboard turns a prompt like "explain how the universe is aging" into a narrated, hand-drawn explainer video (diagrams, real AI-generated imagery, subtitles, and music), exported as an mp4. It runs entirely on your own machine and falls back to free local providers (Ollama + Piper TTS), so the marginal cost per video can be zero. MIT-licensed: yours to fork, embed, and bill for.

▶️ Watch the demo: this whole video was generated by chalkboard, from a single prompt describing how chalkboard works.

Features

Three formats: a 16:9 explainer, a vertical hook-first reel (--short), or a research-backed cinematic documentary (--cinematic) with full-frame imagery, Ken Burns motion, and emotional VO.
Multilingual: dub one production into many languages in a single run (--languages en,hi,es → one mp4 each), and burn subtitles in a different language than the audio (--subtitle-lang).
Any provider: Anthropic, OpenAI, or fully local Ollama for the script; Piper, OpenAI, or ElevenLabs for narration.
Hand-drawn diagrams: a RoughJS sketch aesthetic, with Graphviz auto-layout for graphs, trees, linked lists, and state machines. Add --draw to trace each element on like a pen (a "draw my life" effect) instead of the default fade-in.
Real images: image elements are generated with gpt-image-2 (sharp, accurate in-image text for labels and charts) for visual and science topics (stars, cells, maps), not just boxes and arrows.
Vector art: built-in SVG motifs and custom inline SVG, at zero API cost.
Subtitles: burned into every frame, with no player or libass dependency.
Background music: mood-matched, original CC0 beds (or the real Jamendo catalogue), sidechain-ducked under the narration so the voice stays clear.
Self-correcting: a deterministic layout pass plus an optional vision-model critique that fixes overflow, overlap, and contrast before the final render.
$0 local: Ollama + Piper means zero marginal cost per video; self-host the whole pipeline.

How it works

prompt                  → "explain how the universe is aging"
  llm (any provider)    → SceneScript JSON
  layout repair         → clamp / de-overlap / (optional) vision fix
  image gen             → gpt-image-2 for image elements
  tts (any provider)    → narration per scene
  renderer (Playwright) → canvas video + burned-in captions
  ffmpeg                → narration + ducked music → final mp4

Scenes are independent, so the renderer records them in parallel browser contexts and concatenates, so render wall-clock tracks the longest scene, not the sum. Image generation runs concurrently too. Tune with --render-concurrency.

Quickstart

# 0. system deps
#   - node >= 20.6 (needed for --env-file-if-exists)
#   - pnpm
#   - ffmpeg + ffprobe on PATH
#   - (optional) piper for local TTS

git clone https://github.com/Atharva-Kanherkar/chalkboard
cd chalkboard
pnpm install
pnpm --filter @chalkboard/renderer exec playwright install chromium

# 1. dry-run with stubs (no API keys, no piper)
pnpm --filter chalkboard start generate "explain hash tables" \
  --llm stub --tts stub -o out.mp4

# 2. real run: put your keys in .env at the repo root
cp .env.example .env
# edit .env, then:
pnpm --filter chalkboard start generate "explain hash tables" -o out.mp4

Both apps/server and apps/cli auto-load .env from the repo root (and from their own dir) via Node 20's --env-file-if-exists. No dotenv dep.

Project layout

packages/
  shared/      types (SceneScript, GenerateOptions, ProgressEvent)
  whiteboard/  pure helpers (draw animation, element normalization)
  llm/         LLMProvider + Anthropic/OpenAI/Ollama/Stub adapters
  narration/   TTSProvider + Piper/OpenAI/ElevenLabs/Stub adapters
  research/    grounded, cited briefs (powers --cinematic)
  renderer/    Playwright headless Chromium + ffmpeg mux
  core/        generate() orchestrator
apps/
  cli/         `chalkboard generate ...`
  server/      HTTP API: POST /generate, GET /jobs/:id, GET /jobs/:id/video
  studio/      Next.js chat-style frontend (live pipeline, inline mp4)
  web/         minimal static web UI served by the server

Configuration

LLM provider

kind	env / setup
`anthropic`	`ANTHROPIC_API_KEY` (default model: `claude-haiku-4-5-20251001`)
`openai`	`OPENAI_API_KEY`
`ollama`	`OLLAMA_BASE_URL` (default `http://localhost:11434`), `OLLAMA_MODEL` (default `llama3.1:8b`)
`stub`	none; returns a fixed SceneScript, for tests

Auto-resolution order: ANTHROPIC_API_KEY → OPENAI_API_KEY → ollama. Force a provider with --llm <kind> or CHALKBOARD_LLM=stub.

TTS provider

kind	env / setup
`piper`	`piper` on PATH, `PIPER_MODEL` (or `PIPER_MODEL_<LANG>`) to a `.onnx` voice
`openai`	`OPENAI_API_KEY` (default model `tts-1`, voice `alloy`)
`elevenlabs`	`ELEVENLABS_API_KEY`
`stub`	none; emits a sine-wave WAV, for tests

Subtitles

Captions are on by default and drawn straight onto the canvas during render (bottom-center, white with a dark outline), so they're baked into every frame, with no player support or libass-enabled ffmpeg required. They're built from the narration and timed per scene. Disable with --no-subtitles (CLI) or "subtitles": false (HTTP body / GenerateOptions).

Subtitles can be in a different language than the audio, e.g. Hindi narration with English captions. Each scene's narration is translated for the captions only; the audio and on-canvas text are untouched. Use --subtitle-lang <bcp47> (CLI) or "subtitleLanguage": "en" (GenerateOptions).

Background music

A mood-matched music bed is on by default, looped under the narration and sidechain-ducked: the music automatically dips whenever the voice is speaking, so narration stays clearly legible (plus a gentle intro swell and tail-out fade). The model picks the mood (meta.mood) from the subject:

mood	feel
`wonder`	lush, optimistic, for science / how-it-works (default)
`mystery`	sparse, dark, suspenseful, for open questions
`dramatic`	building, cinematic, for high stakes
`upbeat`	bright, energetic, for products / tutorials
`calm`	soft, slow, for meditative explainers

Every bundled track is original and CC0: a layered chord pad + plucked arpeggio + bass, synthesised with sox/ffmpeg (regenerate via node packages/renderer/scripts/build-music.mjs).

Force a mood: --music-mood mystery / "musicMood": "mystery" (or none).
Use the real Jamendo catalogue (Creative Commons): --music-source jamendo with a free JAMENDO_CLIENT_ID (the required attribution is surfaced in the output and result.music.attribution).
Bring your own: --music-track <path> / "musicTrack": "<path>".
Disable: --no-music / "music": false.

Images

For visual/real-world topics (astronomy, biology, geography…), the model can emit image elements with a prompt, and chalkboard generates real imagery for them with OpenAI's gpt-image-2 (crisp, accurate in-image text for labels, charts and diagrams) and draws it (cover-fit, rounded corners) into the box. Needs OPENAI_API_KEY; each image costs money, so generation is cached per prompt and capped per video. Disable with --no-images / "images": false (image elements then render as neutral placeholders). Override the model with --image-model <id>.

Control cost with --image-quality low|medium|high|auto (default medium, roughly 4× cheaper than high/auto). Each run reports its estimated image spend from the API's token usage, for example around $0.42 for 7 images.

Self-correcting render

Because rendering isn't live, chalkboard can look at what it drew and fix it before finalizing. Two layers:

Deterministic repair (always on): clamps overflow, de-dupes stacked text, separates overlaps. No API calls.
Vision critique (opt-in: --self-correct [passes] / "selfCorrect": true|N): screenshots each scene's final state, sends the still to a vision model (gpt-5.5 by default, OPENAI_VISION_MODEL to override), and applies the corrected elements it returns. Catches what geometry can't: unreadable contrast, text over a dark shape, awkward composition. Bounded passes; needs OPENAI_API_KEY.

Vector art (SVG + motifs)

Beyond the hand-drawn shapes, the model can emit svg elements for crisp vector art at zero API cost:

{ type: "svg", motif: "star", color: "#f08c00", x, y, width, height }: a built-in icon by name (star, bolt, heart, check, cross, sun, cloud, gear, lightbulb, database, arrow-right).
{ type: "svg", svg: "<svg>…</svg>", … }: any custom inline SVG, drawn contained (never cropped) in the box.

Both motifs and inline SVG are pure-vector and add nothing to your API bill.

Usage

CLI

# Render a video
chalkboard generate "explain pointers" \
  --lang en --aspect 16:9 \
  --llm anthropic --tts piper \
  -o pointers.mp4

# Just dump the SceneScript JSON (no render, useful for prompt iteration)
chalkboard script "explain pointers" --llm anthropic > script.json

Run chalkboard --help for the full flag list.

Library

import { generate } from '@chalkboard/core';

await generate({
  prompt: 'explain hash tables',
  outputPath: './hash-tables.mp4',
  language: 'en',
  aspectRatio: '16:9',
  llm: { kind: 'anthropic' },
  tts: { kind: 'piper' },
  onProgress: (e) => console.log(e),
});

HTTP service + web UI

pnpm --filter @chalkboard/server start
# → chalkboard server listening on http://0.0.0.0:4140

Open http://localhost:4140 in a browser for the prompt → mp4 web UI. The same port also serves the HTTP API:

curl -X POST http://localhost:4140/generate \
  -H 'content-type: application/json' \
  -d '{
    "prompt": "explain pointers",
    "llm": { "kind": "anthropic" },
    "tts": { "kind": "piper" }
  }'
# → { "jobId": "...", "status": "running" }

curl http://localhost:4140/jobs/$ID            # poll
curl http://localhost:4140/jobs/$ID/video -o out.mp4

Configure: STORAGE_DIR (default ./out), PORT (default 4140). API keys are read from the server's env; the web UI never sees them.

Studio (Next.js web app)

apps/studio is the polished product frontend: a chat-style workspace where you type a topic, watch the pipeline run live (script → narration → render → mux), and get the mp4 inline with a download. Toggle aspect ratio (or research depth in Cinematic), subtitles, music, images, self-correct, multilingual dubbing, subtitle language, or Demo (stub providers, instant, no API cost).

Type a topic, pick a lane (Explainer · Reels · Cinematic), and generate.

The pipeline runs live (script → narration → render → mux) with a lightweight loading state.

The Cinematic lane adds a Quick/Standard/Deep research depth; dub one production into many languages in a single run.

pnpm --filter @chalkboard/server start            # backend on :4140
pnpm --filter @chalkboard/studio dev              # studio on :3000

Open http://localhost:3000. The studio proxies /api/* to the server (CHALKBOARD_API env overrides the target), so it deploys independently of the backend. Three lanes are live: Explainer (16:9), Reels (vertical 9:16 shorts), and Cinematic (researched documentary with a Quick/Standard/Deep depth control); Repurpose is planned. The studio can also dub one production into multiple languages in a single run (with a per-language switcher on the result) and burn subtitles in a different language than the narration.

Reels (vertical shorts)

The Reels lane (or --short on the CLI) switches to a hook-first vertical preset: 9:16, 3-5 tight scenes, one punchy sentence each, large captions lifted clear of the platform UI, royalty-free music. Music is intentionally royalty-free: Instagram/TikTok's trending songs are licensed for in-app use only, so add them after posting rather than baking copyrighted audio into the export.

chalkboard generate "3 wild facts about black holes" --short -o reel.mp4

Cinematic (research-backed documentary)

--cinematic researches the topic first, then builds a full-frame documentary cut: real generated imagery edge-to-edge, Ken Burns motion, transitions, and a more emotional voiceover, closer to a short film than a whiteboard. The research is grounded and cited, and the brief's sources are injected into the script so the narration stays factual.

chalkboard generate "the race to sequence the human genome" --cinematic -o film.mp4

Research depth: --depth quick | standard | deep (default standard).
Research provider: --research openai-deep-research | basic | stub (default: deep research if OPENAI_API_KEY is set, otherwise a basic web pass).

You can also research a topic on its own (a grounded, cited brief with no render), which is handy for inspecting sources before committing to a video:

chalkboard research "the race to sequence the human genome" --depth deep
chalkboard research "..." --json > brief.json   # raw CitedBrief

Multilingual (dub one production into many languages)

Build the script and images once, then localize narration and on-canvas text per language: one mp4 per language, all sharing the same visuals.

chalkboard generate "explain photosynthesis" --languages en,hi,es -o photosynthesis.mp4
# → photosynthesis.en.mp4, photosynthesis.hi.mp4, photosynthesis.es.mp4

Via the library: "languages": ["en", "hi", "es"] on GenerateOptions.

SceneScript

The contract between LLM and renderer is a typed JSON document:

{
  "version": "1",
  "meta": { "language": "en", "aspectRatio": "16:9", "title": "Hash Tables" },
  "scenes": [
    {
      "id": "scene-1",
      "narration": "A hash table maps keys to values...",
      "elements": [
        {
          "id": "s1-rect",
          "type": "rectangle",
          "x": 200,
          "y": 200,
          "width": 400,
          "height": 80,
          "strokeColor": "#1e1e1e",
          "backgroundColor": "#a5d8ff",
          "fillStyle": "solid",
        },
        {
          "id": "s1-text",
          "type": "text",
          "x": 220,
          "y": 220,
          "text": "hash(key)",
          "fontSize": 32,
          "fontFamily": 1,
        },
      ],
    },
  ],
}

Element types: rectangle, ellipse, diamond, line, arrow, text, code-block, step-marker, group, highlight, graphviz, image, svg. Elements appear progressively (opacity ramp, staggered) while the narration plays; visual timing stretches to match audio.

Smoke tests

# Pure-logic tests (parse, timing, normalize, draw)
pnpm test

# Renderer smoke (hardcoded SceneScript → mp4), ~10 sec
pnpm --filter @chalkboard/renderer smoke

# Full pipeline smoke (stub LLM + stub TTS → mp4), ~6 sec
pnpm --filter @chalkboard/core smoke

Cost per video

Setup	Cost per video
Ollama + Piper (fully local)	$0
Claude Haiku + Piper	~$0.0002
Claude Haiku + OpenAI TTS	~$0.002
Cloud LLM + TTS + generated images	~$0.10 to $2.00

Generated images dominate the cloud cost, so cap or disable them (--no-images) to stay in fractions of a cent. Rendering is real-time: a 90-second video takes a few minutes of wall-clock as Playwright records the canvas, in exchange for portability and no render farm.

Contributing

Issues and pull requests are welcome. See CONTRIBUTING.md for setup, the checks CI runs, and a tour of the monorepo. No API keys are needed to develop: the stub providers render a real mp4 for free.

Acknowledgements

chalkboard stands on a lot of open source:

Diagrams & rendering: RoughJS for the hand-drawn aesthetic, Graphviz via @hpcc-js/wasm-graphviz for auto-layout, an Excalidraw-inspired element model for SceneScript, Playwright for headless Chromium capture, and FFmpeg for encoding and muxing.
Models & voices: Ollama for local LLMs and Piper for local TTS, with optional OpenAI, Anthropic, and ElevenLabs. Background music can come from the Creative Commons Jamendo catalogue.
App & tooling: Next.js, React, Tailwind CSS, Hono, Commander, TypeScript, tsx, Vitest, Prettier, and pnpm. The bundled CC0 music is synthesised with SoX and FFmpeg.

Thank you to everyone who maintains these projects.

License

MIT

Name		Name	Last commit message	Last commit date
Latest commit History 55 Commits
.github/workflows		.github/workflows
apps		apps
docs		docs
examples		examples
packages		packages
testing		testing
.env.example		.env.example
.gitignore		.gitignore
.npmrc		.npmrc
.prettierrc		.prettierrc
CONTRIBUTING.md		CONTRIBUTING.md
LICENSE		LICENSE
README.md		README.md
package.json		package.json
pnpm-lock.yaml		pnpm-lock.yaml
pnpm-workspace.yaml		pnpm-workspace.yaml
tsconfig.base.json		tsconfig.base.json
tsconfig.json		tsconfig.json

Navigation Menu

Repository files navigation

chalkboard

Features

How it works

Quickstart

Project layout

Configuration

LLM provider

TTS provider

Subtitles

Background music

Images

Self-correcting render

Vector art (SVG + motifs)

Usage

CLI

Library

HTTP service + web UI

Studio (Next.js web app)

Reels (vertical shorts)

Cinematic (research-backed documentary)

Multilingual (dub one production into many languages)

SceneScript

Smoke tests

Cost per video

Contributing

Acknowledgements

License

About

Releases

Packages

Contributors

Languages

Folders and files

Latest commit

History

Repository files navigation

chalkboard

Features

How it works

Quickstart

Project layout

Configuration

LLM provider

TTS provider

Subtitles

Background music

Images

Self-correcting render

Vector art (SVG + motifs)

Usage

CLI

Library

HTTP service + web UI

Studio (Next.js web app)

Reels (vertical shorts)

Cinematic (research-backed documentary)

Multilingual (dub one production into many languages)

SceneScript

Smoke tests

Cost per video

Contributing

Acknowledgements

License

About

Resources

Contributing

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages