Skip to content

ROADMAP.md

Latest commit

 

History

History
260 lines (206 loc) · 10.1 KB

ROADMAP.md

File metadata and controls

260 lines (206 loc) · 10.1 KB
Raw

Video Toolkit Roadmap

This document tracks the development of claude-code-video-toolkit.

Repository: https://github.com/digitalsamba/claude-code-video-toolkit

Vision

An open-source, AI-native video production workspace for Claude Code, featuring:

  • Reusable templates for common video types
  • Brand profiles for consistent visual identity
  • Claude skills providing deep domain knowledge
  • Automated asset pipelines (recording, conversion, audio generation)
  • Slash commands for guided workflows

Current Status

Phase: 3 - Templates & Brands (nearly complete) → moving into Phase 4 Focus: Publishing & distribution, generator coverage (image/video/music), external contributions Recent (v0.15–v0.18): YouTube publishing (/publish), 60db TTS provider, Ideogram 4 T2I, LTX-2 Modal + style LoRAs, concept-explainer-short (9:16) template, TTS pacing QC, Qwen3 VoiceDesign

Phases

Phase 1: Foundation ✅ COMPLETE

  • Sprint review template with theme system
  • Config-driven video content
  • /video slash command (unified project creation)
  • Narrator PiP component
  • Remotion skill (stable)
  • ElevenLabs skill (stable)

Phase 2: Skills & Automation ✅ COMPLETE

Skills:

  • FFmpeg skill (beta)
  • Playwright recording skill (beta)

Python Tools:

  • voiceover.py - CLI for ElevenLabs TTS
  • music.py - CLI for background music
  • sfx.py - CLI for sound effects

Commands:

  • /generate-voiceover - streamlined audio generation
  • /record-demo - guided Playwright recording

Infrastructure:

  • Playwright recording setup (playwright/)
  • Centralized config (env var with registry fallback)

Phase 2.5: Open Source Release ✅ COMPLETE

  • Directory restructure for public release
  • Brand profiles system (brands/)
  • Environment variable support
  • README, LICENSE (MIT), CONTRIBUTING.md
  • Documentation (docs/)
  • GitHub repo published

Phase 3: Templates & Brands 🔄 IN PROGRESS

Brand Profiles:

  • Default brand profile
  • Digital Samba brand profile
  • /brand command - list, edit, or create brands

Templates:

  • Product demo template
  • /video command - unified project management
  • /template command - list available templates
  • Shared component library (lib/)
  • Tutorial template
  • Changelog/release notes template

Transitions Library:

  • Transitions library (lib/transitions/)
  • Custom presentations: glitch, rgbSplit, zoomBlur, lightLeak, clockWipe, pixelate, checkerboard
  • Re-exports official transitions: slide, fade, wipe, flip
  • Transitions gallery showcase (showcase/transitions/)
  • Documentation in Remotion skill and CLAUDE.md

Template-Brand Integration:

  • Brand loader utility (lib/brand.ts)
  • Templates use brand.ts for theming
  • /video generates brand.ts from selected brand

Multi-Session Project System:

  • Project schema (lib/project/types.ts)
  • Filesystem reconciliation
  • Auto-generated CLAUDE.md per project
  • /skills command

Review & Validation:

  • /scene-review command - dedicated scene-by-scene review with Remotion Studio
    • Starts Remotion Studio for visual verification
    • Walks through scenes one by one (not summary tables)
    • Generic - works with any template's config
    • /video delegates to /scene-review when phase is review
    • /generate-voiceover warns if review incomplete
  • Pre-render review (timing, sync, polish)
  • Asset validation (ffprobe checks)
  • Enhancement suggestions
  • Note: /review name clashes with Claude Code built-in PR review - using /scene-review

Qwen3-TTS Integration:

  • tools/qwen3_tts.py — standalone CLI tool
  • voiceover.py --provider qwen3 — per-scene generation
  • Docker image: ghcr.io/conalmullan/video-toolkit-qwen3-tts:latest
  • 9 built-in speakers, tone presets, voice cloning
  • /voice-clone command — record, test, save cloned voice to brand
  • Temperature/top_p generation params for expressiveness control
  • Make Qwen3-TTS the default provider (replacing ElevenLabs)

Modal Cloud GPU Provider:

  • tools/cloud_gpu.py — shared provider abstraction (RunPod + Modal)
  • tools/file_transfer.py — shared R2/fallback upload/download
  • --cloud runpod|modal flag on all cloud GPU tools
  • docker/modal-qwen3-tts/app.py — deployed, tested
  • docker/modal-flux2/app.py — deployed, tested
  • docker/modal-upscale/app.py — deployed, tested
  • docker/modal-image-edit/app.py — deployed, tested
  • docker/modal-music-gen/app.py — deployed, tested
  • docker/modal-sadtalker/app.py — deployed, tested
  • --runpod deprecated in upscale.py and dewatermark.py (alias for --cloud runpod)
  • voiceover.py passes --cloud through to Qwen3-TTS
  • dewatermark.py migrated to cloud_gpu.py (removed ~393 lines of inline RunPod code)
  • docs/modal-setup.md — setup guide for Modal deployment
  • Add --setup --cloud modal to tools (currently manual modal deploy)

Sprint Review v2:

  • sprint-review-v2 template — composable scene-based architecture
  • Modular scene components

Generators & Providers (v0.11–v0.18):

  • FLUX.2 Klein text-to-image (tools/flux2.py, RunPod + Modal)
  • Ideogram 4 text-to-image for in-image text (tools/ideogram4.py, hosted API)
  • LTX-2 video generation (tools/ltx2.py, tools/chain_video.py, style LoRAs incl. crt-terminal)
  • ACE-Step 1.5 music generation (tools/music_gen.py, acemusic API + Modal/RunPod)
  • Qwen3-TTS VoiceDesign + shared-prompt cloning (v0.2)
  • 60db TTS provider (--provider 60db, tools/sixtydb_tts.py)
  • TTS pacing QC (tools/pacing.py, --max-wpm)
  • Edge-TTS provider (PR #46, needs rebase over 60db)
  • Make Qwen3-TTS the default provider (replacing ElevenLabs) — still open

Publishing:

  • YouTube upload (tools/youtube_upload.py, /publish) — OAuth, scheduling, thumbnails
  • Other destinations (LinkedIn, X, S3/R2 CDN)

Additional Templates (shipped):

  • concept-explainer-short — 9:16 vertical shorts, first Python/moviepy template
  • Monthly engineering-review template (ticket-driven, presenter-narrated) — in-tree but not yet ready to publish

Additional Components:

  • FilmGrain — SVG noise overlay for cinematic texture
  • MazeDecoration — Animated isometric grid decoration

Contribution & Examples:

  • /contribute command
  • examples/ directory
  • CONTRIBUTORS.md

Testing:

  • Test new project creation with scene-centric flow
  • Test project resumption (multi-session)
  • Verify filesystem reconciliation
  • Verify CLAUDE.md auto-generation

Registry & Roadmap Alignment:

  • Update skill status table to include all current skills
  • Add Qwen3-TTS to roadmap phases
  • Document remotion-official skill sync from upstream (remotion-dev/skills) in roadmap
  • Update Metrics section

Phase 4: Polish & Advanced

Output & Accessibility:

  • Multi-format output (MP4, WebM, GIF, social formats)
  • Subtitle generation from voiceover scripts
  • Thumbnail auto-generation

Skills:

  • Video accessibility skill
  • Terminal recording skill (asciinema)
  • Video timing skill

Skill Maturity Levels

Status Meaning
draft Just created, untested, may have errors
beta Functional, needs real-world validation
stable Battle-tested, well-documented, recommended

Current Skill Status

Skill Status Notes
remotion stable Core framework knowledge
remotion-official stable Synced from remotion-dev/skills (weekly via GitHub Actions)
elevenlabs stable Audio generation
ffmpeg beta Asset conversion
playwright-recording beta Browser demo capture
frontend-design stable Visual design refinement
qwen-edit stable AI image editing prompting patterns
runpod stable Cloud GPU setup, Docker images, endpoint management
modal beta Alternative cloud GPU provider — faster cold starts, all 6 tools deployed
acestep stable ACE-Step 1.5 music generation prompting
ltx2 beta LTX-2 video generation prompting, style LoRAs
moviepy stable Python composition, deterministic text overlay
ideogram4 beta Structured JSON captions for in-image text

Review Process

draft → beta:

  • Verify code examples work
  • Test core functionality
  • Document issues in _internal/reviews/
  • Fix critical issues

beta → stable:

  • Use in a real project
  • Gather feedback
  • Complete documentation
  • No known critical issues

Metrics

Category Count Items
Templates 4 sprint-review, sprint-review-v2, product-demo, concept-explainer-short
Brands 2 default, digital-samba
Skills 12 remotion, remotion-official, elevenlabs, ffmpeg, playwright-recording, frontend-design, qwen-edit, runpod, acestep, ltx2, moviepy, ideogram4
Tools 22 voiceover, qwen3_tts, sixtydb_tts, music, music_gen, sfx, redub, addmusic, sync_timing, align_captions, dewatermark, locate_watermark, notebooklm_brand, image_edit, upscale, flux2, ideogram4, ltx2, chain_video, sadtalker, verify_setup, youtube_upload
Commands 14 setup, video, brand, template, skills, contribute, record-demo, generate-voiceover, scene-review, design, versions, redub, voice-clone, publish
Components 11 AnimatedBackground, SlideTransition, Label, Vignette, FilmGrain, LogoWatermark, SplitScreen, NarratorPiP, Envelope, PointingHand, MazeDecoration
Transitions 7 glitch, rgbSplit, zoomBlur, lightLeak, clockWipe, pixelate, checkerboard
Examples 7 hello-world, quick-spot, data-viz-chart, ds-crt-stinger, sky-blue-short, digital-samba-skill-demo, sprint-review-cho-oyu

Related Files

File Purpose
BACKLOG.md Unscheduled ideas and future enhancements
CHANGELOG.md Historical record of changes
toolkit-registry.json Machine-readable inventory