Motion Canvas Narrator:

Motion Canvas Narrator seamlessly integrates narration into your Motion Canvas workflow. Inspired by Motion Canvas' idea of letting your code define your animations, this package allows you to define narrations in code, making it easy to synchronize voiceovers and subtitles with your animations. You define your narrations and let them guide you through your voice recordings while displaying subtitles in the editor - or you let AI generate the audio for you.

Please note that this package is still in early development, so some bugs and missing features are expected. Contributions and suggestions are highly welcome!

Demo Video (Make sure to enable audio!)

60fps_720_elevenlabs_v3.mp4

The source code is available here: Example Project

Features

• Narration in Code: Define your narrations directly in your Motion Canvas code.
• AI Narration: Use AI to generate voiceovers from text.
• Custom Providers: Easily add your own TTS providers (contributions welcome!).
• Server Caching: Cache audio files on the server to avoid re-generating them.
• Mock Narrator: For testing without audio generation, useful for planning scripts and subtitles.

Provider Support

Provider	TTS	Voice Cloning	Fine Grained Timestamps	Remarks
ElevenLabs TTS	✅	✅		Requires Account
ElevenLabs Sound	✅			Sound effects, Requires Account
Google Vertex AI	wip			Requires Google Cloud Project
Minimax	✅			Requires Account
File Provider	✅	N/A		Load from local files
Mock Provider	✅	N/A		For testing/planning (no audio)

Other potential providers:

• piper1: GPL licensed, can be easily set up on your machine.
• Web Speech API: Built-in browser TTS, no API key required, but does not support exporting audio files.
• speechify

Planned Features

• In-Editor Recording: Record your own narrations directly in the Motion Canvas editor.
• Subtitles: Display precise subtitles with your narrations (check out the example project for rudimentary subtitles).
• Caption Export: Export subtitles in various formats (e.g., WebVTT).
• Detailed Timestamping: Timestamps for individual characters and words allow better synchronization and subtitles (example).

Usage

Using Motion Canvas Narrator in your Motion Canvas project is straightforward and only requires a few steps to set up. You can also check out the example project that includes subtitles used for the demo video here: Example Project

1. Install the Package

npm install motion-canvas-narrator

2. Enable the Cache Plugin in your vite.config.ts

This plugin is responsible for caching audio files on the server, so you don't have to re-generate them every time you run your project.

First, install the cache plugin package:

npm install motion-canvas-cache

Then add the plugin to your vite.config.ts file:

import {defineConfig} from 'vite';
import motionCanvas from '@motion-canvas/vite-plugin';
import ffmpeg from '@motion-canvas/ffmpeg';

import { motionCanvasCachePlugin } from 'motion-canvas-cache/vite-plugin';

export default defineConfig({
  plugins: [
    motionCanvas(),
    ffmpeg(), // make sure that you setup ffmpeg to export audio as well
    // Add the cache plugin for server-side audio caching:
    motionCanvasCachePlugin(),
  ]
});

3. Create a Narrator

The narrator serves as the primary interface for generating audio from text.

Using ElevenLabs TTS:

import { createElevenLabsNarrator } from 'motion-canvas-narrator';
const narrator = createElevenLabsNarrator({
    modelId: 'eleven_v3',
    voiceId: 'JBFqnCBsd6RMkjVDRZzb',
    apiKey: '<YOUR_ELEVENLABS_API_KEY>'
});

Using Minimax:

import { createMinimaxNarrator } from 'motion-canvas-narrator';
const narrator = createMinimaxNarrator({
    apiKey: '<YOUR_MINIMAX_API_KEY>',
    voiceId: 'your-voice-id'
});

Using Mock Provider (for testing):

import { createMockNarrator } from 'motion-canvas-narrator';
const narrator = createMockNarrator({
    wordsPerMinute: 150 // Optional, defaults to 120
});

4. Use the Narrator in Your Motion Canvas Project

To use the narrator, you can start narrations in your scenes as simply as:

  yield* narrator.speak("Welcome!");

This will generate frames for the duration of the narration. Note that the narration seamlessly integrates with Motion Canvas' animation system, allowing you to synchronize animations with the narration, using all for example:

  yield* all(
      // ... other animations ...
    narrator.speak("Welcome!")
  );

5. Advanced Usage

You can also customize the narration further by providing additional options such as volume and playback rate:

    yield* narrator.speak("Hello, world!", { gain: -5.2, playbackRate: 1.2 });

Contributing

If you'd like to contribute to this project, please feel free to open issues or pull requests.