AI Video Caption Generator

A Flask web app that takes a raw video or audio file and turns it into something ready to publish: burned in animated captions, a clean audio track, or an accurate SRT subtitle file. Under the hood it leans on OpenAI's Whisper model for transcription, OpenCV for drawing captions onto every frame, and MoviePy for stitching audio and video back together.

The goal was to make captioning something you do not have to think about. Upload a file, pick what you want out of it, and the app handles transcription, timing, styling, and rendering on its own.

What it does

The app has four modes, each built around the same transcription pipeline.

Video to video. Upload a video and get back a copy with animated, word by word captions burned directly into the frame. As each word is spoken it lights up in a different color while the rest of the line stays white, karaoke style. You can pick from a few preset color schemes before rendering.

Video to audio. Upload a video and pull out just the audio track as a standalone file.

Video to SRT. Upload a video and get back a properly timed SRT subtitle file generated from the spoken words, ready to drop into any video editor or player.

Audio to SRT. Same idea, but starting from an audio file instead of a video.

How it works

1. You upload a file through one of the four modes on the site.
2. faster-whisper (running the large-v3 model on CPU with int8 quantization) transcribes the audio and returns word level timestamps.
3. Those words are grouped into caption lines using simple rules: keep each line under about 30 characters, under 2.5 seconds long, and start a new line whenever there is a gap of more than 1.5 seconds of silence.
4. For video to video mode, OpenCV walks through the source video frame by frame and draws the active caption line on top, highlighting whichever word is being spoken at that instant.
5. MoviePy reattaches the original audio track to the freshly captioned video so the final file is ready to watch and share.
6. The finished video, audio file, or SRT file is handed back to you as a download.

Tech stack

• Backend: Flask
• Transcription: faster-whisper (Whisper large-v3, CPU inference with int8 quantization)
• Video and audio processing: MoviePy
• Caption rendering: OpenCV
• Frontend: HTML and CSS templates served through Flask

Running it locally

You will need Python 3.11 and ffmpeg installed (MoviePy and Whisper both depend on it).

git clone https://github.com/maahipatel05/AI_Video_Caption_Generation.git
cd AI_Video_Caption_Generation
pip install flask moviepy opencv-python faster-whisper
python hello.py

The app will start on http://localhost:5000. From the home page you can choose to add captions to a video, extract its audio, or generate an SRT file from a video or audio upload.

The first run will take a little longer while Whisper downloads the large-v3 model.

Project layout

AI_Video_Caption_Generation/
├── hello.py            Flask app: routes, transcription, caption rendering, and file handling
├── templates/          Pages for each mode (home, video to video, video to audio, video to SRT, audio to SRT, about, contact)
├── uploads/            Incoming files land here
├── downloads/          Working files generated mid pipeline
├── output/             Finished files ready for download
└── LICENSE

License

MIT. See LICENSE for details.

---

Built by Maahi Patel.