OpenShorts.app 🚀🎬

OpenShorts is an all-in-one open-source solution to automate the creation and distribution of viral vertical content. It transforms long YouTube videos or local files into high-potential short clips optimized for TikTok, Instagram Reels, and YouTube Shorts.

📺 Video Tutorial: How it works

Click the image above to watch the full walkthrough.

---

✨ Key Features

OpenShorts leverages state-of-the-art AI to handle the entire content lifecycle:

1. 🧠 Viral Moment Detection:

   • Faster-Whisper: High-speed, CPU-optimized transcription and word-level timestamps.
   • Google Gemini 2.0 Flash: Advanced AI analysis to identify the 3-15 most viral moments based on hooks and engagement potential.
   • Automatic Copywriting: Generates SEO-optimized titles and descriptions for all platforms.

2. ✂️ Smart AI Cropping & Tracking (New V2 Engine):

   • Dual-Mode Strategy: Automatically detects scene composition to apply the best framing strategy.
     • TRACK Mode (Single Subject): Uses MediaPipe Face Detection + YOLOv8 fallback for ultra-fast, robust subject tracking. Features a "Heavy Tripod" stabilization engine that eliminates jitter and unnatural movements, providing smooth, cinematic reframing. Includes Speaker Identification to stick to the active speaker and avoid erratic switching.
     • GENERAL Mode (Groups/Landscapes): For scenes with multiple people or no clear subject, it automatically switches to a professional blurred-background layout, preserving the full width of the original shot while filling the 9:16 vertical space.
   • Intelligent Scene Analysis: Pre-scans every scene to determine the optimal strategy before processing.

3. ☁️ Automated S3 Backup:

   • Silent Background Upload: Once clips are generated, they are automatically uploaded to an AWS S3 bucket.
   • Seamless Integration: Operates in the background without affecting processing logs or UI performance.

4. 📲 Direct Social Posting:

   • Upload-Post Integration: Share your generated clips directly to TikTok, Instagram, and YouTube with a single click.
   • Profile Selector: Manage multiple social accounts easily through the dashboard.

5. 🎙️ AI Voice Dubbing:

   • ElevenLabs Integration: Translate your clips to 30+ languages with AI-powered voice dubbing.
   • Voice Cloning: Preserves the original speaker's voice characteristics in the new language.
   • Auto-Subtitles: Automatically generates subtitles in the dubbed language.

6. 🎨 Modern Web Dashboard:

   • Real-time Progress: Watch clips appear as they are generated with a live results feed.
   • Log Streaming: Follow the technical process with real-time log updates.
   • Responsive Design: A premium, dark-mode glassmorphism interface.

7. 📺 YouTube Studio (Full Publishing Pipeline):

   • AI Title Generation: Upload a video and let Gemini analyze the transcript to suggest viral, SEO-optimized titles with AI-ranked top picks.
   • Title Refinement Chat: Iteratively refine suggested titles via a built-in chat interface — ask for more clickbait, different tone, etc.
   • AI Thumbnail Generation: Generate professional thumbnails with optional face overlay, custom background, and extra creative instructions.
   • AI Description with Chapters: Automatically generates a YouTube description with chapter timestamps derived from the Whisper transcript.
   • One-Click YouTube Publishing: Publish the final video with its title, thumbnail, and description directly to YouTube via Upload-Post integration.
   • Manual Mode: Skip video analysis and enter your own title to jump straight to thumbnail generation.

---

🛠️ Requirements

• Docker & Docker Compose.
• Google Gemini API Key (Get it for free here).
• Upload-Post API Key (Optional, for direct social posting. Free tier available, no credit card required).
• ElevenLabs API Key (Optional, for AI voice dubbing. Get it here).

📲 Social Media Setup (Upload-Post)

To enable direct posting, follow these steps:

1. Login/Register: app.upload-post.com/login
2. Create Profile: Go to Manage Users and create a user profile.
3. Connect Accounts: In the same section, connect your TikTok, Instagram, or YouTube accounts to that profile.
4. Get API Key: Navigate to API Keys and generate your key.
5. Use in OpenShorts: Paste the API Key and select your Profile in the dashboard.

☁️ AWS S3 Setup (Optional)

To enable automatic backup of your clips to S3:

1. Environment Variables: Set the following in your .env file or system environment:
   • AWS_ACCESS_KEY_ID: Your AWS access key.
   • AWS_SECRET_ACCESS_KEY: Your AWS secret key.
   • AWS_REGION: (Optional) Defaults to us-east-1.
   • AWS_S3_BUCKET: (Optional) Defaults to openshorts.app-clips.
2. Bucket: Clips are uploaded to the specified bucket automatically after generation.

---

🚀 Getting Started

The easiest way to run OpenShorts is using Docker Compose.

1. Setup

git clone https://github.com/your-username/OpenShorts.git
cd OpenShorts

2. Launch the Application

docker compose up --build

3. Access the Dashboard

Open your browser and navigate to: http://localhost:5173

1. Enter your Gemini API Key.
2. (Optional) Enter your Upload-Post API Key to enable social sharing.
3. Paste a YouTube URL or Upload a Video.
4. Click "Generate Clips" and watch the magic happen!

---

🏗️ Technical Pipeline

1. Ingestion: Downloads YouTube videos via yt-dlp or handles local uploads.
2. Transcription: faster-whisper converts audio to text in seconds.
3. AI Intelligence: Gemini reads the transcript and selects periods of high interest.
4. Extraction: FFmpeg precisely cuts the selected segments.
5. Reframing: AI-powered visual tracking crops clips to vertical format.
6. Backup: Automated silent upload of clips and metadata to AWS S3.
7. Distribution: One-click posting via Upload-Post API.
8. YouTube Studio: Full publishing pipeline — AI titles, thumbnails, descriptions with chapters, and direct YouTube upload.

---

🔒 Security & Performance

• Non-Root Execution: Containers run as a dedicated appuser for security.
• Concurrency Control: Configurable job queue (MAX_CONCURRENT_JOBS).
• Auto-Cleanup: Automatic purging of old jobs and temporary files.
• File Limits: Built-in protection against oversized uploads.

---

🤝 Contributions

Contributions are welcome! Whether it's adding new AI models or improving the cropping engine, feel free to open a PR.

📄 License

MIT License. OpenShorts is yours to use, modify, and scale.