EchoStudio

EchoStudio is a self-hosted, full-stack AI audio generation platform. It empowers users to generate text-to-speech, perform seamless voice conversion, and create custom sound effects—all without relying on expensive external APIs.

By self-hosting powerful open-source machine learning models, EchoStudio offers complete control over the audio generation pipeline.

Key Features & Technical Highlights

• Self-Hosted AI Architecture: Architected and deployed a self-hosted AI audio generation platform utilizing Next.js (T3 Stack) and Python FastAPI, replacing external APIs with open-source PyTorch models (StyleTTS2, Seed-VC) for text-to-speech, voice conversion, and sound effect generation.
• Scalable ML Workflows: Containerized complex ML workflows using Docker, managing multiple AI inference endpoints and deploying an asynchronous task queue with Inngest to prevent server overload and ensure high-availability audio processing.
• Secure Full-Stack Web App: Built a secure and scalable full-stack web application featuring user authentication (Auth.js), a custom credit management system, and secure AWS S3 integration for robust media storage and retrieval.

System Components

• Frontend: A responsive UI built with Next.js, React, and Tailwind CSS.
• Text-to-Speech (TTS): Powered by StyleTTS2, delivering high-quality natural voice synthesis.
• Voice Conversion: Utilizes Seed-VC to transform existing audio into different voices.
• Audio/SFX Generation: Employs Make-An-Audio to create sound effects directly from text prompts.
• Backend & Infrastructure: AI endpoints are served via Python/FastAPI and containerized with Docker. Long-running tasks are managed by Inngest, and audio files are securely stored in AWS S3.

Setup Instructions

1. Clone the Repository

git clone https://github.com/Arunil-jaiswal/EchoStudio.git
cd EchoStudio

2. Frontend Setup

Navigate to the frontend directory and install dependencies:

cd frontend
npm install

3. AI Models Setup

For each of the AI model directories (StyleTTS2, seed-vc, Make-An-Audio), you need to create a Python virtual environment (Python 3.10 recommended) and install the dependencies:

cd seed-vc
pip install -r requirements.txt

4. AWS S3 Configuration

Ensure you have an AWS S3 bucket named echostudio (or update the names in the .env and api.py files). Your AWS IAM user/role will need the following permissions:

• s3:PutObject
• s3:GetObject
• s3:ListBucket

5. Running the Application

• Use docker-compose up to start the AI inference endpoints.
• Start the Next.js development server using npm run dev in the frontend folder.

License

MIT License