AI 3D Virtual Teacher - GPTAvatar

A 3D AI-powered virtual teaching platform with animated teacher avatars for interactive voice-based education. Students can engage in natural conversations with AI teachers for personalized instruction, language learning, quizzes, and conversational practice.

Features

• 🎙️ Voice Input - Natural speech recognition via OpenAI Whisper
• 🤖 AI Conversations - Intelligent responses powered by ChatGPT
• 🔊 Voice Output - Text-to-speech using OpenAI TTS, ElevenLabs, or Google Cloud
• 👨‍🏫 Multiple Characters - Japanese Teacher, Seth (Game Dev), Burger Barn employee
• 🌍 Multilingual - Support for English and Japanese instruction
• 🎨 3D Avatars - Realistic character models with animations

Use Cases

Primary Use Case: AI-powered virtual teaching platform with 3D animated teacher avatars for interactive educational experiences. Students can engage in voice conversations with specialized AI teachers (like Atsuko-sensei for Japanese language learning) who provide personalized instruction, quizzes, and conversational practice.

Character Scenarios:

• Japanese Teacher (Atsuko) - Language learning, grammar practice, roleplay
• Seth - Game development discussions, retro gaming trivia
• Burger Barn - Customer service roleplay scenarios

Tech Stack

• Unity 2022.2.13f1 - Game engine with C# scripting
• Universal Render Pipeline (URP) - 3D rendering
• OpenAI APIs:
  • Whisper - Speech-to-text recognition
  • ChatGPT (GPT-3.5/4) - Conversation AI
  • TTS - Text-to-speech synthesis
• ElevenLabs - High-quality voice synthesis (optional)
• Google Cloud TTS - Multilingual speech support (optional)

System Architecture

Inputs:

• Student voice recordings via microphone
• Teacher configuration files with personalities and teaching styles
• API keys for AI services

Database:

• config.txt stores teacher profiles, prompts, and credentials
• In-memory chat history queue tracks conversation context

Process:

1. Whisper API transcribes student speech to text
2. ChatGPT generates teaching responses with context awareness
3. Text converted to speech via TTS
4. Audio played with animated 3D avatar

Outputs:

• AI-generated educational responses (text)
• Synthesized teacher voice through speakers
• Animated 3D avatar with lip-sync

Quick Start

Prerequisites

• Unity Hub
• Unity 2022.2.13f1 or later
• OpenAI API Key (required)
• Google Cloud TTS API Key (optional)
• ElevenLabs API Key (optional)

Installation

1. Clone the repository:

git clone https://github.com/AhmedRaoofuddin/AI-3D-Model-.git
cd AI-3D-Model-

2. Configure API Keys:

   • Open config.txt
   • Add your OpenAI API key:

     set_openai_api_key|your-api-key-here
     

   • (Optional) Add Google Cloud and ElevenLabs keys for enhanced voice

3. Open in Unity:

   • Open Unity Hub
   • Click "Add" → Select this project folder
   • Open the project (Unity 2022.2.13f1)

4. Open Main Scene:

   • Navigate to Assets/Main.unity
   • Double-click to open

5. Run the Application:

   • Click Play ▶️ button in Unity Editor
   • Click "Start" to initialize conversation
   • Click "Record" to speak, click again to stop
   • AI will respond with text and voice

Configuration

Edit config.txt to customize:

• Character personalities and teaching styles
• AI model selection (GPT-3.5 vs GPT-4)
• Voice settings (pitch, speed)
• Token memory limits
• Temperature/creativity settings

Example Character Configuration:

add_friend|Japanese Teacher
set_friend_language|japanese
set_friend_token_memory|1000
set_friend_temperature|1.1 
set_friend_visual|japanese_teacher
set_friend_base_prompt
You are a gentle, sweet, expert Japanese teacher named Atsuko...
<END_TEXT>

Controls

• Record - Start/stop voice recording
• Start - Character introduces themselves
• Stop - Stop current audio playback
• Copy - Copy dialogue to clipboard
• Forget - Clear conversation history
• < / > - Switch between characters
• Config - Edit configuration in-app

Cost Warning ⚠️

These APIs cost real money:

• OpenAI Whisper - Speech-to-text
• ChatGPT - Conversation (~$0.002 per 1K tokens for GPT-3.5)
• OpenAI TTS - Voice output (~$15 per 1M characters)
• ElevenLabs - Premium voices (most expensive)
• Google Cloud TTS - Multilingual (cheaper alternative)

Monitor your usage at respective API dashboards.

Troubleshooting

No voice output:

• Ensure OpenAI API key is correctly set in config.txt
• Check Console tab for errors
• Restart Unity after config changes

Pink/Magenta materials:

• Materials need URP shader conversion
• Select materials → Change shader to "Universal Render Pipeline → Lit"
• Or use Edit → Rendering → Materials → Convert to URP

Compilation errors:

• SALSA LipSync is disabled by default (paid asset)
• If you have it, uncomment #define CRAZY_MINNOW_PRESENT in AIManager.cs

401 Unauthorized errors:

• Verify API keys are valid
• Check API key permissions and billing

Project Structure

Assets/
├── _Script/           # Main application scripts
│   ├── AIManager.cs   # Core AI conversation handler
│   ├── Config.cs      # Configuration loader
│   └── GameLogic.cs   # Main game controller
├── RT/AI/             # AI service managers
│   ├── OpenAITextToSpeechManager.cs
│   ├── GoogleTextToSpeechManager.cs
│   └── ElevenLabsTextToSpeechManager.cs
├── CC3Seth/           # Seth character model & materials
├── CC4Teacher/        # Teacher character model & materials
└── Main.unity         # Main scene

Development

Adding New Characters:

1. Add character model to Assets
2. Configure in config.txt using add_friend|Name
3. Set visual reference: set_friend_visual|character_name
4. Define personality with set_friend_base_prompt

Customizing AI Behavior:

• Modify prompts in config.txt
• Adjust temperature for creativity (0.1-2.0)
• Set token memory for conversation context

Credits

• Original project by Seth A. Robinson (@rtsoft)
• Modified for educational AI applications
• Character models: Reallusion Character Creator
• Lip sync: SALSA LipSync Suite (optional)

License

BSD-style attribution license. See LICENSE.md

Note: License applies to source code only. Character models and assets may have separate licenses.

Contributing

Contributions welcome! Please:

1. Fork the repository
2. Create a feature branch
3. Submit a pull request

Support

For issues and questions:

• Check the Issues page
• Review troubleshooting section above
• Check console logs in Unity for detailed errors

Roadmap

• WebGL build support
• Additional character models
• Voice activity detection
• Conversation history export
• Multi-student support
• VR/AR integration

---

Note: Requires active API keys and internet connection. This is an educational technology demonstration project.