AI Virtual Mouse using Eye Gestures

A sophisticated eye-tracking system that allows users to control their computer mouse using eye movements and blink gestures. This project provides a production-ready solution for hands-free computer interaction.

🚀 Features

Core Functionality

• Eye Tracking: Real-time iris and gaze direction tracking using webcam
• Mouse Movement: Smooth cursor movement based on eye gaze direction
• Blink Detection:
  • Single blink = Left click
  • Double blink = Right click
  • Long blink = Drag and drop
• Scroll Control: Vertical scrolling when looking up or down
• Anti-Jitter: Advanced smoothing algorithms to prevent cursor jitter

Advanced Features

• Debug Mode: Visual calibration interface with landmark detection
• Cross-Platform: Compatible with Linux, Windows, and macOS
• Configurable: Customizable sensitivity, thresholds, and behavior
• Production Ready: Optimized performance with minimal CPU usage
• Accessibility: Designed for users with mobility limitations

📋 Requirements

System Requirements

• Python 3.8 or higher
• Webcam (USB or built-in)
• Minimum 4GB RAM
• Modern CPU (Intel i5/AMD Ryzen 5 or better recommended)

Dependencies

See requirements.txt for complete list of Python packages.

🛠️ Installation

1. Clone the Repository

git clone https://github.com/yourusername/ai-virtual-mouse.git
cd ai-virtual-mouse

2. Create Virtual Environment

python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

3. Install Dependencies

pip install -r requirements.txt

4. Platform-Specific Setup

Linux

# Install additional system dependencies
sudo apt-get update
sudo apt-get install python3-opencv libgl1-mesa-glx

Windows

• Ensure Visual Studio Build Tools are installed
• Install MediaPipe dependencies through pip

macOS

# Install additional dependencies
brew install opencv

🎯 Usage

Production Mode

Run the main application:

python src/main.py

Debug/Calibration Mode

For calibration and testing:

python src/debug.py

Configuration

Edit config/settings.json to customize:

• Sensitivity settings
• Blink detection thresholds
• Mouse movement smoothing
• Scroll sensitivity

⚙️ Configuration

Settings File (config/settings.json)

{
  "eye_tracking": {
    "sensitivity": 1.0,
    "smoothing_factor": 0.7,
    "dead_zone": 0.1
  },
  "blink_detection": {
    "single_blink_threshold": 0.3,
    "double_blink_threshold": 0.5,
    "long_blink_threshold": 1.0
  },
  "mouse_control": {
    "movement_speed": 2.0,
    "scroll_sensitivity": 1.0,
    "click_delay": 0.1
  },
  "debug": {
    "show_landmarks": false,
    "show_blink_detection": false,
    "fps_display": true
  }
}

🎮 Controls

Gesture	Action
Eye Movement	Move mouse cursor
Single Blink	Left click
Double Blink	Right click
Long Blink	Drag and drop
Look Up	Scroll up
Look Down	Scroll down
Look Left/Right	Move cursor horizontally

🏗️ Project Structure

ai_virtual_mouse/
├── src/
│   ├── core/
│   │   ├── eye_tracker.py      # Core eye tracking logic
│   │   ├── mouse_controller.py # Mouse control implementation
│   │   └── blink_detector.py   # Blink detection algorithms
│   ├── ui/
│   │   ├── debug_interface.py  # Debug/calibration UI
│   │   └── main_interface.py   # Main application UI
│   ├── calibration/
│   │   ├── calibrator.py       # Calibration system
│   │   └── settings_manager.py # Configuration management
│   ├── main.py                 # Production application
│   └── debug.py                # Debug/calibration application
├── utils/
│   ├── image_processing.py     # Image processing utilities
│   ├── smoothing.py            # Smoothing algorithms
│   └── platform_utils.py      # Cross-platform utilities
├── config/
│   └── settings.json           # Configuration file
├── data/
│   └── calibration_data/       # Calibration data storage
├── tests/
│   ├── test_eye_tracker.py
│   ├── test_blink_detector.py
│   └── test_mouse_controller.py
├── docs/
│   ├── API.md
│   └── CALIBRATION.md
├── requirements.txt
└── README.md

🔧 Development

Running Tests

python -m pytest tests/

Code Style

black src/ tests/
flake8 src/ tests/

🚀 Future Enhancements

Planned Features

• Voice Commands: Integration with speech recognition
• Multi-Monitor Support: Extended desktop navigation
• Gesture Recognition: Hand gesture integration
• Machine Learning: Personalized calibration using ML
• Mobile Support: Android/iOS companion app
• Web Interface: Browser-based control panel
• Accessibility Features: Enhanced accessibility options
• Performance Optimization: GPU acceleration support

Advanced Features

• Multi-User Support: User profiles and switching
• Cloud Calibration: Sync settings across devices
• Analytics: Usage tracking and optimization
• Plugin System: Extensible architecture
• API Integration: Third-party application support

🤝 Contributing

1. Fork the repository
2. Create a feature branch (git checkout -b feature/amazing-feature)
3. Commit your changes (git commit -m 'Add amazing feature')
4. Push to the branch (git push origin feature/amazing-feature)
5. Open a Pull Request

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

🙏 Acknowledgments

• MediaPipe for facial landmark detection
• OpenCV for computer vision capabilities
• PyAutoGUI for cross-platform mouse control
• The open-source community for inspiration and support

📞 Support

• Issues: GitHub Issues
• Discussions: GitHub Discussions
• Email: [email protected]

🔬 Research

This project is based on research in:

• Computer Vision
• Human-Computer Interaction
• Accessibility Technology
• Eye Tracking Systems

---

Made with ❤️ for accessibility and innovation