SkyPoint is a gesture-controlled virtual mouse that allows users to control their computer without touching a physical mouse or trackpad.
Using computer vision and real-time hand tracking, SkyPoint detects hand gestures from a webcam and converts them into mouse actions such as cursor movement, clicking, dragging, and right-clicking.
This project demonstrates touchless human–computer interaction (HCI) using Python, OpenCV, and MediaPipe.
SkyPoint is currently in a demo / beta stage.
This version focuses on demonstrating the core concept of gesture-based mouse control using hand tracking.
The project will continue evolving as improvements and refinements are made over time.
Detects 21 hand landmarks using MediaPipe for accurate gesture recognition.
| Gesture | Action |
|---|---|
| Move Index Finger | Move Cursor |
| Thumb + Index Pinch | Left Click |
| Hold Pinch | Drag |
| Pinky Up + Fist | Right Click |
Uses an adaptive smoothing algorithm to reduce cursor jitter and improve precision.
Maps camera coordinates to the entire screen resolution.
Supports detection for:
- Tap gestures
- Hold gestures
- Drag gestures
- Right-click gestures
The system displays:
- Virtual cursor marker
- Gesture instructions
- Pinch distance indicator
SkyPoint processes webcam input through the following pipeline:
Frames are captured from the webcam using OpenCV.
MediaPipe Hands detects and tracks 21 hand landmarks.
Distances between landmarks determine gestures.
pinch_dist = get_normalized_distance(hand, 4, 8)The index fingertip coordinates are mapped to the screen coordinates so that hand movement directly controls the mouse pointer.
This converts the camera position of the index finger into the actual screen cursor position.
An Exponential Moving Average (EMA) smoothing algorithm is used to make cursor movement natural and reduce jitter.
| Technology | Purpose |
|---|---|
| Python | Core programming |
| OpenCV | Webcam processing |
| MediaPipe | Hand landmark detection |
| PyAutoGUI | Mouse automation |
| Math | Distance calculations |
git clone https://github.com/yourusername/skypoint.git
cd skypoint
pip install opencv-python mediapipe pyautogui
Or install from requirements file:
pip install -r requirements.txt
python SkyPoint.py
Your webcam will start and SkyPoint will begin detecting gestures.
| Gesture | Action |
|---|---|
| Move Index Finger | Move Cursor |
| Thumb + Index Pinch | Left Click |
| Hold Pinch (>0.5s) | Drag |
| Pinky Up + Closed Fingers | Right Click |
You can adjust gesture sensitivity in the source code.
| Parameter | Description |
|---|---|
| SMOOTHING FACTOR | Controls cursor smoothness |
| PINCH THRESHOLD | Pinch detection sensitivity |
| DRAG THRESHOLD | Drag activation sensitivity |
| CLICK COOLDOWN | Prevents rapid repeated clicks |
SkyPoint
│
├── skypoint.py
├── README.md
├── requirements.txt
Contributions are welcome.
Steps:
- Fork the repository
- Create a new branch
- Make your changes
- Submit a pull request
This project is licensed under the MIT License.
Developed by Charan Bantumilli