🧠 HastSanket: Real-Time Indian Sign Language to Speech Translator

HastSanket is an AI-powered real-time translator that converts Indian Sign Language (ISL) gestures into voice and text, helping deaf and mute individuals communicate effortlessly in day-to-day interactions.

---

🌟 Project Vision

In a fast-paced, voice-driven world, people with speech and hearing disabilities are often left unheard. HastSanket aims to create an inclusive communication bridge between ISL users and the rest of society, enabling barrier-free interaction in schools, hospitals, offices, and more.

---

📸 Demo Screenshots

Landing Page:

---

Translate Page:

---

🚀 Features

🖐️ ISL Gesture Recognition via webcam

🧠 Machine Learning Model for dynamic gesture interpretation

🔊 Real-time Voice Output using browser Speech Synthesis

📜 Live Text Captions for each detected sign

💻 Modern UI with React + Tailwind CSS

🔗 FastAPI WebSocket Backend for real-time communication

🔧 Tech Stack

Frontend	Backend	ML/AI
React.js	FastAPI (Python)	TensorFlow (Keras)
Tailwind CSS	WebSockets	MediaPipe (hand tracking)
Web Speech API	OpenCV	NumPy, deque

🏁 Setup & Run Locally

🔹 Frontend

cd client
npm install
npm start

🔹 Backend

cd python_script
python -m venv venv
venv\Scripts\activate      # Windows
pip install -r requirements.txt
python server.py

Note: Ensure webcam is connected. Trained model and label map should exist in the model/ folder.

🧠 How It Works

1. MediaPipe tracks hand landmarks in real-time.

2. Sequences of landmarks (40 frames) are passed to a trained LSTM model.

3. The model predicts the gesture, which is mapped to a label.

4. The result is sent via WebSocket to the React frontend.

5. The app displays text and speaks it using the browser's Text-to-Speech API.

📈 Roadmap

Real-time ISL gesture capture

LSTM model training and prediction

FastAPI WebSocket communication

UI integration with live webcam and captions

📦 Expand vocabulary beyond basic phrases

🤝 Support for custom user gestures

📱 Convert to PWA / Mobile App

---

🤝 Team Script_It

🐍 Python/ML:

- Priyanka Bhandari

---

🧑‍💻 React Devs:

-Saloni Suthar

- Sahil Salap

---

📝 Final Note:

Built with ❤️ for Code for Bharat Season 2 | Team: Script_It