Table of Contents
Have you struggled with tongue slips?
Done either by your favourite streamer or perhaps even yourself?
Happens occasionally but when it does, it calls for disaster for the creator. One wrong word could mean losing portions of your audience, or worse, a long developed career.
This is where StreamGuard come to save the day...
StreamGuard is an innovative content moderation tool designed to enhance the live streaming experience by filtering out unwanted language in real-time. Whether you’re a Twitch streamer aiming to maintain a family-friendly channel or a broadcaster ensuring professional standards during live interviews, StreamGuard has you covered.
In the fast-paced world of live streaming, maintaining a clean and professional broadcast can be challenging. StreamGuard provides a reliable solution to prevent unintended slip-ups from reaching your audience, protecting your brand and ensuring a positive viewing experience for all.
We built StreamGuard's frontend using Next.js and Tailwind CSS for smooth transitions, creating a visually appealing user interface, all deployed on Vercel for easy updates. The backend, powered by Flask, handles HTTP requests, while PyDub and PyAudio manage audio processing. We leveraged PyTorch for machine learning alongside with OpenAI’s Whisper providing real-time speech recognition for accurate transcription and filtering. Real-time communication is facilitated by Socket, ensuring minimal latency. Finally, our final goal is for StreamGuard to be deployed on Google Cloud Platform, where it can thrive in larger clusters, taking advantage of the larger AI models via more resources.
As you might imagine, with one big difficult problem, you must first break it down into many manageable chunks. So that is exactly what we did! Our software breaks down the live feed into 1s to sub 1s segments which are processed individually then put together to create the final transcript you see. The system is intelligent and hence the chunk cuts happen when you finish the word you are speaking, i.e. the instant before you say the next word in your sentence. This opens many gateways and possibilities for us, concurrent-programming/parallel-computing, ability for APIs and faster response times are a few to mention. But for this project we are using concurrent programming at small scale to get the job done.
 BK Kang |
 Ashton Sicard |
 Hossein Molavi |
 Joel Jishu |
- Real-Time Audio Transcribing
- Real time processing
- Balance the model, adjusting speed to performance ratio
- Compile all of the audio chunks back to singular output, making it user friendly
- Detection of blacklisted words
- Detection of a single blacklisted word in live feed
- Integerating a quick algorithm to check for words in a list. (Hashmap!)
- Customizable Alerts (beep, mute, etc.)
- Live Translation / Expand the blacklisted words feature to multiple languages
- Ability to translate on the fly -> Need full sentences not small chunks
- Bleep out words from the translated speech
- Dashboard: A web-based dashboard for configuration and monitoring
- Integration with Major Streaming Platforms (Twitch, YouTube, etc.)