[REQ] Add Speaker Diarization to Real-time Transcription

[TannerOdenthal]

Is your feature request related to a problem? Please describe.

Yes, the current real-time transcription and translation features are very useful, but they lack the ability to identify who is speaking. In conversations with multiple participants (e.g., meetings, Discord calls, podcasts), it's difficult to follow the transcript without knowing which person each segment of text belongs to. The transcribed output is just a continuous stream of text, making it hard to track the flow of the conversation when multiple people are talking.

Describe the solution you'd like

I'd like to request the addition of speaker diarization to the real-time transcription feature. This would involve:

1. Integrating a speaker diarization library/model: The application should use a library like pyannote.audio (or a similar robust diarization solution) to analyze the incoming audio stream and identify different speakers.
2. Associating Speaker Labels with Transcribed Text: The transcribed text segments should be labeled with the corresponding speaker (e.g., "Speaker 1:", "Speaker 2:", etc.). Ideally, the UI would visually differentiate the text from different speakers (e.g., using different colors, indentation, or prefixes).
3. Real-time Diarization: The diarization should happen in real-time, along with the transcription, so that speaker labels are displayed as the conversation progresses.
4. Settings/Configuration (Optional, but Highly Desirable):
   • Enable/Disable Diarization: A toggle in the settings to enable or disable speaker diarization.
   • Diarization Model Selection (Advanced): Potentially allow users to choose between different diarization models (if the chosen library supports it) to balance accuracy and performance.
   • Speaker Label Customization (Advanced): Potentially allow users to manually assign names to identified speakers (e.g., replace "Speaker 1" with "John").
   • VRAM usage warning/suggestion: If the user enables Diarization, perhaps pop up a warning about increased vram usage and a suggestion to use smaller models to minimize issues if they occur.

Describe alternatives you've considered

1. Manual Speaker Identification: Users could manually identify speakers after using Generative AI or by simply by going in myself and marking different speakers with different colored text after exporting the transcript.
2. Using a separate diarization tool after recording: Users could record the audio and then process it with a separate diarization tool offline. This is not ideal for real-time use cases where immediate feedback is needed.

Additional context

• Target Use Cases: This feature would be extremely valuable for:
  • Online meetings and conferences.
  • Discord calls and voice chats.
  • Podcast transcription.
  • Live captioning of multi-person conversations.
  • Accessibility for users who are deaf or hard of hearing.
• Technical Considerations:
  • Real-time Performance: The diarization process should be optimized to minimize latency and maintain real-time performance.
  • Accuracy: Diarization is not perfect. The implementation should strive for reasonable accuracy, but users should be aware of potential limitations.
  • Open Source: The use of an open-source diarization library like pyannote.audio is preferred to maintain the open-source nature of Speech-Translate.