Add configurable Whisper decoding options to parallel diarization#362
Add configurable Whisper decoding options to parallel diarization#362Alexandrsv wants to merge 2 commits into
Conversation
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request enhances the parallel diarization pipeline by introducing configurable Whisper decoding options. These new parameters provide greater flexibility for users to tune and optimize transcription quality without altering the core diarization logic. Additionally, it improves repository hygiene by ensuring virtual environment files are ignored. Highlights
🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Changelog
Activity
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces configurable Whisper decoding options to diarize_parallel.py, which is a valuable addition for tuning transcription quality. The changes are well-structured, using a dictionary to pass the new options, which keeps the code clean. My review includes a few suggestions for improvement: using an argument group for the new CLI options to enhance readability, modifying the .gitignore entry to be more specific, and extending the temperature argument to support multiple values for temperature fallback, a feature supported by faster-whisper.
| *.mp3 | ||
| *.m4a | ||
|
|
||
| venv No newline at end of file |
There was a problem hiding this comment.
To be more specific and avoid accidentally ignoring a file named venv, it's a good practice to add a trailing slash to directory entries in .gitignore.
venv/
| parser.add_argument( | ||
| "--beam-size", | ||
| type=int, | ||
| default=5, | ||
| help="Beam size for Whisper decoding.", | ||
| ) | ||
|
|
||
| parser.add_argument( | ||
| "--best-of", | ||
| type=int, | ||
| default=5, | ||
| help="Number of candidates considered during Whisper decoding.", | ||
| ) | ||
|
|
||
| parser.add_argument( | ||
| "--patience", | ||
| type=float, | ||
| default=2.0, | ||
| help="Beam search patience for Whisper decoding.", | ||
| ) | ||
|
|
||
| parser.add_argument( | ||
| "--compression-ratio-threshold", | ||
| type=float, | ||
| default=2.4, | ||
| help="Threshold used to reject repetitive Whisper outputs.", | ||
| ) | ||
|
|
||
| parser.add_argument( | ||
| "--log-prob-threshold", | ||
| type=float, | ||
| default=-1.0, | ||
| help="Threshold used to reject low-confidence Whisper outputs.", | ||
| ) | ||
|
|
||
| parser.add_argument( | ||
| "--condition-on-previous-text", | ||
| action=argparse.BooleanOptionalAction, | ||
| default=False, | ||
| help="Condition Whisper decoding on the previous segment text.", | ||
| ) | ||
|
|
||
| parser.add_argument( | ||
| "--temperature", | ||
| type=float, | ||
| default=0.0, | ||
| help="Sampling temperature for Whisper decoding.", | ||
| ) | ||
|
|
There was a problem hiding this comment.
To improve readability and organization, consider grouping all these new Whisper decoding options under a specific argument group.
Additionally, faster-whisper supports a tuple of temperatures for fallback. You can enable this by allowing multiple values for the --temperature argument using nargs='+'. A corresponding change will be needed where this argument is used.
decoding_options = parser.add_argument_group("Whisper decoding options")
decoding_options.add_argument(
"--beam-size",
type=int,
default=5,
help="Beam size for Whisper decoding.",
)
decoding_options.add_argument(
"--best-of",
type=int,
default=5,
help="Number of candidates considered during Whisper decoding.",
)
decoding_options.add_argument(
"--patience",
type=float,
default=2.0,
help="Beam search patience for Whisper decoding.",
)
decoding_options.add_argument(
"--compression-ratio-threshold",
type=float,
default=2.4,
help="Threshold used to reject repetitive Whisper outputs.",
)
decoding_options.add_argument(
"--log-prob-threshold",
type=float,
default=-1.0,
help="Threshold used to reject low-confidence Whisper outputs.",
)
decoding_options.add_argument(
"--condition-on-previous-text",
action=argparse.BooleanOptionalAction,
default=False,
help="Condition Whisper decoding on the previous segment text.",
)
decoding_options.add_argument(
"--temperature",
type=float,
nargs="+",
default=[0.0],
help="Sampling temperature for Whisper decoding. Can be a single value or a space-separated list of values for temperature fallback.",
)| "compression_ratio_threshold": args.compression_ratio_threshold, | ||
| "log_prob_threshold": args.log_prob_threshold, | ||
| "condition_on_previous_text": args.condition_on_previous_text, | ||
| "temperature": args.temperature, |
There was a problem hiding this comment.
To correctly handle the list of temperatures from argparse (as suggested in another comment), you should convert it to a tuple if it contains multiple values, or extract the single float value. The transcribe function expects a float or a tuple for the temperature argument.
"temperature": tuple(args.temperature) if len(args.temperature) > 1 else args.temperature[0],1 participant
This PR adds CLI options for Whisper decoding in
diarize_parallel.py.These parameters are needed for tuning and improving transcription quality, while keeping the existing parallel diarization pipeline unchanged.
It also adds the local virtualenv directory to
.gitignore.