feat(whisper): Enhance model selection with benchmarking and resource detection#459
Open
Yogesh-333 wants to merge 7 commits into
Open
feat(whisper): Enhance model selection with benchmarking and resource detection#459Yogesh-333 wants to merge 7 commits into
Yogesh-333 wants to merge 7 commits into
Conversation
… detection This commit enhances the Whisper model selection process by introducing benchmarking and system resource detection: - Implements a benchmarking feature to measure the performance of different Whisper models on the user's system, storing the results in `app_settings.editable_settings` for future suggestions. - Uses system resource detection to dynamically categorize Whisper models, providing tailored recommendations based on the user's hardware. - Adds a fallback mechanism for system resource detection, ensuring robust model suggestions even when detection fails. - Improves the user experience by recommending optimal models based on benchmark results and detected system resources. - Stored Benchmark result to avoid testing all the time.
Reviewer's Guide by SourceryThis pull request enhances the Whisper model selection process by implementing benchmarking and system resource detection. It provides users with more informed choices based on their system's capabilities and measured model performance. A fallback mechanism is included for cases where system resource detection fails. A message box alerts the user when the current model's performance is not optimal, suggesting alternative models. Sequence diagram for Whisper model benchmarkingsequenceDiagram
participant User
participant Client
participant SettingsWindowUI
participant app_settings
participant faster_whisper
User->>Client: Selects a Whisper model in settings
Client->>SettingsWindowUI: Updates the selected model
Client->>Client: Loads STT model
Client->>Client: Calls benchmark_model()
Client->>app_settings: Retrieves current model
alt Test results exist
app_settings->>Client: Retrieves test results
Client->>Client: Checks if last time taken >= 10 seconds
alt Time taken is not optimal
Client->>User: Shows Model Performance Alert
end
else Test results do not exist
Client->>faster_whisper: Transcribes test audio
faster_whisper-->>Client: Returns transcription time
Client->>app_settings: Stores transcription time
Client->>Client: Checks if time taken >= 10 seconds
alt Time taken is not optimal
Client->>User: Shows Model Performance Alert
end
end
File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
There was a problem hiding this comment.
Hey @Yogesh-333 - I've reviewed your changes - here's some feedback:
Overall Comments:
- Consider adding a unit test for the
get_system_memory_infofunction to ensure it returns reasonable values across different platforms. - The
benchmark_modelfunction directly accesses and modifiesapp_settings.editable_settings; consider using a more structured approach for managing application settings.
Here's what I looked at during the review
- 🟡 General issues: 1 issue found
- 🟢 Security: all looks good
- 🟢 Testing: all looks good
- 🟢 Complexity: all looks good
- 🟢 Documentation: all looks good
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
|
New and removed dependencies detected. Learn more about Socket for GitHub ↗︎
🚮 Removed packages: pypi/pydantic-core@2.23.4 |
This commit addresses two code quality issues: - Replaces a bare `except:` block with `except Exception as e:` in the GPU detection code. This improves exception handling by catching specific exceptions and logging error messages for debugging. - Replaces string concatenation with an f-string in the model selection code. This improves code readability and follows Python best practices for string formatting.
ItsSimko
requested changes
Mar 12, 2025
Collaborator
|
Feel free to discuss or disagree with any of my comments |
Collaborator
|
Do we maybe want to merge this into dev @yingbull? Since the current version should be stable. We can get this along with some of Xuns stuff and mac compatibility in dev. and then I think we can make a test build off of dev branch with out affect main code. |
This commit introduces several improvements to the Whisper model handling: - Splits `get_system_memory_info` into `get_system_memory` and `get_system_vram` functions, allowing for independent retrieval of RAM and VRAM information. In a seperate system_utils file. - Creates a `WhisperModelEnum` class to represent Whisper models and their attributes, enhancing code organization and readability. - Updates `generate_whisper_model_list` to utilize the `WhisperModelEnum` class, making the code more structured and maintainable.
- Modified `update_whisper_model` to clean the new model value by removing any spaces and indicators.
Author
|
@yingbull Resolved the conflicts |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description:
Enhancements to the Whisper model selection process, providing users with more informed choices based on their system's capabilities and measured model performance.
Changes:
app_settings.editable_settingsunder theWHISPER_PERFORMANCE_TEST_RESULTSkey, keyed by the current model name.Summary by Sourcery
Enhance the Whisper model selection process by providing users with informed choices based on system capabilities and measured model performance. This includes benchmarking models, detecting system resources, and providing model recommendations.
New Features: