Update nemotron-speech-streaming-en-0.6b#3555
Conversation
dosubot
Bot
added
the
size:L
|
Caution Review failedPull request was closed or merged during review 📝 WalkthroughWalkthroughThe PR extends the nemotron-speech-streaming-en-0.6b model support by introducing multi-chunk-size exports (80/160/560/1120ms) with int8 quantization variants, updating the NeMo installation to PyPI, registering four new model variants in the APK script and Kotlin API, and modifying the transducer decoder to emit multiple symbols per frame. Changes
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~35 minutes Possibly related PRs
Suggested labels
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces support for Nemotron speech streaming models with latencies of 80ms, 160ms, 560ms, and 1120ms by updating the APK generation script, ONNX export utility, and Kotlin API. The C++ engine was also modified to support multiple symbols per frame during greedy search. Feedback identifies a critical issue where the 80ms model's chunk shift calculation results in zero, which could lead to infinite loops in the recognizer. Additionally, suggestions were made to improve the robustness of shell commands in the export script and to make the frame symbol limit configurable.
| chunk_size = ms // 80 - 1 | ||
| print("chunk_size", chunk_size) | ||
| asr_model.encoder.set_default_att_context_size([70, chunk_size]) | ||
|
|
||
| print("streaming_cfg", asr_model.encoder.streaming_cfg) | ||
| print("att_context_size", asr_model.encoder.att_context_size) | ||
| print( | ||
| "pre_encode_cache_size", | ||
| asr_model.encoder.streaming_cfg.pre_encode_cache_size, | ||
| ) | ||
|
|
||
| print(meta_data) | ||
| if isinstance(asr_model.encoder.streaming_cfg.pre_encode_cache_size, list): | ||
| pre_encode_cache_size = ( | ||
| asr_model.encoder.streaming_cfg.pre_encode_cache_size[1] | ||
| ) | ||
| else: | ||
| pre_encode_cache_size = ( | ||
| asr_model.encoder.streaming_cfg.pre_encode_cache_size | ||
| ) | ||
|
|
||
| if isinstance(asr_model.encoder.streaming_cfg.chunk_size, list): | ||
| chunk_size = asr_model.encoder.streaming_cfg.chunk_size[1] | ||
| else: | ||
| chunk_size = asr_model.encoder.streaming_cfg.chunk_size | ||
|
|
||
| window_size = chunk_size + pre_encode_cache_size | ||
|
|
||
| print("chunk_size", chunk_size) | ||
| print("pre_encode_cache_size", pre_encode_cache_size) | ||
| print("window_size", window_size) | ||
|
|
||
| chunk_shift = chunk_size |
There was a problem hiding this comment.
The chunk_size calculation ms // 80 - 1 results in 0 for the 80ms model. This value is assigned to chunk_shift at line 103 and subsequently stored in the model metadata. In the sherpa-onnx C++ engine (e.g., in online-recognizer-transducer-nemo-impl.h), chunk_shift is used to advance the frame pointer: ss[i]->GetNumProcessedFrames() += chunk_shift;. If chunk_shift is 0, the recognizer will enter an infinite loop and hang. Please ensure chunk_size and chunk_shift in the metadata represent the actual number of frames (e.g., 1 output frame for 80ms if the subsampling factor is 8).
| os.system( | ||
| f""" | ||
| mkdir {ms} | ||
| mv -v *.onnx {ms} | ||
| mv -v *.data {ms} | ||
| ls -lh {ms} | ||
| rm Constant_*_attr__value | ||
| rm onnx__MatMul_* | ||
| rm layers.*.conv* | ||
| rm pre_encode.conv.*.weight | ||
| ls -lh | ||
| """ | ||
| ) |
There was a problem hiding this comment.
The os.system call uses a multi-line string with significant leading indentation, which is passed literally to the shell. While most shells handle this, it is brittle and messy. Furthermore, the rm commands should use the -f flag to prevent the script from failing if the temporary files (like Constant_*_attr__value) are not present. Consider cleaning up the indentation and using -f.
os.system(f"mkdir -p {ms}")
os.system(f"mv -v *.onnx {ms}/")
os.system(f"mv -v *.data {ms}/")
os.system("rm -f Constant_*_attr__value onnx__MatMul_* layers.*.conv* pre_encode.conv.*.weight")|
|
||
| bool emitted = false; | ||
|
|
||
| int32_t max_symbols_per_frame = 10; |
There was a problem hiding this comment.
The max_symbols_per_frame is hardcoded to 10. It is recommended to make this value configurable via OnlineRecognizerConfig to allow users to balance between recognition thoroughness and computational cost, especially for real-time streaming applications where latency is critical.
1 participant
Fixes #3408
Please see the doc at
https://k2-fsa.github.io/sherpa/onnx/nemo/nemotron-streaming.html#nemotron-asr-streaming
cc @wangfeng35
Summary by CodeRabbit
New Features