Test the best_model_dns3.tar export to onnx and stream

[lloovve]

denoise_audio_streaming.py

export_streaming_onnx.py

gtcrn_micro_stream_opset11.zip

I test best_model_dns3.tar export onnx

source file
souce_mix.zip

GTCRN-Micro onnx denoise output wav file
mix.zip

GTCRN onnx denoise output wav file
result.zip

[lloovve]

单文件降噪

python denoise_audio_streaming.py \
    --input gtcrn_micro/examples/gtcrn_micro/noisy1.wav \
    --output output_test/clean1_streaming.wav

批量降噪

python denoise_audio_streaming.py \
    --input-dir gtcrn_micro/examples/gtcrn_micro/ \
    --output-dir output_test_batch/

[bglid]

Thank you for sharing this work! I just went through and listened to the audio and looked through the code you shared. That's great to see - So it looks like the streaming version still is having solid Speech Enhancement performance now, even in the ONNX format. That's great to see! I am going to add my conversion scripts and metric results for the ONNX streaming model. I will actually reference the way you did the onnx streaming conversion, if that's okay?

Then it'll be wrangling the int8 quantized export of the ONNX model and seeing how that one performans. Currently still trying to fix that part up because there are still some performance issues for the tflite model. I think it has to do with the calibration data.

[lloovve]

Okay, all the code can be referenced and shared.

[bglid]

Awesome, thank you!

[bglid]

So I know I just posted about this in the other issue but I closed that out so I will share again in here:

I have been constantly running into issues converting the onnx model properly to a tflite format. When I have been able to, I have only been able to create dynamic quantized, float32, and float 16 .tflite files. Even then, their performance drops substantially.

Some of the issues stem from TF representing things differently (NHWC vs NCHW in PyTorch) - however others come from still ops that are creating issues in the quantization stage. I am unsure if the ConvTranspose2d is viable. There are other issues arising as well. I am starting to wonder whether this approach for MCU deployable SE is the right way forward. In other words, The model may just have too many moving pieces and it's design may just have too much to wrangle for an MCU deployment - which I am not even sure of the performance.

Unfortunately, I am going to have to step away from this project soon due to other upcoming projects. With that, I am going to rename it to GTCRN-TCN, since what this achieved was a rebuild of GTCRN with a purely convolutional approach, and write up the changes and the initial goal. That said, I do want to encourage you to try and do this last quantization stage with the provided new architecture, both/either the PyTorch model or the ONNX model. If you do manage to get this to work, please let me know!

I am going to likely be working on some other approaches to MCU Speech Enhancement/Audio denoising, if I manage to get anything else to work in a different project, I will let you know!