[Test] Add e2e tests for Qwen3-TTS speech endpoint#1206
[Test] Add e2e tests for Qwen3-TTS speech endpoint#1206linyueqian wants to merge 9 commits intovllm-project:mainfrom
Conversation
Signed-off-by: linyueqian <linyueqian@outlook.com>
|
could you please also add the api server endpoint in doc? |
There was a problem hiding this comment.
Pull request overview
This PR adds comprehensive e2e tests for the Qwen3-TTS model's /v1/audio/speech endpoint. The tests were created in response to bug #1159, where unit tests with mocks didn't catch real behavior issues. The new tests verify actual model inference without mocks.
Changes:
- Adds e2e tests for Qwen3-TTS CustomVoice and VoiceDesign models
- Tests
/v1/audio/speech,/v1/audio/voices, and/v1/modelsendpoints - Includes regression test for multimodal_output bug that caused "TTS model did not produce audio output" error
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Signed-off-by: linyueqian <linyueqian@outlook.com>
a3d7d9b to
79677e2
Compare
OK. I have added that doc. |
Signed-off-by: linyueqian <linyueqian@outlook.com>
Signed-off-by: linyueqian <linyueqian@outlook.com>
Signed-off-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
Signed-off-by: linyueqian <linyueqian@outlook.com>
Signed-off-by: linyueqian <linyueqian@outlook.com>
Thank you! I have added them. |
|
fix ci please |
Need to merge #1203 first. The test catches that bug so it's expected to fail without it. |
PLEASE FILL IN THE PR DESCRIPTION HERE ENSURING ALL CHECKLIST ITEMS (AT THE BOTTOM) HAVE BEEN CONSIDERED.
Purpose
Add e2e tests for Qwen3-TTS
/v1/audio/speechendpoint. The existing unit tests used mocks that didn't match real behavior, allowing bugs like #1159 to slip through undetected.Also add doc for tts.
Test Plan
Test Result
Verified the test correctly catches the multimodal_output bug:
Without fix (main branch):
FAILED tests/e2e/online_serving/test_qwen3_tts.py::TestQwen3TTSCustomVoice::test_speech_english_basic
Server returns {"error":{"message":"TTS model did not produce audio output."}} instead of WAV audio.
With fix:
PASSED - Valid WAV audio returned (87KB, 24000Hz mono)
Essential Elements of an Effective PR Description Checklist
supported_models.mdandexamplesfor a new model.BEFORE SUBMITTING, PLEASE READ https://github.com/vllm-project/vllm-omni/blob/main/CONTRIBUTING.md (anything written below this line will be removed by GitHub Actions)