Skip to content

[WIP][CI] Add CI for Ming Omni#348

Open
edwingao28 wants to merge 10 commits intosgl-project:mainfrom
edwingao28:feat/ming-mmmu-ci
Open

[WIP][CI] Add CI for Ming Omni#348
edwingao28 wants to merge 10 commits intosgl-project:mainfrom
edwingao28:feat/ming-mmmu-ci

Conversation

@edwingao28
Copy link
Copy Markdown
Collaborator

@edwingao28 edwingao28 commented Apr 25, 2026

Motivation

Closes #295. This PR builds the multi-stage Ming-flash-omni-2.0 CI suite mirroring the qwen3-omni structure and covers the following metrics: TTFA, RTF, WER, plus MMMU and MMSU accuracy

Modification

Workflow DAG

  docs ──► stage-1-thinker ──► stage-2-tts
                            ├─► stage-3-mmmu
                            └─► stage-4-mmsu

stage-1-thinker — text generation

Pure text path, thinker only. tests/test_model/test_ming_omni_thinker_length.py exercises Ming thinker across short / medium / long prompts and asserts response shape.

stage-2-tts — text-to-speech

RTF performance + non-stream TTFA baseline diagnostic + WER accuracy. Single job with two tests sharing one server (test_tts_non_streaming_perf + test_tts_wer).

tests/test_model/test_ming_omni_tts_ci.py uses voice_clone=False

stage-3-mmmu — image recognition

Image+text → text via thinker only. tests/test_model/test_ming_omni_mmmu_ci.py runs mmmu-ci-50 with warmup=2

stage-4-mmsu — audio-in understanding

Text+audio → text via MMSU benchmark with modalities="text+audio". This covers the #295 ASR requirement as a superset tests/test_model/test_ming_omni_mmsu_ci.py runs thinker-only (talker OFF).

Accuracy Test

Benchmark & Profiling

benchmark & ci threshold in progress

Checklist

  • Format your code according with pre-commit.
  • Add unit tests.
  • Update documentation / docstrings / example tutorials as needed.
  • Provide throughput / latency benchmark results and accuracy evaluation results as needed.
  • For reviewers: If you haven't made any contributions to this PR and are only assisting with merging the main branch, please remove yourself as a co-author when merging the PR.

@edwingao28 edwingao28 changed the title [WIP][Feat] Add CI for Ming Omni [WIP][CI] Add CI for Ming Omni Apr 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] Ming-Omni ci test supporting

1 participant