Skip to content

refactor(runtime): remove Python interpreter bridges - #1003

Open
yifeif-nv wants to merge 1 commit into
NVIDIA:mainfrom
yifeif-nv:agent/remove-runtime-hf-python
Open

refactor(runtime): remove Python interpreter bridges#1003
yifeif-nv wants to merge 1 commit into
NVIDIA:mainfrom
yifeif-nv:agent/remove-runtime-hf-python

Conversation

@yifeif-nv

@yifeif-nv yifeif-nv commented Aug 22, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Background

Native bundle inference still exposed a Hugging Face Python interpreter path through the CLI, C++ and C APIs, the Python wrapper, benchmarks, and E2E orchestration. PersonaPlex could launch a tokenizer subprocess, while Qwen3-Omni could launch a persistent Python Talker worker. That made Python availability part of deployed inference behavior.

Exit Criteria

  • Native runtime and public inference APIs accept no Python interpreter path and launch no Python subprocess.
  • A source guard fails if an interpreter marker, embedded-Python API, or process-spawn bridge is added under native runtime sources.
  • Built native runtime ELFs reject direct libpython dependencies.
  • Build, reference, and diagnostic Python environments remain available without being forwarded to trtmc runtime commands.
  • Pre-1.0 compatibility shims are not carried for the retired API, context, or plugin layouts.

Implementation

  • Removed --hf-python, hf_python fields, and positional interpreter arguments from the CLI, C++ API, C ABI, Python wrapper, examples, benchmarks, validation tools, and E2E runtime commands.
  • Removed the PersonaPlex runtime tokenizer subprocess. PersonaPlex accepts pre-tokenized speech_text_prompt_ids and fails closed when a bundle configures only speech_system_prompt. Prompt IDs use the shared JSON helper, and the unrelated output-planning tests formerly colocated with the subprocess seam now have a dedicated model-owned test.
  • Removed the Qwen3-Omni Python Talker worker. Audio generation remains fail-closed until a native Talker exists.
  • Added a native Qwen3-Omni text path using only the bundle tokenizer and TensorRT Thinker. Its active premerge/nightly case is explicitly L4 runtime-invariant evidence: non-empty native text, not HF parity, semantic correctness, audio evidence, or a performance claim.
  • Narrowed the active Qwen3-Omni bundle contract to Thinker plus tokenizer only. Dormant Vision, Audio Encoder, Talker, and Code2Wav helpers are not loaded, built, packaged, or deserialized by the text-only path.
  • Removed stale audio validation/performance claims, stale Qwen-Image kind="hf_python" metadata, the tokenizer bridge asset, dead source-directory plumbing, and misleading tokenizer fallback messages.
  • Removed retired interpreter fields and overloads without reserved slots, layout mirrors, or binary-only shims. Pre-1.0 source and binary compatibility is intentionally not preserved.
  • Added an exact-match model-plugin ABI v2 epoch. Missing or mismatched DSOs are rejected before model identity or registration entrypoints run; only the current ABI is supported.
  • Canonicalized only nested decoder fields at the bundle config root after the latest root-only JSON parser migration. Nested source config remains intact, structured encoder fields remain model-owned, generation-config EOS retains precedence, model-owned overrides remain authoritative, and zero/list token IDs are preserved.
  • Added tests/tools/test_native_runtime_python_free.py to the always-on source-quality architecture gate. Model proof also rejects direct libpython dependencies in scratch and selected-wheel runtime ELF artifacts, including the packaged benchmark worker.
  • Made model-proof orphan cleanup concurrency-safe with a bounded fail-closed wait when another removal is already in progress.

Change categories

  • Model or runtime behavior
  • Public API
  • ABI
  • Bundle or artifact format
  • Dependencies
  • Documentation only
  • CI or developer tooling

Validation

Commands and Results

  • python3 -m tools.community_ci unit --scope all: 3,688 Python tests passed, 2 skipped; 20 allocator tests passed; 40 source-only C++ platform tests passed.
  • python3 -m tools.community_ci unit --scope builder: 811 builder tests passed, 1 skipped.
  • python3 -m pytest tests/builder/test_config.py tests/builder/test_config_coverage.py tests/builder/test_generation_config_eos.py tests/tools/test_native_runtime_python_free.py -q -p no:cacheprovider: 77 passed, including the SegFormer structured-config regression and native-runtime Python-free guard.
  • PYTHONPATH=python:. python3 -m pytest tests/tools/test_github_actions_ci.py tests/tools/test_model_proof_inner.py tests/tools/test_selected_wheel_runtime.py tests/tools/test_model_proof_runner.py tests/tools/test_warm_hf_cache_static.py -q -p no:cacheprovider: 298 passed after rebasing across the CI cache and exact-case changes.
  • python3 -m tools.community_ci source-quality --base github/main: cyclomatic complexity passed with maximum CCN 10; changed-file lint/formatting passed; 163 architecture tests passed.
  • python3 -m tools.community_ci impact --base github/main: passed with mode=all and unit_scope=all.
  • cmake --build /tmp/trtmc-pr1003-host.O0u5TS --target test_c_abi_entry test_optimized_runtime_host test_model_plugin_abi test_model_plugin_loader test_personaplex_speech_output_plan -j2: passed; the loader dependency rebuilt all 84 model DSOs.
  • XDG_CACHE_HOME=/tmp/trtmc-pr1003-cache ctest --test-dir /tmp/trtmc-pr1003-host.O0u5TS --output-on-failure -R '^(test_c_abi_entry|test_optimized_runtime_host|test_model_plugin_abi|test_model_plugin_loader|test_personaplex_speech_output_plan)$': 5/5 passed.
  • Network-isolated trtmc-dev-gb300:latest build of test_personaplex_speech_pipeline and test_personaplex_speech_output_plan, followed by their focused CTest: 2/2 passed without a GPU.
  • python3 tools/model_ci.py validate: passed for 84 model families.
  • python3 tools/test_impact.py --validate: passed with existing allowlist warnings.
  • python3 tools/check_doc_file_references.py --strict website/docs: 107 documents, 0 errors, 0 warnings.
  • python3 tools/legal_headers.py --check: 0 findings.
  • git diff --check github/main...HEAD: passed.
  • nm -D -C on libtrtmc_core.so: only the current one-argument and LoadOptions load/factory symbols remain; both retired four-argument symbols are absent.
  • readelf -d plus nm -D over the fresh core and all 84 model DSOs: 85 ELFs scanned, 84 ABI entrypoints present, 0 direct libpython dependencies.

Hardware, Environment, and Revisions

  • Repository head: 08b0f9298da7bd4bfe845dfc1a8fac1511fdf512; base/main: 7e6325412b2ab9a5e04aa9fe1b3f73cdcbb53c45.
  • Full Community CPU gate: repository-managed Python 3.12 container with Transformers 5.2.0.
  • Host/static build: Ubuntu 24.04.4 LTS on AArch64; GCC 13.3; CUDA toolkit 13.0; TensorRT disabled for the aggregate model-DSO and platform-test build.
  • Focused TensorRT build: trtmc-dev-gb300:latest, TensorRT 11.1.0, CUDA toolkit 13.3, no NVIDIA driver or GPU attached; compile and CPU-only CTest only.
  • PersonaPlex checkpoint revision remains fdaf4090a61cb315c138a1faee287ffd6c716309; Qwen3-Omni checkpoint revision remains 26291f793822fb6be9555850f06dfe95f2d7e695. Neither checkpoint was loaded locally in this validation cycle.

Not Run / Remaining Gaps

  • Real checkpoint/model GPU E2E was not run locally. Exact-head repository CI runs separately.
  • Qwen3-Omni validation proves only that the native Thinker path returns non-empty text. It does not prove HF parity, semantic quality, audio, multimodal input, or performance; audio remains intentionally unavailable.
  • No release performance benchmark was run. Qwen3-Omni remains excluded until a qualified native text receipt exists.
  • ELF evidence is a direct DT_NEEDED guarantee. It does not claim a recursive third-party dependency closure or arbitrary runtime dlopen() coverage.

Notes For Future Readers

  • This is an intentional pre-1.0 source and binary break. Callers must remove --hf-python and every hf_python field or argument, then rebuild and upgrade core, CLI, wheel, bundles, and every model DSO together.
  • The runtime supports only the current model-plugin ABI epoch. It carries no compatibility branches, reserved slots, layout mirrors, or retired symbols.
  • trtmc build and test/reference helpers may still use Python. The invariant targets loaded bundle execution, not the build control plane.
  • Re-enable Qwen3-Omni audio validation only after a native Talker implementation exists.
  • Suggested review order: public API/context deletion, PersonaPlex and Qwen3-Omni fail-closed behavior, model-plugin ABI gate, then source/ELF guards and broad mechanical tooling cleanup.

Risk level

  • Low
  • Medium
  • High

High risk because this deliberately breaks pre-1.0 source, binary, plugin, and bundle compatibility across the runtime stack and removes Qwen3-Omni audio behavior. The risk is controlled by mandatory whole-release rebuilds, an exact-match ABI gate, fail-closed unsupported paths, complete local CPU gates, and exact-head CI.

@yifeif-nv
yifeif-nv force-pushed the agent/remove-runtime-hf-python branch 2 times, most recently from f1be418 to d57df2d Compare August 22, 2026 18:56
@yifeif-nv

yifeif-nv commented Aug 22, 2026

Copy link
Copy Markdown
Collaborator Author

/run-ci

@github-actions

github-actions Bot commented Aug 22, 2026 •
edited
Loading

Copy link
Copy Markdown

Community CPU

Status: PASSED

  • Exact merge: 595852df542a6e6cd9cd3db5d29db84356ad3510
  • Source quality: success
  • Ownership and impact: success
  • Unit / C++ and Python: success

All required public CPU stages passed.

View public logs

@yifeif-nv
yifeif-nv force-pushed the agent/remove-runtime-hf-python branch from d57df2d to eac14e7 Compare August 22, 2026 19:46
@yifeif-nv

yifeif-nv commented Aug 22, 2026

Copy link
Copy Markdown
Collaborator Author

/run-ci

@yifeif-nv yifeif-nv added the run-internal-ci Maintainer-approved dispatch to internal CI label Aug 22, 2026
@github-actions github-actions Bot removed the run-internal-ci Maintainer-approved dispatch to internal CI label Aug 22, 2026
@yifeif-nv
yifeif-nv force-pushed the agent/remove-runtime-hf-python branch from eac14e7 to a1c6031 Compare August 22, 2026 20:31
@yifeif-nv

yifeif-nv commented Aug 22, 2026

Copy link
Copy Markdown
Collaborator Author

/run-ci

@yifeif-nv
yifeif-nv force-pushed the agent/remove-runtime-hf-python branch from a1c6031 to d95d8e4 Compare August 22, 2026 20:45
@yifeif-nv

yifeif-nv commented Aug 22, 2026

Copy link
Copy Markdown
Collaborator Author

/run-ci

@yifeif-nv yifeif-nv added the run-internal-ci Maintainer-approved dispatch to internal CI label Aug 22, 2026
@github-actions github-actions Bot removed the run-internal-ci Maintainer-approved dispatch to internal CI label Aug 22, 2026
@yifeif-nv
yifeif-nv force-pushed the agent/remove-runtime-hf-python branch from d95d8e4 to 0d8bb88 Compare August 22, 2026 21:07
@yifeif-nv

yifeif-nv commented Aug 22, 2026

Copy link
Copy Markdown
Collaborator Author

/run-ci

@yifeif-nv yifeif-nv added the run-internal-ci Maintainer-approved dispatch to internal CI label Aug 22, 2026
@github-actions github-actions Bot removed the run-internal-ci Maintainer-approved dispatch to internal CI label Aug 22, 2026
@yifeif-nv yifeif-nv added the run-internal-ci Maintainer-approved dispatch to internal CI label Aug 22, 2026
@github-actions github-actions Bot removed the run-internal-ci Maintainer-approved dispatch to internal CI label Aug 22, 2026
@yifeif-nv
yifeif-nv force-pushed the agent/remove-runtime-hf-python branch from 0d8bb88 to 83cccf7 Compare August 22, 2026 21:42
@yifeif-nv

yifeif-nv commented Aug 22, 2026

Copy link
Copy Markdown
Collaborator Author

/run-ci

@yifeif-nv yifeif-nv added the run-internal-ci Maintainer-approved dispatch to internal CI label Aug 22, 2026
@github-actions github-actions Bot removed the run-internal-ci Maintainer-approved dispatch to internal CI label Aug 22, 2026
@yifeif-nv
yifeif-nv force-pushed the agent/remove-runtime-hf-python branch from 83cccf7 to 15e65f8 Compare August 23, 2026 21:20
@yifeif-nv

yifeif-nv commented Aug 23, 2026

Copy link
Copy Markdown
Collaborator Author

/run-ci

@yifeif-nv
yifeif-nv force-pushed the agent/remove-runtime-hf-python branch from 15e65f8 to 606c5df Compare August 23, 2026 22:35
@yifeif-nv

yifeif-nv commented Aug 23, 2026

Copy link
Copy Markdown
Collaborator Author

/run-ci

@yifeif-nv
yifeif-nv force-pushed the agent/remove-runtime-hf-python branch from 606c5df to 6300af0 Compare August 23, 2026 23:07
@yifeif-nv

yifeif-nv commented Aug 23, 2026

Copy link
Copy Markdown
Collaborator Author

/run-ci

@yifeif-nv

yifeif-nv commented Aug 23, 2026

Copy link
Copy Markdown
Collaborator Author

run-internal-ci

@yifeif-nv yifeif-nv added the run-internal-ci Maintainer-approved dispatch to internal CI label Aug 23, 2026
@github-actions github-actions Bot removed the run-internal-ci Maintainer-approved dispatch to internal CI label Aug 23, 2026
@yifeif-nv
yifeif-nv force-pushed the agent/remove-runtime-hf-python branch from 6300af0 to be0b107 Compare August 24, 2026 23:06
@yifeif-nv yifeif-nv added the run-internal-ci Maintainer-approved dispatch to internal CI label Aug 24, 2026
@github-actions github-actions Bot removed the run-internal-ci Maintainer-approved dispatch to internal CI label Aug 24, 2026
@yifeif-nv
yifeif-nv force-pushed the agent/remove-runtime-hf-python branch from be0b107 to bfc9df4 Compare August 26, 2026 22:57
@coderabbitai

coderabbitai Bot commented Aug 26, 2026 •
edited
Loading

Copy link
Copy Markdown

Important

Review skipped

Too many files!

This PR contains 324 files, which is 24 over the limit of 300.

To get a review, reduce the PR to 300 files or fewer by splitting it into smaller PRs or changing its base branch.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: ee6f6247-a5f9-40de-a21e-c6d7cea6a1dd

📥 Commits

Reviewing files that changed from the base of the PR and between 7e63254 and 08b0f92.

📒 Files selected for processing (324)
  • CMakeLists.txt
  • benchmarks/performance/release.yaml
  • cmake/register_model_plugin.cpp.in
  • examples/trtmc_benchmark_worker.cpp
  • examples/trtmc_dataset_benchmark.cpp
  • include/trtmc/pipeline.h
  • include/trtmc/runtime/pipeline_factory.h
  • include/trtmc/runtime/pipeline_plugin.h
  • include/trtmc/runtime/pipeline_plugin_loader.h
  • plugins/trtmc-agent-skills/skills/debug-trt-mismatch/SKILL.md
  • plugins/trtmc-agent-skills/skills/profile-model/SKILL.md
  • python/tensorrt_model_connect/benchmark/task_adapters.py
  • python/tensorrt_model_connect/config.py
  • python/tensorrt_model_connect/engine_builder.py
  • python/tensorrt_model_connect/families/bark/diff_audio.py
  • python/tensorrt_model_connect/families/flux/bench_flux2_perf.py
  • python/tensorrt_model_connect/families/personaplex/diff_personaplex.py
  • python/tensorrt_model_connect/families/personaplex/plugin.py
  • python/tensorrt_model_connect/families/qwen/bench_flashinfer_e2e.py
  • python/tensorrt_model_connect/families/qwen3_omni/audio_runtime.py
  • python/tensorrt_model_connect/families/qwen3_omni/plugin.py
  • python/tensorrt_model_connect/families/qwen_image/qwen_image_bundle_config.py
  • python/tensorrt_model_connect/families/wan_t2v/build_wan14b.py
  • python/tensorrt_model_connect/pipeline.py
  • python/tensorrt_model_connect/runtime_config/schemas/platform.py
  • scripts/eval_mmlu.py
  • scripts/hf_tokenizer.py
  • scripts/perf_evolve_prompt.py
  • scripts/validate_family.sh
  • src/cabi/api/trtmc_c.cpp
  • src/cli/args.cpp
  • src/cli/args.h
  • src/cli/main.cpp
  • src/runtime/config/schemas/platform.cpp
  • src/runtime/models/albert/plugin_helpers.cpp
  • src/runtime/models/bark/plugin_helpers.cpp
  • src/runtime/models/bart/plugin_helpers.cpp
  • src/runtime/models/bert/plugin_helpers.cpp
  • src/runtime/models/bloom/plugin_helpers.cpp
  • src/runtime/models/canary/plugin_helpers.cpp
  • src/runtime/models/chronos_bolt/plugin_helpers.cpp
  • src/runtime/models/codegen/plugin_helpers.cpp
  • src/runtime/models/convbert/plugin_helpers.cpp
  • src/runtime/models/deberta/plugin_helpers.cpp
  • src/runtime/models/deepseek_ocr/plugin_helpers.cpp
  • src/runtime/models/deepseek_v2/plugin_helpers.cpp
  • src/runtime/models/distilbert/plugin_helpers.cpp
  • src/runtime/models/dpr/plugin_helpers.cpp
  • src/runtime/models/eagle_vlm/plugin_helpers.cpp
  • src/runtime/models/electra/plugin_helpers.cpp
  • src/runtime/models/elf_flow/plugin_helpers.cpp
  • src/runtime/models/falcon/plugin_helpers.cpp
  • src/runtime/models/flux/plugin_helpers.cpp
  • src/runtime/models/fnet/plugin_helpers.cpp
  • src/runtime/models/gemma/plugin_helpers.cpp
  • src/runtime/models/glm/plugin_helpers.cpp
  • src/runtime/models/gpt2/plugin_helpers.cpp
  • src/runtime/models/gpt_neo/plugin_helpers.cpp
  • src/runtime/models/gpt_neox/plugin_helpers.cpp
  • src/runtime/models/gpt_oss/plugin_helpers.cpp
  • src/runtime/models/granite/plugin_helpers.cpp
  • src/runtime/models/internlm/plugin_helpers.cpp
  • src/runtime/models/internvl/plugin_helpers.cpp
  • src/runtime/models/lance/plugin_helpers.cpp
  • src/runtime/models/llama/plugin_helpers.cpp
  • src/runtime/models/locateanything/plugin_helpers.cpp
  • src/runtime/models/ltx_video/plugin_helpers.cpp
  • src/runtime/models/m2m_100/plugin_helpers.cpp
  • src/runtime/models/magpie/plugin_helpers.cpp
  • src/runtime/models/mamba/plugin_helpers.cpp
  • src/runtime/models/marian/plugin_helpers.cpp
  • src/runtime/models/mistral/plugin_helpers.cpp
  • src/runtime/models/mixtral/plugin_helpers.cpp
  • src/runtime/models/modernbert/plugin_helpers.cpp
  • src/runtime/models/mpnet/plugin_helpers.cpp
  • src/runtime/models/nemotron/plugin_helpers.cpp
  • src/runtime/models/nemotron_h/plugin_helpers.cpp
  • src/runtime/models/nemotron_labs_diffusion/plugin_helpers.cpp
  • src/runtime/models/nemotron_speech_streaming/plugin_helpers.cpp
  • src/runtime/models/olmo/plugin_helpers.cpp
  • src/runtime/models/olmo2/plugin_helpers.cpp
  • src/runtime/models/opt/plugin_helpers.cpp
  • src/runtime/models/patchtsmixer/plugin_helpers.cpp
  • src/runtime/models/patchtst/plugin_helpers.cpp
  • src/runtime/models/personaplex/MODEL.toml
  • src/runtime/models/personaplex/audio_helpers.cpp
  • src/runtime/models/personaplex/audio_helpers.h
  • src/runtime/models/personaplex/pipeline.cpp
  • src/runtime/models/personaplex/pipeline.h
  • src/runtime/models/personaplex/plugin.cpp
  • src/runtime/models/personaplex/plugin_helpers.cpp
  • src/runtime/models/personaplex/speech_config.h
  • src/runtime/models/personaplex/speech_runtime_plan.h
  • src/runtime/models/personaplex/subprocess_runner.h
  • src/runtime/models/phi/plugin_helpers.cpp
  • src/runtime/models/phi4_multimodal/plugin_helpers.cpp
  • src/runtime/models/phi_moe/plugin_helpers.cpp
  • src/runtime/models/pixart/plugin_helpers.cpp
  • src/runtime/models/qwen/plugin_helpers.cpp
  • src/runtime/models/qwen3_5/plugin_helpers.cpp
  • src/runtime/models/qwen3_omni/MODEL.toml
  • src/runtime/models/qwen3_omni/omni_config.h
  • src/runtime/models/qwen3_omni/omni_thinker_plan.h
  • src/runtime/models/qwen3_omni/pipeline.cpp
  • src/runtime/models/qwen3_omni/pipeline.h
  • src/runtime/models/qwen3_omni/plugin.cpp
  • src/runtime/models/qwen3_omni/plugin_helpers.cpp
  • src/runtime/models/qwen3_omni/talker_runtime.cpp
  • src/runtime/models/qwen3_omni/talker_runtime.h
  • src/runtime/models/qwen_image/plugin_helpers.cpp
  • src/runtime/models/qwen_image/qwen_image_types.cpp
  • src/runtime/models/qwen_image/qwen_image_types.h
  • src/runtime/models/qwen_moe/plugin_helpers.cpp
  • src/runtime/models/qwen_vl/plugin_helpers.cpp
  • src/runtime/models/roberta/plugin_helpers.cpp
  • src/runtime/models/rwkv/plugin_helpers.cpp
  • src/runtime/models/sam/plugin_helpers.cpp
  • src/runtime/models/segformer/plugin_helpers.cpp
  • src/runtime/models/stablelm/plugin_helpers.cpp
  • src/runtime/models/starcoder2/plugin_helpers.cpp
  • src/runtime/models/t5/plugin_helpers.cpp
  • src/runtime/models/timesfm/plugin_helpers.cpp
  • src/runtime/models/timm_vit/plugin_helpers.cpp
  • src/runtime/models/wan/plugin_helpers.cpp
  • src/runtime/models/whisper/plugin_helpers.cpp
  • src/runtime/models/xglm/plugin_helpers.cpp
  • src/runtime/models/xlnet/plugin_helpers.cpp
  • src/runtime/models/z_image/plugin_helpers.cpp
  • src/runtime/registry/pipeline_factory.cpp
  • src/runtime/registry/pipeline_plugin_loader.cpp
  • src/runtime/registry/pipeline_plugin_loader_internal.h
  • src/utils/data_dir.cpp
  • src/utils/data_dir.h
  • tests/builder/test_config.py
  • tests/builder/test_config_coverage.py
  • tests/builder/test_generation_config_eos.py
  • tests/builder/test_pipeline.py
  • tests/cpp/fakes/fake_model_plugin_abi.cpp
  • tests/cpp/models/personaplex/test_personaplex_speech_output_plan.cpp
  • tests/cpp/models/personaplex/test_personaplex_speech_pipeline.cpp
  • tests/cpp/models/personaplex/test_personaplex_speech_runtime_plan.cpp
  • tests/cpp/models/qwen3_omni/test_qwen3_omni_audio_plan.cpp
  • tests/cpp/models/qwen3_omni/test_qwen3_omni_pipeline.cpp
  • tests/cpp/models/qwen3_omni/test_qwen3_omni_talker_runtime.cpp
  • tests/cpp/test_c_abi_entry.cpp
  • tests/cpp/test_cli_args.cpp
  • tests/cpp/test_data_dir.cpp
  • tests/cpp/test_model_plugin_abi.cpp
  • tests/cpp/test_optimized_runtime_host.cpp
  • tests/e2e/models/albert/e2e_plugins/runners/encoder_only.py
  • tests/e2e/models/bark/e2e_plugins/contract.py
  • tests/e2e/models/bark/e2e_plugins/runners/audio_speech.py
  • tests/e2e/models/bark/perf_validation.json
  • tests/e2e/models/bart/e2e_plugins/runners/text_generation.py
  • tests/e2e/models/bert/e2e_plugins/runners/encoder_only.py
  • tests/e2e/models/bert/perf_validation.json
  • tests/e2e/models/bloom/e2e_plugins/runners/text_generation.py
  • tests/e2e/models/canary/e2e_plugins/runners/audio_speech.py
  • tests/e2e/models/codegen/e2e_plugins/runners/text_generation.py
  • tests/e2e/models/convbert/e2e_plugins/runners/encoder_only.py
  • tests/e2e/models/deberta/e2e_plugins/runners/encoder_only.py
  • tests/e2e/models/deepseek_ocr/e2e_plugins/repro.py
  • tests/e2e/models/deepseek_ocr/e2e_plugins/runners/vision_language.py
  • tests/e2e/models/deepseek_v2/e2e_plugins/runners/text_generation.py
  • tests/e2e/models/distilbert/e2e_plugins/runners/encoder_only.py
  • tests/e2e/models/dpr/e2e_plugins/runners/encoder_only.py
  • tests/e2e/models/eagle_vlm/e2e_plugins/runners/embedding.py
  • tests/e2e/models/eagle_vlm/e2e_plugins/runners/reranking.py
  • tests/e2e/models/electra/e2e_plugins/runners/encoder_only.py
  • tests/e2e/models/elf_flow/e2e_plugins/repro.py
  • tests/e2e/models/falcon/e2e_plugins/runners/text_generation.py
  • tests/e2e/models/flux/e2e_plugins/runners/diffusion.py
  • tests/e2e/models/flux/perf_validation.json
  • tests/e2e/models/fnet/e2e_plugins/runners/encoder_only.py
  • tests/e2e/models/gemma/e2e_plugins/runners/text_generation.py
  • tests/e2e/models/glm/e2e_plugins/runners/text_generation.py
  • tests/e2e/models/gpt2/e2e_plugins/runners/text_generation.py
  • tests/e2e/models/gpt_neo/e2e_plugins/runners/text_generation.py
  • tests/e2e/models/gpt_neox/e2e_plugins/runners/text_generation.py
  • tests/e2e/models/gpt_oss/e2e_plugins/runners/text_generation.py
  • tests/e2e/models/granite/e2e_plugins/runners/text_generation.py
  • tests/e2e/models/internlm/e2e_plugins/runners/text_generation.py
  • tests/e2e/models/internvl/e2e_plugins/repro.py
  • tests/e2e/models/internvl/e2e_plugins/runners/vision_language.py
  • tests/e2e/models/lance/e2e_plugins/repro.py
  • tests/e2e/models/lance/e2e_plugins/runners/vision_language.py
  • tests/e2e/models/lfm2/e2e_plugins/runners/text_generation.py
  • tests/e2e/models/llama/e2e_plugins/runners/text_generation.py
  • tests/e2e/models/locateanything/e2e_plugins/repro.py
  • tests/e2e/models/locateanything/e2e_plugins/runners/object_detection.py
  • tests/e2e/models/locateanything/e2e_plugins/runners/vision_language.py
  • tests/e2e/models/ltx_video/e2e_plugins/repro.py
  • tests/e2e/models/ltx_video/e2e_plugins/runners/diffusion.py
  • tests/e2e/models/m2m_100/e2e_plugins/runners/text_generation.py
  • tests/e2e/models/magpie_tts/e2e_plugins/contract.py
  • tests/e2e/models/magpie_tts/e2e_plugins/runners/audio_speech.py
  • tests/e2e/models/magpie_tts/perf_validation.json
  • tests/e2e/models/mamba/e2e_plugins/runners/text_generation.py
  • tests/e2e/models/mamba/perf_validation.json
  • tests/e2e/models/marian/e2e_plugins/runners/text_generation.py
  • tests/e2e/models/mistral/e2e_plugins/runners/text_generation.py
  • tests/e2e/models/mixtral/e2e_plugins/runners/text_generation.py
  • tests/e2e/models/mixtral/perf_validation.json
  • tests/e2e/models/modernbert/e2e_plugins/runners/encoder_only.py
  • tests/e2e/models/mpnet/e2e_plugins/runners/encoder_only.py
  • tests/e2e/models/nemotron/e2e_plugins/runners/text_generation.py
  • tests/e2e/models/nemotron_h/e2e_plugins/runners/text_generation.py
  • tests/e2e/models/nemotron_labs_diffusion/e2e_plugins/runners/text_generation.py
  • tests/e2e/models/nemotron_speech_streaming/e2e_plugins/runners/audio_speech.py
  • tests/e2e/models/olmo/e2e_plugins/runners/text_generation.py
  • tests/e2e/models/olmo2/e2e_plugins/runners/text_generation.py
  • tests/e2e/models/opt/e2e_plugins/runners/text_generation.py
  • tests/e2e/models/personaplex/manifests/personaplex-7b-l0-tp4.json
  • tests/e2e/models/personaplex/manifests/personaplex-7b-l0.json
  • tests/e2e/models/personaplex/manifests/personaplex-7b.json
  • tests/e2e/models/personaplex/test_personaplex_registry_contract.py
  • tests/e2e/models/phi/e2e_plugins/runners/text_generation.py
  • tests/e2e/models/phi/perf_validation.json
  • tests/e2e/models/phi4_multimodal/e2e_plugins/repro.py
  • tests/e2e/models/phi4_multimodal/e2e_plugins/runners/vision_language.py
  • tests/e2e/models/phi_moe/e2e_plugins/runners/text_generation.py
  • tests/e2e/models/pixart/e2e_plugins/runners/diffusion.py
  • tests/e2e/models/qwen/e2e_plugins/runners/text_generation.py
  • tests/e2e/models/qwen/perf_validation.json
  • tests/e2e/models/qwen/test_perf_parity.py
  • tests/e2e/models/qwen3_5/e2e_plugins/runners/text_generation.py
  • tests/e2e/models/qwen3_omni/MODEL.toml
  • tests/e2e/models/qwen3_omni/e2e_plugins/reference.py
  • tests/e2e/models/qwen3_omni/e2e_plugins/references/invariant_only.py
  • tests/e2e/models/qwen3_omni/e2e_plugins/runners/omni.py
  • tests/e2e/models/qwen3_omni/manifests/qwen3-omni-30b-a3b-instruct.json
  • tests/e2e/models/qwen3_omni/test_hidden_state_flow.py
  • tests/e2e/models/qwen3_omni/test_qwen3_omni_audio_runtime.py
  • tests/e2e/models/qwen3_omni/test_qwen3_omni_comparator.py
  • tests/e2e/models/qwen3_omni/test_qwen3_omni_reference.py
  • tests/e2e/models/qwen3_omni/test_qwen3_omni_registry_contract.py
  • tests/e2e/models/qwen3_omni/thresholds/qwen3-omni-30b-a3b-instruct.json
  • tests/e2e/models/qwen3_omni/validation/qwen3-omni-native-thinker.json
  • tests/e2e/models/qwen_image/e2e_plugins/repro.py
  • tests/e2e/models/qwen_image/e2e_plugins/runners/diffusion.py
  • tests/e2e/models/qwen_moe/e2e_plugins/runners/text_generation.py
  • tests/e2e/models/qwen_vl/e2e_plugins/repro.py
  • tests/e2e/models/qwen_vl/e2e_plugins/runners/vision_language.py
  • tests/e2e/models/qwen_vl/perf_validation.json
  • tests/e2e/models/roberta/e2e_plugins/runners/encoder_only.py
  • tests/e2e/models/rwkv/e2e_plugins/runners/text_generation.py
  • tests/e2e/models/rwkv/perf_validation.json
  • tests/e2e/models/sam/e2e_plugins/repro.py
  • tests/e2e/models/sam/e2e_plugins/runners/segmentation.py
  • tests/e2e/models/sam/perf_validation.json
  • tests/e2e/models/sam3/e2e_plugins/repro.py
  • tests/e2e/models/sam3/e2e_plugins/runners/segmentation.py
  • tests/e2e/models/segformer/e2e_plugins/repro.py
  • tests/e2e/models/segformer/e2e_plugins/runners/segmentation.py
  • tests/e2e/models/segformer/perf_validation.json
  • tests/e2e/models/stablelm/e2e_plugins/runners/text_generation.py
  • tests/e2e/models/starcoder2/e2e_plugins/runners/text_generation.py
  • tests/e2e/models/t5/e2e_plugins/runners/text_generation.py
  • tests/e2e/models/t5/perf_validation.json
  • tests/e2e/models/timm_vit/e2e_plugins/repro.py
  • tests/e2e/models/timm_vit/e2e_plugins/runners/image_classification.py
  • tests/e2e/models/wan_t2v/e2e_plugins/runners/diffusion.py
  • tests/e2e/models/wan_t2v/perf_validation.json
  • tests/e2e/models/whisper/e2e_plugins/runners/audio_speech.py
  • tests/e2e/models/whisper/perf_validation.json
  • tests/e2e/models/xglm/e2e_plugins/runners/text_generation.py
  • tests/e2e/models/xlnet/e2e_plugins/runners/encoder_only.py
  • tests/e2e/models/z_image/e2e_plugins/runners/diffusion.py
  • tests/e2e/models/z_image/perf_validation.json
  • tests/e2e/test_cache_correctness.py
  • tests/e2e/test_diffusion_pipeline.py
  • tests/e2e/test_error_handling.py
  • tests/e2e/test_full_pipeline.py
  • tests/e2e/test_inference.py
  • tests/e2e/test_runner_parity.py
  • tests/e2e/test_vl_pipeline.py
  • tests/e2e_harness/contracts.py
  • tests/e2e_harness/orchestrator.py
  • tests/runtime_strategy_matrix.yaml
  • tests/tools/test_auto_perf_tune.py
  • tests/tools/test_e2e_python_profiles.py
  • tests/tools/test_e2e_repro_commands.py
  • tests/tools/test_github_actions_ci.py
  • tests/tools/test_model_plugin_encapsulation_static.py
  • tests/tools/test_model_proof_inner.py
  • tests/tools/test_model_proof_runner.py
  • tests/tools/test_native_runtime_python_free.py
  • tests/tools/test_perf_compare.py
  • tests/tools/test_perf_matrix.py
  • tests/tools/test_performance_catalog.py
  • tests/tools/test_plugin_runtime_context.py
  • tests/tools/test_prepare_model_plugin_validation_datasets.py
  • tests/tools/test_selected_wheel_runtime.py
  • tests/tools/test_trtmc_bench.py
  • tests/tools/test_trtmc_validate.py
  • tests/validation/README.md
  • tests/validation/model_workloads.yaml
  • tests/validation/workloads.yaml
  • tools/auto_perf_tune.py
  • tools/ci/README.md
  • tools/ci/model_proof.py
  • tools/ci/model_proof_inner.py
  • tools/ci/quality.py
  • tools/diff.py
  • tools/diff_framework/protocol.py
  • tools/diff_vl.py
  • tools/legal_header_exceptions.toml
  • tools/perf_compare.py
  • tools/prepare_model_plugin_validation_datasets.py
  • tools/test_impact_fallback_allowlist.txt
  • tools/test_runner_parity.py
  • tools/trtmc_profile.py
  • tools/validation/engine.py
  • website/docs/api/cli-reference.md
  • website/docs/api/cpp-api.md
  • website/docs/api/python-builder.md
  • website/docs/architecture/runtime-lifecycle.md
  • website/docs/context/config-registry-status.md
  • website/docs/extend/model-validation.md
  • website/docs/features/config-and-backends.md
  • website/docs/getting-started/build-and-run.md
  • website/docs/reference/profiling.md
  • website/docs/reference/testing.md
  • website/docs/tutorials/advanced/multi-device-inference.md

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Comment @coderabbitai help to get the list of available commands.

@yifeif-nv yifeif-nv added the run-internal-ci Maintainer-approved dispatch to internal CI label Aug 26, 2026
@github-actions github-actions Bot removed the run-internal-ci Maintainer-approved dispatch to internal CI label Aug 26, 2026
@yifeif-nv yifeif-nv added the run-internal-ci Maintainer-approved dispatch to internal CI label Aug 26, 2026
@github-actions github-actions Bot removed the run-internal-ci Maintainer-approved dispatch to internal CI label Aug 26, 2026
@yifeif-nv
yifeif-nv force-pushed the agent/remove-runtime-hf-python branch from bfc9df4 to f18f4f7 Compare August 27, 2026 00:17
@yifeif-nv yifeif-nv added the run-internal-ci Maintainer-approved dispatch to internal CI label Aug 27, 2026
@github-actions github-actions Bot removed the run-internal-ci Maintainer-approved dispatch to internal CI label Aug 27, 2026
@yifeif-nv
yifeif-nv force-pushed the agent/remove-runtime-hf-python branch from f18f4f7 to 6bca3bc Compare August 27, 2026 01:18
@yifeif-nv yifeif-nv added run-internal-ci Maintainer-approved dispatch to internal CI and removed run-internal-ci Maintainer-approved dispatch to internal CI labels Aug 27, 2026
@github-actions github-actions Bot removed the run-internal-ci Maintainer-approved dispatch to internal CI label Aug 27, 2026
@yifeif-nv
refactor(runtime): remove Python interpreter bridges
08b0f92 
Remove interpreter paths from the native API, CLI, wrappers, benchmark, and E2E runtime surfaces. Delete the PersonaPlex tokenizer subprocess and Qwen3-Omni Talker worker, keep unsupported audio generation fail-closed, and add a repository guard against future interpreter or process-spawn bridges.

Treat pre-1.0 runtime and model-plugin interfaces as current-epoch only: delete retired fields and overloads instead of carrying compatibility slots or symbols, require all artifacts to be rebuilt together, and reject stale model DSOs through the exact-match ABI v2 gate. Restore PersonaPlex output-plan coverage under its own model-owned test.

Canonicalize nested decoder fields at the bundle config root so the root-only JSON parser never depends on recursive key scans. Keep structured encoders model-owned while preserving valid zero/list token IDs, generation-config EOS precedence, and model-owned overrides.

BREAKING CHANGE: Remove --hf-python and the hf_python public fields and arguments. Core, wheel, bundles, and model DSOs must be rebuilt and upgraded together. Qwen3-Omni audio generation remains unavailable until a native Talker is implemented.

Signed-off-by: yifeif-nv <277870278+yifeif-nv@users.noreply.github.com>
@yifeif-nv
yifeif-nv force-pushed the agent/remove-runtime-hf-python branch from 6bca3bc to 08b0f92 Compare August 27, 2026 22:31
@yifeif-nv yifeif-nv added the run-internal-ci Maintainer-approved dispatch to internal CI label Aug 27, 2026
@github-actions github-actions Bot removed the run-internal-ci Maintainer-approved dispatch to internal CI label Aug 27, 2026
@yifeif-nv yifeif-nv added the run-internal-ci Maintainer-approved dispatch to internal CI label Aug 28, 2026
@github-actions github-actions Bot removed the run-internal-ci Maintainer-approved dispatch to internal CI label Aug 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@yifeif-nv