fix(internvl): flatten decoder bundle config

[yifeif-nv]

Background

The protected premerge run for source PR #1053 exposed an existing InternVL3-2B full-generation regression: native inference emitted !!!!!!!!!! while the Hugging Face reference emitted White. The vision encoder, checkpoint loading, C++ tests, and Python model tests still passed, which localized the failure to decoder runtime configuration rather than model weights or vision preprocessing.

RCCA

Impact

• InternVL composite checkpoints built successfully but native text generation used invalid decoder runtime geometry.
• The failure was silent until semantic E2E comparison because the engine and vision path remained operational.

Root Cause

• PR [Refactor] Replace the shared JSON scanner with nlohmann/json (#975) #1012 correctly replaced the shared handwritten JSON scanner with strict nlohmann::json parsing.
• The previous scanner searched the full JSON text and therefore implicitly found decoder keys inside InternVL's nested text_config object.
• The strict parser correctly limits ordinary key lookup to the current object, but parse_base_config() continued to query only the root object.
• InternVL therefore fell back to invalid defaults for vocabulary size, hidden size, layer count, attention heads, KV heads, and BOS token ID.

Why Existing Tests Missed It

• JSON helper tests covered root-level values, malformed input, and strict parsing behavior, but not a real composite Hugging Face decoder contract.
• InternVL model tests did not exercise parse_base_config() with its nested text_config shape.
• As a shared platform change, PR [Refactor] Replace the shared JSON scanner with nlohmann/json (#975) #1012's selective protected premerge used representative fallback models; none consumed the InternVL nested decoder contract.

Correction And Prevention

• Preserve the strict shared JSON parser unchanged.
• Make the InternVL family-owned bundle hook promote its already parsed decoder geometry and BOS token to top-level runtime fields.
• Add the missing assertion to the family-owned CPU test file. Community impact selection routes that test into Unit / C++ and Python with no broad fallback tier, so the regression is rejected before protected GPU model proof.

Exit Criteria

• InternVL decoder geometry and BOS resolve from text_config without changing shared JSON behavior.
• The source diff remains entirely family-owned.
• Community CPU directly selects the InternVL unit and fails if bundle flattening is removed or incomplete.
• InternVL3-2B native full generation matches the external reference.

Implementation

• Add InternVLPlugin.get_bundle_config_overrides() to emit the typed decoder geometry and BOS token that InternVL's family-owned ModelConfig already resolves from text_config.
• Extend test_internvl_family_plugin_weights.py with a CPU-only composite-config assertion covering 28 layers, hidden size 1536, vocabulary size 151674, GQA geometry, and nested BOS.

Change categories

• Model or runtime behavior
• Public API
• ABI
• Bundle or artifact format
• Dependencies
• Documentation only
• CI or developer tooling

Validation

Commands and Results

• python -m pytest tests/e2e/models/internvl/test_internvl_family_plugin_weights.py -q -p no:cacheprovider: passed, 7 tests.
• python3 tools/test_impact.py --base github/main --json: selected the InternVL family unit in builder_tests; fallback_tiers was empty.
• ruff check --config ruff.toml python/tensorrt_model_connect/families/internvl/plugin.py tests/e2e/models/internvl/test_internvl_family_plugin_weights.py: passed.
• python -m pytest tests/tools/test_model_plugin_encapsulation_static.py -q -p no:cacheprovider: passed, 158 tests.
• python tools/legal_headers.py --check: passed with zero findings.
• python3 -m tools.ci model-proof --model internvl --revision 09cdba012ac2cef8e69e0808a5266dbc55bd103b --suite premerge: passed locally; native and reference full generation both produced White.

Hardware, Environment, and Revisions

• Source head: 09cdba012ac2cef8e69e0808a5266dbc55bd103b, based directly on github/main@8fb1b89d447d12067930d19d1a7d1d1340ffc77f.
• Model: OpenGVLab/InternVL3-2B-hf snapshot cb57a075cb75a2e6d1b668b128d48bb00ae321d2.
• Model proof: NVIDIA GB300, driver 580.105.08, CUDA 13.3.33, TensorRT 11.2.1.2, FP16, Python 3.12, PyTorch 2.12.0+cu130, Transformers 5.2.0.
• CPU validation: Linux aarch64 CI container, Python 3.12, TensorRT 11.2.1.2 bindings, no GPU device supplied to the unit test.

Not Run / Remaining Gaps

• TensorRT 11.1 and InternVL tensor-parallel variants were not run; exact model parity was demonstrated on the affected TRT 11.2 single-GPU testcase.
• Full all-model GPU E2E was not run. Impact analysis is family-direct and the missing contract is now covered in Community CPU.
• Remote GitHub checks must rerun on the rewritten head; validation listed above is local exact-head evidence.

Notes For Future Readers

• Review python/tensorrt_model_connect/families/internvl/plugin.py first, then its regression in tests/e2e/models/internvl/test_internvl_family_plugin_weights.py.
• This intentionally fixes the missing CPU contract instead of expanding selective premerge to every full GPU model proof.
• Shared registry and JSON helper behavior are intentionally unchanged; the composite-config policy belongs to InternVL.

Risk level

• Low
• Medium
• High

Risk rationale: the change is family-local and does not alter shared parsing, public API, ABI, or bundle format. It does change InternVL bundle metadata, with direct Community CPU coverage and restored model parity.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 1051fa2b-eafb-40aa-bef8-472d61f31135

📥 Commits

Reviewing files that changed from the base of the PR and between 8fb1b89 and 09cdba0.

📒 Files selected for processing (2)

• python/tensorrt_model_connect/families/internvl/plugin.py
• tests/e2e/models/internvl/test_internvl_family_plugin_weights.py

Included review availability: Your plan provides up to 12 included reviews per hour; 11 remain after this review.

---

📝 Walkthrough

Summary by CodeRabbit

• New Features

  • Exposed key InternVL text decoder settings at the bundle level, including vocabulary size, model dimensions, attention configuration, head size, and beginning-of-sequence token ID.

• Tests

  • Added end-to-end coverage to verify that bundle settings are correctly derived from the model’s nested text configuration.

Walkthrough

The InternVL plugin now exposes nested text-decoder settings as bundle-level overrides. An end-to-end test verifies vocabulary, dimensions, attention geometry, head dimension, and BOS token values.

Changes

InternVL bundle configuration

Layer / File(s)	Summary
Flatten text-decoder configuration python/tensorrt_model_connect/families/internvl/plugin.py, tests/e2e/models/internvl/test_internvl_family_plugin_weights.py	get_bundle_config_overrides exposes nested text-decoder configuration values. The end-to-end test verifies the returned vocabulary, dimensions, layer and head counts, head dimension, and BOS token ID.

Estimated code review effort: 2 (Simple) | ~10 minutes

Merge Risk: ⚪ Minimal · up to 09cdb

This localized InternVL configuration fix has direct regression coverage and restored native/reference generation parity; no actionable merge-blocking risk remains beyond normal checks and review.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly and concisely identifies the main change: flattening the InternVL decoder bundle configuration.
Description check	✅ Passed	The description is complete and relevant. It covers the background, root cause, exit criteria, implementation, change category, validation results, environment, remaining gaps, future notes, and risk …
Docstring Coverage	✅ Passed	Docstring coverage is 80.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 5 functions across 2 files.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

Full details: Description check

Explanation

The description is complete and relevant. It covers the background, root cause, exit criteria, implementation, change category, validation results, environment, remaining gaps, future notes, and risk rationale. It also identifies source PR #1053, although it does not provide a direct issue or discussion link.

---

_{Comment @coderabbitai help to get the list of available commands.}