refactor: isolate model families end to end

[yifeif-nv]

Background

The previous repository split each model across central builder, runtime, registration, validation, and documentation systems. Adding a family therefore required shared model changes and increased cross-family coupling. This PR atomically replaces that layout with the architecture documented in website/docs/architecture/ai-native-horizontal-scaling.md.

Exit Criteria

• Every supported family owns its build, runtime DSO, and direct tests under families/<family>/.
• Shared Python and native code remain closed, minimal contract and loading surfaces.
• Family implementations do not import, include, or link sibling families.
• BYOK, benchmark, Cosmos dual-Spark, and VoiceChat remain working one-way applications over public ModelConnect APIs.
• No legacy compatibility, migration, dual-path, implicit worker/catalog discovery, or content-hash validation remains.
• Required CI passes on the exact PR head and GitHub reports the PR mergeable and clean.

Implementation

• Moved all 85 model families into independent vertical slices with a plain model.py::build(request, writer) structural contract. Builder inheritance is forbidden.
• Made concrete family pipelines implement abstract Task APIs and depend only on thin bundle, factory, task, and engine contracts. Each family builds an independent libtrtmc_model_<family>.so.
• Replaced the bundle/build/runtime paths in one incompatible change; existing bundles must be rebuilt.
• Kept checkpoint reading, graph construction, weights, tokenization, scheduling, bindings, preprocessing, and tests family-owned. Repeated implementation remains family-local rather than becoming shared model infrastructure.
• Restored BYOK as an explicit TVM-FFI DSO/function contract with no graph guessing or source/ABI hashes. Core, runtime, backend, and ordinary families do not link TVM-FFI.
• Restored trtmc-bench, the performance matrix, the native worker, and dataset benchmark over the public build/load/Task APIs. Output contracts are closed and fail on unknown or mismatched results.
• Ported the Cosmos3 dual-Spark and Nemotron VoiceChat full-duplex examples to the new APIs without moving their orchestration policy into core.
• Added family-owned time-series normalization and explicit frequency semantics so raw requests work at the abstract Task boundary.
• Made package validation consume the preinstalled offline toolchain, and added the MPI/NCCL runtime needed by real tensor-parallel family tests.
• Corrected affected decoder families to allocate rank-local KV cache widths while preserving Qwen-MoE replication and DeepSeek-V2 MLA geometry.

Change categories

• Model or runtime behavior
• Public API
• ABI
• Bundle or artifact format
• Dependencies
• Documentation only
• CI or developer tooling

Validation

Commands and Results

Exact head: 9ba50b00c0d2f8f0aa89c5a4aa14922a7f7b214a

• python3 -m tools.ci pipeline premerge-unit in a network-disabled, GPU-free, read-only-source container: 203 Python tests passed, 271 explicitly unselected model E2Es skipped, 775 native targets built, and 47/47 CTests passed.
• python3 -m tools.ci pipeline package with no network: built, installed, imported, RTLD-loaded, and validated one wheel containing 85 builders, 85 family DSOs, BYOK, benchmark executables, manifests, and referenced benchmark assets.
• python3 -m build --no-isolation --sdist followed by an offline wheel build from the unpacked sdist: passed; the rebuilt wheel passed the same archive, install, CLI, import, and RTLD validation.
• GPT-2 and DeepSeek-V2 family-owned parallel-KV CTests: 2/2 passed, including GPT-2 TP4 width 192 and DeepSeek-V2 MLA TP4 width 768.
• mpirun --tag-output -np 1 true as the container root and direct libnccl.so.2 loading through the configured runtime path: passed.
• The GPT-2 distilgpt2 and gpt2-125m checkpoint-to-bundle-to-native Task API cases passed. The corrected TP4 case built all four rank engines, passed the previous cache-shape failure, and entered NCCL/CUDA execution; rank 3 then hit an uncorrectable NVLink hardware error, so TP4 inference is not claimed as passed locally.
• Fresh TRTMC_ENABLE_BYOK=OFF configure/build: passed; ordinary core/runtime/backend/CLI remain buildable without the BYOK DSO.
• Installed CMake consumer linking only trtmc::trtmc_byok_tvm_ffi: compiled with <trtmc/byok.h>.
• Installed-wheel trtmc-bench media dry-run resolved its image from the packaged catalog outside the source checkout.
• npm run test:model-support and the production Docusaurus build passed; exact-head Community CPU Docs also passed.
• Exact-head Community CPU Source quality, Ownership and impact, Unit, aggregate Required, PR Metadata, and DCO all passed.

No merge has been attempted.

Hardware, Environment, and Revisions

• Repository head: 9ba50b00c0d2f8f0aa89c5a4aa14922a7f7b214a
• Base: 10270b7f9e3d0610a9f7f04c59511d190ee83ce0
• Local native validation: NVIDIA GB300, AArch64 Ubuntu 24.04 container, CUDA 13.3, TensorRT 11.1.0.106, sm_100.

Not Run / Remaining Gaps

• Real two-DGX-Spark/RoCE execution and physical ALSA device execution were not available locally. Their source contracts, dry-run orchestration, queue behavior, compilation, and one-way dependencies passed.
• The full 85-family checkpoint/TP/CP matrix was not rerun locally because all checkpoints and healthy topologies were not available. The protected matrix remains required.
• GPT-2 TP4 local inference was blocked by a rank-3 uncorrectable NVLink hardware error after the corrected code reached NCCL/CUDA execution; a healthy protected runner must provide the terminal result.
• The automated internal premerge gate has not run on this exact head, so the PR remains blocked and is not ready to merge.

Notes For Future Readers

• This is intentionally one atomic breaking replacement. There is no compatibility or migration layer.
• Suggested review order: architecture document; shared contracts/loaders; two structurally different family slices; BYOK and benchmark application boundaries; closed-set tests; CI/package changes.
• Do not extract duplicated family code into a shared model helper without a concrete cross-family contract requirement.

Risk level

• Low
• Medium
• High

Risk rationale: this deliberately replaces every family build/runtime/test path, the bundle/API/ABI contract, packaging, documentation, and CI in one incompatible change.

[coderabbitai]

Caution

CodeRabbit couldn't post its review summary.

Error details

Validation Failed: {"resource":"IssueComment","code":"unprocessable","field":"data","message":"Body is too long (maximum is 65536 characters)"} - https://docs.github.com/rest/issues/comments#create-an-issue-comment

[coderabbitai]

Caution

CodeRabbit couldn't post its review summary.

Error details

Validation Failed: {"resource":"IssueComment","code":"unprocessable","field":"data","message":"Body is too long (maximum is 65536 characters)"} - https://docs.github.com/rest/issues/comments#create-an-issue-comment

[coderabbitai]

Caution

CodeRabbit couldn't post its review summary.

Error details

Validation Failed: {"resource":"IssueComment","code":"unprocessable","field":"data","message":"Body is too long (maximum is 65536 characters)"} - https://docs.github.com/rest/issues/comments#create-an-issue-comment

[coderabbitai]

Caution

CodeRabbit couldn't post its review summary.

Error details

Validation Failed: {"resource":"IssueComment","code":"unprocessable","field":"data","message":"Body is too long (maximum is 65536 characters)"} - https://docs.github.com/rest/issues/comments#create-an-issue-comment

[coderabbitai]

Caution

CodeRabbit couldn't post its review summary.

Error details

Validation Failed: {"resource":"IssueComment","code":"unprocessable","field":"data","message":"Body is too long (maximum is 65536 characters)"} - https://docs.github.com/rest/issues/comments#create-an-issue-comment

[coderabbitai]

Caution

CodeRabbit couldn't post its review summary.

Error details

Validation Failed: {"resource":"IssueComment","code":"unprocessable","field":"data","message":"Body is too long (maximum is 65536 characters)"} - https://docs.github.com/rest/issues/comments#create-an-issue-comment

[coderabbitai]

Caution

CodeRabbit couldn't post its review summary.

Error details

Validation Failed: {"resource":"IssueComment","code":"unprocessable","field":"data","message":"Body is too long (maximum is 65536 characters)"} - https://docs.github.com/rest/issues/comments#create-an-issue-comment

[coderabbitai]

Caution

CodeRabbit couldn't post its review summary.

Error details

Validation Failed: {"resource":"IssueComment","code":"unprocessable","field":"data","message":"Body is too long (maximum is 65536 characters)"} - https://docs.github.com/rest/issues/comments#create-an-issue-comment

[github-actions]

This is an automated Internal CI result; no review from an individual maintainer is requested.

TRTMC Protected CI failure
==========================

This log contains only approved structured failure fields.

Status: FAILED
Repository: NVIDIA/TensorRT-Model-Connect
Pull request: #1093
Head commit: 89bf793f71f2b3edc3e6eda0270155804236e4ec
Tested revision: 26c50c4e6790b061ac6783b40bd6bd46e735082d (merge)
Run attempt: 1
Generated at: 2026-08-30T07:26:09Z
Disclosure policy: 2026-08-27

Failure summary
---------------
Failure 1
  Class: source_quality_error
  Reason: unknown
  Cause: No structured failure detail was safe to disclose.
  Stage: source-quality
  Model: other-model
  Backend: other-backend
  GPU: protected-gpu
  Test: withheld
  Evidence: details withheld

Failure 2
  Class: infrastructure_error
  Reason: unknown
  Cause: No structured failure detail was safe to disclose.
  Stage: runtime-control
  Model: other-model
  Backend: other-backend
  GPU: protected-gpu
  Test: withheld
  Evidence: details withheld

Report ID: trtmc-pr1093-89bf793-attempt1

Open the public Source Actions run from the automated status link above.

[coderabbitai]

Caution

CodeRabbit couldn't post its review summary.

Error details

Validation Failed: {"resource":"IssueComment","code":"unprocessable","field":"data","message":"Body is too long (maximum is 65536 characters)"} - https://docs.github.com/rest/issues/comments#create-an-issue-comment

[coderabbitai]

Caution

CodeRabbit couldn't post its review summary.

Error details

Validation Failed: {"resource":"IssueComment","code":"unprocessable","field":"data","message":"Body is too long (maximum is 65536 characters)"} - https://docs.github.com/rest/issues/comments#create-an-issue-comment

[coderabbitai]

Caution

CodeRabbit couldn't post its review summary.

Error details

Validation Failed: {"resource":"IssueComment","code":"unprocessable","field":"data","message":"Body is too long (maximum is 65536 characters)"} - https://docs.github.com/rest/issues/comments#create-an-issue-comment