Skip to content

Feature: Add ornith-ai/Ornith-1.5-9B checkpoint support #1050

Description

@yifeif-nv

This issue was drafted by AI at a user's request.

Request type

New model or checkpoint support

Problem and use case

Ornith-1.5-9B is a public,
MIT-licensed, dense 9B reasoning model derived from Qwen3.5. Its documented use
cases include coding, agentic workflows, long-context chat, reasoning output,
and tool calling. The checkpoint is small enough for single-GPU deployment, but
TensorRT-Model-Connect does not currently list or validate it as a supported
checkpoint.

TensorRT-Model-Connect already has a qwen3_5 family and native runtime path
for Qwen/Qwen3.5-9B. Users should be able to build and run the Ornith
checkpoint through the same public Model Connect workflow when its architecture
is compatible, with any checkpoint-specific differences handled and validated
explicitly.

Proposed outcome

Add supported, tested ornith-ai/Ornith-1.5-9B checkpoint coverage. The initial
scope can focus on the model card's primary text-generation/chat path:

  • Build a bundle from the pinned public checkpoint through the normal trtmc
    builder interface.
  • Run the bundle through the native runtime path on a single supported GPU.
  • Reuse the existing Qwen3.5 family where the checkpoint is architecture- and
    weight-layout-compatible; record and handle any real differences rather than
    adding checkpoint-name-only behavior.
  • Preserve the checkpoint's chat-template behavior, including thinking output
    and XML tool-call serialization, within the capabilities of the public
    runtime interface.
  • Add focused upstream parity validation and list the checkpoint in the public
    supported-model documentation.

The checkpoint also contains a Qwen3.5 vision configuration and is tagged for
image-text-to-text use. If the first implementation supports only text input,
document that boundary clearly and track image/video input support separately.

Acceptance criteria

  • The exact checkpoint and an immutable revision are registered in the
    model manifest/test inventory.
  • An FP16 engine builds from the BF16 source checkpoint through the public
    Model Connect builder without manual checkpoint rewriting.
  • The resulting bundle runs through the normal native runtime on an NVIDIA
    H100 80GB for a representative chat prompt.
  • A normal reasoning/chat response and a tool-calling prompt are validated
    against the pinned upstream Transformers behavior using meaningful,
    behavior-aligned criteria.
  • The implementation records whether the existing qwen3_5 family is
    reused unchanged or which checkpoint-specific config, weights, tokenizer,
    or chat-template differences require support.
  • The supported-model documentation states the supported precision, task,
    GPU/runtime path, context-length boundary, and multimodal boundary.
  • No test threshold, passing criterion, or validator is weakened to make
    the checkpoint pass.

Model and target details

  • Model ID: ornith-ai/Ornith-1.5-9B
  • Revision: 489cb97981b8654bcfcf30ce1f94ed1b62e07b53
  • License: MIT
  • Architecture: Qwen3_5ForConditionalGeneration (model_type: qwen3_5)
  • Size/source dtype: approximately 9.65B parameters, BF16
  • Initial task: reasoning-oriented text generation/chat and tool-call
    serialization
  • Initial engine precision: FP16
  • Initial target: NVIDIA H100 80GB, single GPU
  • Expected path: public trtmc builder -> bundle -> native runtime
  • Declared context window: 262,144 tokens; initial validation may use a smaller,
    explicitly documented bounded context

Alternatives considered

  • Serve the checkpoint with vLLM, SGLang, or llama.cpp. These are documented
    upstream alternatives, but they do not produce or validate a Model Connect
    bundle for the native runtime.
  • Use the already-supported Qwen/Qwen3.5-9B checkpoint. That exercises a
    related architecture but does not provide Ornith's trained weights and
    behavior.
  • Point the existing qwen3_5 family at Ornith without adding a manifest and
    parity coverage. This would leave compatibility and regressions unsupported.

Additional context

Submission checks

  • I searched open and closed issues and found no duplicate request.
  • I removed secrets, private/internal evidence, personal paths, and
    restricted artifacts.

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions