Skip to content

Feature request: hy_v3 (Tencent Hy3) architecture support - working proof-of-concept adapter available #88

Description

@konradzamojski

Hi — thanks for maintaining this plugin, it's the only realistic path to GGUF on vLLM since the in-tree deprecation. Wanted to flag a gap and offer something concrete rather than just a bare request.

What's missing

hy_v3 (Tencent Hy3, HYV3ForCausalLM) isn't in gguf.MODEL_ARCH_NAMES — confirmed by grepping the repo (zero hits for hy_v3/hyv3) and checking open PRs/issues (none reference it either, as of this writing). Loading any hy_v3 GGUF currently fails with Unknown gguf model_type: hy_v3 in the plugin's weight loader, even though HYV3ForCausalLM is natively implemented in vLLM core for safetensors/NVFP4 — the gap is specifically in this plugin's GGUF tensor-name mapping, not in vLLM's model support.

What we already got working

We hit this ourselves and built a working adapter rather than just wait: build_name_map_from_files() — dynamically constructs the tensor→module mapping straight from the GGUF file's own metadata at load time, instead of relying on a static entry in MODEL_ARCH_NAMES. Confirmed on our end:

  • CPU-side dry-run gate: 1298/1298 tensors resolved, zero unmapped, zero name collisions, against a hy3-1M-MTP-*.gguf (satgeze conversion, standard llama.cpp blk.N.* naming, includes the MTP/NextN layer).
  • GPU load: weights actually land on-device (66.14 GiB/worker at TP=2), server reaches health, generates coherent output on our test battery.
  • Had to separately work around an unrelated FlashInfer JIT issue on SM120/CUDA<12.9 (VLLM_USE_FLASHINFER_SAMPLER=0) to get past profile_run — not a plugin issue, just adjacent noise worth mentioning in case anyone else hits it testing this on Blackwell.

Honest caveat — this is a starting point, not a finished patch

Model loads and runs, but end-to-end it's not yet a case for using this over llama.cpp or native vLLM: cold TTFT at ~28k context came in at 51.6s (vs sub-second warm from prefix-cache hits, and vs single-digit seconds on other paths we tested), so on-the-fly GGUF dequant at this architecture's scale needs more than just the name-mapping fix to be genuinely competitive — likely dequant-kernel-level work beyond what our adapter touches. Flagging this so nobody reads "loads and runs" as "ready to serve."

Offer

Happy to share the adapter code (dynamic name-map + the small plugin registration shim) if it's useful as a starting point for someone picking this up, and to test any attempt on our hardware (2x RTX PRO 6000 Blackwell, TP=2) — we already have the measurement harness (TTFT/decode benchmarks, a quality battery) set up from getting this far, so turnaround on testing a patch would be quick.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions