v0.6.0

[nvluxiaoz]

TensorRT Edge-LLM 0.6.0 Release 2026-03-16

We are very excited to announce the 0.6.0 release of TensorRT Edge-LLM!

• TensorRT Edge-LLM is featured in GTC 2026! Link to our blog.
• Our developer roadmap for H1 2026 is listed in Developer Roadmap for TensorRT Edge-LLM (H1 2026) #32.
• Please see the demo "Run Physical AI Models Faster on Jetson" at GTC 2026. This demo uses TensorRT Edge-LLM as one of the backends to showcase ASR/LLM/TTS capability of NVIDIA Jetson AGX Thor.

Breaking Changes

• Due to the ViT Attention Plugin and split QKV changes, ONNX models exported by previous versions are not compatible with 0.6.0.

Key Features

Model Support

• Added Nemotron-Nano-9B-v2 support via mamba_ssm and causal_conv1d
• Added Qwen3-30B-A3B-GPTQ-Int4 support via INT4 MoE Plugin
• Added Qwen3-ASR and Qwen3-TTS end-to-end support

Performance Improvements

• Added cutedsl FMHA kernels to speed up prefill performance on Blackwell
• Used ViT Attention Plugin with fmha-v2 and cutedsl kernels to speed up multi image ViT performance
• Updated Attention Plugin to split q, k and v to save memory usage

Runtime Maturity

• Added LoRA support for Speculative Decoding
• Fixed several compiler warnings and document exceptions for functions
• Added coverage tests

NVIDIA Contributors

@fans-nv @nvluxiaoz @nvamberl @fkaster-nvidia @poweiw @charllll @mahu888 @xinrzhang-star @byte-deve @JCalafato @nvmbreughe

---

This discussion was created from the release v0.6.0.

[MevHe]

Missing definition of "kWindowAttentionMask" in cpp/common/bindingNames.h, which leads to failure of building target llm_build on jetson orin.

[nvluxiaoz]

Can you try to remove your build folder (if you have an older version's CMake Cache that might happen)? I do not see this variable kWindowAttentionMask anywhere in 0.6.0.

[MevHe]

It's indeed a complete fresh build, still failed, the variable was presented in 0.5.0 but missing in 0.6.0 in file bindingNames.h, build process just failed without this variable, i add the line inline constexpr char const* kWindowAttentionMask = "window_attention_mask"; to bindingNames.h, and rebuild successfully.

[nvluxiaoz]

I see. Thanks. We will resolve this ASAP. Seems that builder.cpp and builder.h should not appear.

[nvluxiaoz]

@MevHe The release has been taken down and reuploaded with the fix MR merged. Thanks for catching this!