Search results

Dockerfile is using internal base images

Status: Open.

#694 In triton-inference-server/tensorrtllm_backend;

· lkm2835 opened on Feb 6, 2025

Mllama ignores input image when deployed in triton

bugSomething isn't workingSomething isn't working

Status: Open.

#692 In triton-inference-server/tensorrtllm_backend;

· mutkach opened on Feb 5, 2025

Build triton_trt_llm image in debug mode

Status: Open.

#691 In triton-inference-server/tensorrtllm_backend;

· nzarif opened on Feb 5, 2025

Performance of triton+trtllm on llava-onevision compared to vllm and sglang

Status: Open.

#689 In triton-inference-server/tensorrtllm_backend;

· alexemme opened on Feb 3, 2025

Unable to build from source for tag v0.16.0.

bugSomething isn't workingSomething isn't working

Status: Open.

#686 In triton-inference-server/tensorrtllm_backend;

· jingzhaoou opened on Jan 30, 2025

DeepSeek-R1-Distill-Qwen-32B FP16 model does not work with Triton server + tensorrtllm_backend (but it works with just TensorRT-LLM)

bugSomething isn't workingSomething isn't working

Status: Open.

#685 In triton-inference-server/tensorrtllm_backend;

· kelkarn opened on Jan 30, 2025

What is the purpose of shm-region-prefix-name and what is the prefix0_ files used for?

Status: Open.

#684 In triton-inference-server/tensorrtllm_backend;

· sugam-nexusflow opened on Jan 28, 2025

"error": "Unable to parse 'inputs': attempt to access non-existing object member 'inputs'"

Status: Open.

#683 In triton-inference-server/tensorrtllm_backend;

· adityarap opened on Jan 28, 2025

Beam search diversity lost with in-flight batching

bugSomething isn't workingSomething isn't working

Status: Open.

#682 In triton-inference-server/tensorrtllm_backend;

· Grace-YingHuang opened on Jan 24, 2025

How to deploy a quantized FP8 multimodel llm ?

Status: Open.

#681 In triton-inference-server/tensorrtllm_backend;

· zhishao opened on Jan 22, 2025

obj_size <= remaining_buffer_size

Status: Open.

#680 In triton-inference-server/tensorrtllm_backend;

· qzq-123 opened on Jan 20, 2025

Assertion failed: sizeof(T) <= remaining_buffer_size

bugSomething isn't workingSomething isn't working

Status: Open.

#679 In triton-inference-server/tensorrtllm_backend;

· gawain000000 opened on Jan 14, 2025