Search results

[Question] Help with Client-Side Batching for Large Requests in Triton

Status: Open.

#818 In triton-inference-server/client;

· harsh-boloai opened on Jan 17, 2025

Read back data written in cuda shared memory in python

Status: Open.

#816 In triton-inference-server/client;

· MaximeDebarbat opened on Dec 16, 2024

Performance Discrepancy Between Triton Client SDK and perf_analyzer

Status: Open.

#815 In triton-inference-server/client;

· wensimin opened on Dec 10, 2024

InferenceServerHttpClient::Create() failed with error: std::bad_alloc

Status: Open.

#806 In triton-inference-server/client;

· megadl opened on Nov 13, 2024

Incorrect grpcio version Python client

Status: Open.

#804 In triton-inference-server/client;

· tigersoldier opened on Nov 8, 2024

[GRPC_CLIENT] LoadModel with serialized model raise an error: "Poll failed for model directory" on native windows triton.

Status: Open.

#803 In triton-inference-server/client;

· FilipLeja opened on Nov 6, 2024

The dependency information of the Python package needs to be updated

Status: Open.

#796 In triton-inference-server/client;

· penguin-wwy opened on Oct 16, 2024

[Question] numpy 2 support

Status: Open.

#792 In triton-inference-server/client;

· m3at opened on Oct 10, 2024

Potential memory leak

Status: Open.

#791 In triton-inference-server/client;

· prasannagn16 opened on Oct 8, 2024

Tritonclient CUDA shared memory set_shared_memory_region_from_dlpack fails with non-set stream for Torch tensor.

Status: Open.

#789 In triton-inference-server/client;

· piotrm-nvidia opened on Oct 5, 2024

genai-perf exception offline-mode

Status: Open.

#785 In triton-inference-server/client;

· sochiseung opened on Sep 24, 2024

原始是byte[]的数据，没法传。会报triton.client.InferenceException: [request id: <id_unknown>] inference input data-type is 'INT8', model expects 'BYTES' for 'xxxxx'

Status: Open.

#784 In triton-inference-server/client;

· freeleeq opened on Sep 24, 2024