Pinned Loading

1. NVIDIA/TensorRT-LLM NVIDIA/TensorRT-LLM Public

   TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficie…

   C++ 9.3k 1.1k

2. triton-inference-server/server triton-inference-server/server Public

   The Triton Inference Server provides an optimized cloud and edge inferencing solution.

   Python 8.7k 1.5k

3. triton-inference-server/python_backend triton-inference-server/python_backend Public

   Triton backend that enables pre-process, post-processing and other logic to be implemented in Python.

   C++ 586 155

4. triton-inference-server/model_analyzer triton-inference-server/model_analyzer Public

   Triton Model Analyzer is a CLI tool to help with better understanding of the compute and memory requirements of the Triton Inference Server models.

   Python 449 78

5. learning-to-quantize learning-to-quantize Public

   Code for "Adaptive Gradient Quantization for Data-Parallel SGD", published in NeurIPS 2020.

   Jupyter Notebook 29 5