Skip to content

Commit 53770ec

Browse files
authored
Add Neuron vLLM HF Containers (#5578)
1 parent 3a715d3 commit 53770ec

File tree

1 file changed

+10
-0
lines changed

1 file changed

+10
-0
lines changed

available_images.md

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -328,6 +328,16 @@ Please refer to the following pages to view all available versions and tags for
328328

329329
To get the latest one, you can check the Hugging Face [documentation](https://huggingface.co/docs/optimum-neuron/en/containers#available-optimum-neuron-containers).
330330

331+
HuggingFace Neuron vLLM Containers
332+
===============================
333+
334+
To get the latest one, you can check the Hugging Face [documentation](https://huggingface.co/docs/optimum-neuron/en/containers#available-optimum-neuron-containers).
335+
336+
| Framework | Neuron SDK Version | Job Type | Supported EC2 Instance Type | Python Version Options | Example URL |
337+
|--------------------------------------------------------------------|--------------------|-----------|-----------------------------|------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------|
338+
| vLLM 0.11.0 with NeuronX Inference and HuggingFace Optimum | Neuron 2.26.0 | inference | inf2/trn2/trn1 | 3.10 (py310) | 763104351884.dkr.ecr.us-west-2.amazonaws.com/huggingface-vllm-inference-neuronx:0.11.0-optimum0.4.2-neuronx-py310-sdk2.26.0-ubuntu22.04 |
339+
| vLLM 0.10.2 with NeuronX Inference and HuggingFace Optimum | Neuron 2.26.0 | inference | inf2/trn2/trn1 | 3.10 (py310) | 763104351884.dkr.ecr.us-west-2.amazonaws.com/huggingface-vllm-inference-neuronx:0.10.2-neuronx-py310-sdk2.26.0-ubuntu22.04 |
340+
331341
HuggingFace Neuron Inference Containers
332342
===============================
333343

0 commit comments

Comments
 (0)