@@ -258,8 +258,14 @@ AutoGluon Inference Containers
258258HuggingFace Training Containers
259259===============================
260260
261+ Please refer to the following page to view all available versions and tags for GPU containers:
262+ * [ GPU Release Page] ( https://github.com/aws/deep-learning-containers/releases?q=huggingface-pytorch-training+AND+NOT+neuronx&expanded=true )
263+
264+ To get the latest one, you can check the Hugging Face [ documentation] ( https://huggingface.co/docs/sagemaker/en/dlcs/available#training ) .
265+
261266| Framework | Job Type | CPU/GPU | Python Version Options | Example URL |
262267| -----------------------------------------------| -----------| -----------| -----------------------| -----------------------------------------------------------------------------------------------------------------------------------|
268+ | PyTorch 2.5.1 with HuggingFace transformers | training | GPU | 3.11 (py311) | 763104351884.dkr.ecr.us-east-1.amazonaws.com/huggingface-pytorch-training:2.5.1-transformers4.49.0-gpu-py311-cu124-ubuntu22.04 |
263269| PyTorch 2.1.0 with HuggingFace transformers | training | GPU | 3.10 (py310) | 763104351884.dkr.ecr.us-east-1.amazonaws.com/huggingface-pytorch-training:2.1.0-transformers4.36.0-gpu-py310-cu121-ubuntu20.04 |
264270| PyTorch 2.0.0 with HuggingFace transformers | training | GPU | 3.10 (py310) | 763104351884.dkr.ecr.us-east-1.amazonaws.com/huggingface-pytorch-training:2.0.0-transformers4.28.1-gpu-py310-cu118-ubuntu20.04 |
265271| PyTorch 1.13.1 with HuggingFace transformers | training | GPU | 3.9 (py39) | 763104351884.dkr.ecr.us-east-1.amazonaws.com/huggingface-pytorch-training:1.13.1-transformers4.26.0-gpu-py39-cu117-ubuntu20.04 |
@@ -269,8 +275,15 @@ HuggingFace Training Containers
269275HuggingFace Inference Containers
270276===============================
271277
278+ Please refer to the following page to view all available versions and tags for GPU and CPU containers:
279+ * [ GPU and CPU Release Page] ( https://github.com/aws/deep-learning-containers/releases?q=huggingface-pytorch-inference+AND+NOT+tgi+AND+NOT+neuronx&expanded=true )
280+
281+ To get the latest one, you can check the Hugging Face [ documentation] ( https://huggingface.co/docs/sagemaker/en/dlcs/available#pytorch-inference-dlc ) .
282+
272283| Framework | Job Type | CPU/GPU | Python Version Options | Example URL |
273284| --------------------------------------------------| ------------| -----------| -----------------------| -----------------------------------------------------------------------------------------------------------------------------------|
285+ | PyTorch 2.6.0 with HuggingFace transformers | inference | CPU | 3.12 (py312) | 763104351884.dkr.ecr.us-east-1.amazonaws.com/huggingface-pytorch-inference:2.6.0-transformers4.49.0-cpu-py312-ubuntu22.04 |
286+ | PyTorch 2.6.0 with HuggingFace transformers | inference | GPU | 3.12 (py312) | 763104351884.dkr.ecr.us-east-1.amazonaws.com/huggingface-pytorch-inference:2.6.0-transformers4.49.0-gpu-py312-cu124-ubuntu22.04 |
274287| PyTorch 2.1.0 with HuggingFace transformers | inference | CPU | 3.10 (py310) | 763104351884.dkr.ecr.us-east-1.amazonaws.com/huggingface-pytorch-inference:2.1.0-transformers4.37.0-cpu-py310-ubuntu22.04 |
275288| PyTorch 2.1.0 with HuggingFace transformers | inference | GPU | 3.10 (py310) | 763104351884.dkr.ecr.us-east-1.amazonaws.com/huggingface-pytorch-inference:2.1.0-transformers4.37.0-gpu-py310-cu118-ubuntu20.04 |
276289| PyTorch 2.0.0 with HuggingFace transformers | inference | CPU | 3.10 (py310) | 763104351884.dkr.ecr.us-east-1.amazonaws.com/huggingface-pytorch-inference:2.0.0-transformers4.28.1-cpu-py310-ubuntu20.04 |
@@ -283,25 +296,51 @@ HuggingFace Inference Containers
283296HuggingFace Text Generation Inference (TGI) Containers
284297===============================
285298
286- Please refer to the following pages to view all available versions and tags for GPU and NeuronX containers:
299+ Please refer to the following pages to view all available versions and tags for GPU containers:
287300* [ GPU Release Page] ( https://github.com/aws/deep-learning-containers/releases?q=tgi+AND+gpu&expanded=true )
301+
302+ To get the latest one, you can check the Hugging Face [ documentation] ( https://huggingface.co/docs/sagemaker/en/dlcs/available#llm-tgi ) .
303+
304+ HuggingFace Text Embeddings Inference (TEI) Containers
305+ ===============================
306+
307+ To get the latest one, you can check the Hugging Face [ documentation] ( https://huggingface.co/docs/sagemaker/dlcs/available#text-embedding-inference ) .
308+
309+ HuggingFace Neuron Text Generation Inference (TGI) Containers
310+ ===============================
311+
312+ Please refer to the following pages to view all available versions and tags for NeuronX containers:
288313* [ NeuronX Release Page] ( https://github.com/aws/deep-learning-containers/releases?q=tgi+AND+neuronx&expanded=true )
289314
315+ To get the latest one, you can check the Hugging Face [ documentation] ( https://huggingface.co/docs/optimum-neuron/en/containers#available-optimum-neuron-containers ) .
316+
290317HuggingFace Neuron Inference Containers
291318===============================
292319
320+ Please refer to the following pages to view all available versions and tags for NeuronX containers:
321+ * [ NeuronX Release Page] ( https://github.com/aws/deep-learning-containers/releases?q=huggingface-pytorch-inference-neuronx+AND+NOT+tgi&expanded=true )
322+
323+ To get the latest one, you can check the Hugging Face [ documentation] ( https://huggingface.co/docs/optimum-neuron/en/containers#available-optimum-neuron-containers ) .
324+
293325| Framework | Neuron SDK Version | Job Type | Supported EC2 Instance Type | Python Version Options | Example URL |
294326| ------------------------------------------------------------------| -------------------| -----------| ----------------------------| -----------------------| ----------------------------------------------------------------------------------------------------------------------------------------------|
295327| PyTorch 1.10.2 with Neuron Inference and HuggingFace transformers | Neuron 1.19.1 | inference | inf1 | 3.7 (py37) | 763104351884.dkr.ecr.us-east-1.amazonaws.com/huggingface-pytorch-inference-neuron:1.10.2-transformers4.20.1-neuron-py37-sdk1.19.1-ubuntu18.04 |
296328| PyTorch 1.13.1 with NeuronX Inference and HuggingFace transformers | Neuron 2.15.0 | inference | inf2/trn1 | 3.10 (py310) | 763104351884.dkr.ecr.us-east-1.amazonaws.com/huggingface-pytorch-inference-neuronx:1.13.1-transformers4.34.1-neuronx-py310-sdk2.15.0-ubuntu20.04 |
297329| PyTorch 2.1.2 with NeuronX Inference and HuggingFace transformers | Neuron 2.18.0 | inference | inf2/trn1 | 3.10 (py310) | 763104351884.dkr.ecr.us-east-1.amazonaws.com/huggingface-pytorch-inference-neuronx:2.1.2-transformers4.36.2-neuronx-py310-sdk2.18.0-ubuntu20.04 |
330+ | PyTorch 2.1.2 with NeuronX Inference and HuggingFace transformers | Neuron 2.20.0 | inference | inf2/trn1 | 3.10 (py310) | 763104351884.dkr.ecr.us-east-1.amazonaws.com/huggingface-pytorch-inference-neuronx:2.1.2-transformers4.43.2-neuronx-py310-sdk2.20.0-ubuntu20.04 |
298331
299332HuggingFace Neuron Training Containers
300333===============================
301334
335+ Please refer to the following pages to view all available versions and tags for NeuronX containers:
336+ * [ NeuronX Release Page] ( https://github.com/aws/deep-learning-containers/releases?q=huggingface-pytorch-training-neuronx+AND+NOT+tgi&expanded=true )
337+
338+ To get the latest one, you can check the Hugging Face [ documentation] ( https://huggingface.co/docs/optimum-neuron/en/containers#available-optimum-neuron-containers ) .
339+
302340| Framework | Neuron SDK Version | Job Type | Supported EC2 Instance Type | Python Version Options | Example URL |
303341| ------------------------------------------------------------------| -------------------| -----------| ----------------------------| -----------------------| ----------------------------------------------------------------------------------------------------------------------------------------------|
304342| PyTorch 1.13.1 with NeuronX Training and HuggingFace transformers | Neuron 2.18.0 | training | trn1 | 3.10 (py310) | 763104351884.dkr.ecr.us-east-1.amazonaws.com/huggingface-pytorch-training-neuronx:1.13.1-transformers4.36.2-neuronx-py310-sdk2.18.0-ubuntu20.04 |
343+ | PyTorch 2.1.2 with NeuronX Training and HuggingFace transformers | Neuron 2.20.0 | training | trn1 | 3.10 (py310) | 763104351884.dkr.ecr.us-east-1.amazonaws.com/huggingface-pytorch-training-neuronx:2.1.2-transformers4.48.1-neuronx-py310-sdk2.20.0-ubuntu20.04 |
305344
306345StabilityAI Inference Containers
307346===============================
0 commit comments