Skip to content

Commit c67598b

Browse files
fix public ecr setup
1 parent 39b631f commit c67598b

File tree

2 files changed

+11
-5
lines changed

2 files changed

+11
-5
lines changed

vllm-samples/sagemaker/README.md

Lines changed: 10 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -30,8 +30,9 @@ aws iam attach-role-policy --role-name SageMakerExecutionRole --policy-arn arn:a
3030
### 1. Set Environment Variables
3131

3232
```bash
33-
# Check available images: https://gallery.ecr.aws/deep-learning-containers/vllm
34-
export CONTAINER_URI="public.ecr.aws/deep-learning-containers/0.11-gpu-py312"
33+
34+
# Note: Using a Public Gallery image to create an SM endpoint is currently not supported
35+
export CONTAINER_URI="763104351884.dkr.ecr.us-east-1.amazonaws.com/vllm:0.11.2-gpu-py312"
3536
export IAM_ROLE="SageMakerExecutionRole"
3637
export HF_TOKEN="your-huggingface-token"
3738
```
@@ -76,13 +77,18 @@ Recommended GPU instances:
7677
Test NixlConnector locally - [NixlConnector Documentation](https://docs.vllm.ai/en/latest/features/nixl_connector_usage.html#transport-configuration)
7778

7879
```bash
80+
# Login to aws ecr
81+
aws ecr get-login-password --region us-west-2 | docker login \
82+
--username AWS --password-stdin 763104351884.dkr.ecr.us-east-1.amazonaws.com
83+
7984
# Pull latest vLLM DLC for EC2
80-
docker pull public.ecr.aws/deep-learning-containers/vllm:0.11-gpu-py312
85+
# Note: Using a Public Gallery image to create an SM endpoint is currently not supported
86+
docker pull 763104351884.dkr.ecr.us-east-1.amazonaws.com/vllm:0.11.2-gpu-py312
8187

8288
# Run container with GPU access
8389
docker run -it --entrypoint=/bin/bash --gpus=all \
8490
-v $(pwd):/workspace \
85-
public.ecr.aws/deep-learning-containers/vllm:0.11-gpu-py312
91+
763104351884.dkr.ecr.us-east-1.amazonaws.com/vllm:0.11.2-gpu-py312
8692

8793
# Inside container, run the NixlConnector test
8894
export HF_TOKEN= "<TOKEN>"

vllm-samples/sagemaker/deploy_and_test_sm_endpoint.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -105,7 +105,7 @@ def main():
105105
help="DLC image URI",
106106
default=os.getenv(
107107
"CONTAINER_URI",
108-
"public.ecr.aws/deep-learning-containers/vllm:0.11.0-gpu-py312",
108+
"763104351884.dkr.ecr.us-east-1.amazonaws.com/vllm:0.11.2-gpu-py312",
109109
),
110110
)
111111
parser.add_argument(

0 commit comments

Comments
 (0)