Merge branch 'master' into vllm-arm64-release

Author: Jyothirmaikottu

.github/PULL_REQUEST_TEMPLATE.md

@@ -7,13 +7,13 @@
 
 ### Description
 
-### Tests run
-
-**NOTE: By default, docker builds are disabled. In order to build your container, please update dlc_developer_config.toml and specify the framework to build in "build_frameworks"**
-- [ ] I have run builds/tests on commit <INSERT COMMIT ID> for my changes.
+### Tests Run
+By default, docker image builds and tests are disabled. Two ways to run builds and tests:
+1. Using dlc_developer_config.toml
+2. Using this PR description (currently only supported for PyTorch, TensorFlow, vllm, and base images)
 
 <details>
-<summary>Confused on how to run tests? Try using the helper utility...</summary>
+<summary>How to use the helper utility for updating dlc_developer_config.toml</summary>
 
 Assuming your remote is called `origin` (you can find out more with `git remote -v`)...
 
@@ -28,50 +28,34 @@ Assuming your remote is called `origin` (you can find out more with `git remote
 - Restore TOML file when ready to merge
 
 `python src/prepare_dlc_dev_environment.py -rcp origin`
-</details>
-
-**NOTE: If you are creating a PR for a new framework version, please ensure success of the standard, rc, and efa sagemaker remote tests by updating the dlc_developer_config.toml file:**
-<details>
-<summary>Expand</summary>
 
+**NOTE: If you are creating a PR for a new framework version, please ensure success of the local, standard, rc, and efa sagemaker tests by updating the dlc_developer_config.toml file:**
 - [ ] `sagemaker_remote_tests = true`
 - [ ] `sagemaker_efa_tests = true`
 - [ ] `sagemaker_rc_tests = true`
-
-**Additionally, please run the sagemaker local tests in at least one revision:**
 - [ ] `sagemaker_local_tests = true`
-
 </details>
 
-### Formatting
-- [ ] I have run `black -l 100` on my code (formatting tool: https://black.readthedocs.io/en/stable/getting_started.html)
-
-### DLC image/dockerfile
-
-#### Builds to Execute
 <details>
-<summary>Expand</summary>
-
-Fill out the template and click the checkbox of the builds you'd like to execute
-
-*Note: Replace with <X.Y> with the major.minor framework version (i.e. 2.2) you would like to start.*
-
-- [ ] build_pytorch_training_<X.Y>_sm
-- [ ] build_pytorch_training_<X.Y>_ec2
-
-- [ ] build_pytorch_inference_<X.Y>_sm
-- [ ] build_pytorch_inference_<X.Y>_ec2
-- [ ] build_pytorch_inference_<X.Y>_graviton
+<summary>How to use PR description</summary>
+Use the code block below to uncomment commands and run the PR CodeBuild jobs. There are two commands available:
 
-- [ ] build_tensorflow_training_<X.Y>_sm
-- [ ] build_tensorflow_training_<X.Y>_ec2
+- `# /buildspec <buildspec_path>`
+  - e.g.: `# /buildspec pytorch/training/buildspec.yml`
+  - If this line is commented out, dlc_developer_config.toml will be used. 
+- `# /tests <test_list>`
+  - e.g.: `# /tests sanity security ec2`
+  - If this line is commented out, it will run the default set of tests (same as the defaults in dlc_developer_config.toml): `sanity, security, ec2, ecs, eks, sagemaker, sagemaker-local`.
 
-- [ ] build_tensorflow_inference_<X.Y>_sm
-- [ ] build_tensorflow_inference_<X.Y>_ec2
-- [ ] build_tensorflow_inference_<X.Y>_graviton
 </details>
 
-### Additional context
+```
+# /buildspec <buildspec_path>
+# /tests <test_list> 
+```
+
+### Formatting
+- [ ] I have run `black -l 100` on my code (formatting tool: https://black.readthedocs.io/en/stable/getting_started.html)
 
 ### PR Checklist 
 <details>
@@ -84,14 +68,6 @@ Fill out the template and click the checkbox of the builds you'd like to execute
 - [ ] (If applicable) I've documented below the tests I've run on the DLC image
 - [ ] (If applicable) I've reviewed the licenses of updated and new binaries and their dependencies to make sure all licenses are on the Apache Software Foundation Third Party License Policy Category A or Category B license list.  See [https://www.apache.org/legal/resolved.html](https://www.apache.org/legal/resolved.html).
 - [ ] (If applicable) I've scanned the updated and new binaries to make sure they do not have vulnerabilities associated with them.
-
-#### NEURON/GRAVITON Testing Checklist
-* When creating a PR:
-- [ ] I've modified `dlc_developer_config.toml` in my PR branch by setting `neuron_mode = true` or `graviton_mode = true`
-
-#### Benchmark Testing Checklist
-* When creating a PR:
-- [ ] I've modified `dlc_developer_config.toml` in my PR branch by setting `ec2_benchmark_tests = true` or `sagemaker_benchmark_tests = true`
 </details>
 
 ### Pytest Marker Checklist

base/buildspec-cu128-ubuntu24.yml

@@ -0,0 +1,54 @@
+account_id: &ACCOUNT_ID <set-$ACCOUNT_ID-in-environment>
+prod_account_id: &PROD_ACCOUNT_ID 763104351884
+region: &REGION <set-$REGION-in-environment>
+framework: &FRAMEWORK base
+version: &VERSION 12.8.1
+short_version: &SHORT_VERSION "12.8"
+arch_type: &ARCH_TYPE x86_64
+autopatch_build: "False"
+
+repository_info:
+  base_repository: &BASE_REPOSITORY
+    image_type: &IMAGE_TYPE gpu
+    root: .
+    repository_name: &REPOSITORY_NAME !join [ pr, "-", *FRAMEWORK ]
+    repository: &REPOSITORY !join [ *ACCOUNT_ID, .dkr.ecr., *REGION, .amazonaws.com/, *REPOSITORY_NAME ]
+    release_repository_name: &RELEASE_REPOSITORY_NAME !join [ *FRAMEWORK ]
+    release_repository: &RELEASE_REPOSITORY !join [ *PROD_ACCOUNT_ID, .dkr.ecr., *REGION, .amazonaws.com/, *RELEASE_REPOSITORY_NAME ]
+
+context:
+  base_context: &BASE_CONTEXT
+    deep_learning_container:
+      source: src/deep_learning_container.py
+      target: deep_learning_container.py
+    install_python:
+      source: scripts/install_python.sh
+      target: install_python.sh
+    install_cuda:
+      source: scripts/install_cuda.sh
+      target: install_cuda.sh
+    install_efa:
+      source: scripts/install_efa.sh
+      target: install_efa.sh
+
+images:
+  base_x86_64_gpu_cuda128:
+    <<: *BASE_REPOSITORY
+    context:
+      <<: *BASE_CONTEXT
+    image_size_baseline: 11000
+    device_type: &DEVICE_TYPE gpu
+    cuda_version: &CUDA_VERSION cu128
+    python_version: &DOCKER_PYTHON_VERSION py3
+    tag_python_version: &TAG_PYTHON_VERSION py312
+    os_version: &OS_VERSION ubuntu24.04
+    tag: !join [ *VERSION, "-", *DEVICE_TYPE, "-", *TAG_PYTHON_VERSION, "-", *CUDA_VERSION, "-", *OS_VERSION, "-ec2" ]
+    latest_release_tag: !join [ *VERSION, "-", *DEVICE_TYPE, "-", *TAG_PYTHON_VERSION, "-", *CUDA_VERSION, "-", *OS_VERSION, "-ec2" ]
+    docker_file: !join [ *FRAMEWORK, /, *ARCH_TYPE, /, *DEVICE_TYPE, /, *CUDA_VERSION, /, *OS_VERSION, /Dockerfile ]
+    target: final
+    build: true
+    enable_common_stage_build: false
+    test_configs:
+      test_platforms:
+        - sanity
+        - security

base/buildspec-cu129-ubuntu22.yml

@@ -0,0 +1,54 @@
+account_id: &ACCOUNT_ID <set-$ACCOUNT_ID-in-environment>
+prod_account_id: &PROD_ACCOUNT_ID 763104351884
+region: &REGION <set-$REGION-in-environment>
+framework: &FRAMEWORK base
+version: &VERSION 12.9.1
+short_version: &SHORT_VERSION "12.9"
+arch_type: &ARCH_TYPE x86_64
+autopatch_build: "False"
+
+repository_info:
+  base_repository: &BASE_REPOSITORY
+    image_type: &IMAGE_TYPE gpu
+    root: .
+    repository_name: &REPOSITORY_NAME !join [ pr, "-", *FRAMEWORK ]
+    repository: &REPOSITORY !join [ *ACCOUNT_ID, .dkr.ecr., *REGION, .amazonaws.com/, *REPOSITORY_NAME ]
+    release_repository_name: &RELEASE_REPOSITORY_NAME !join [ *FRAMEWORK ]
+    release_repository: &RELEASE_REPOSITORY !join [ *PROD_ACCOUNT_ID, .dkr.ecr., *REGION, .amazonaws.com/, *RELEASE_REPOSITORY_NAME ]
+
+context:
+  base_context: &BASE_CONTEXT
+    deep_learning_container:
+      source: src/deep_learning_container.py
+      target: deep_learning_container.py
+    install_python:
+      source: scripts/install_python.sh
+      target: install_python.sh
+    install_cuda:
+      source: scripts/install_cuda.sh
+      target: install_cuda.sh
+    install_efa:
+      source: scripts/install_efa.sh
+      target: install_efa.sh
+
+images:
+  base_x86_64_gpu_cuda129_ubuntu22:
+    <<: *BASE_REPOSITORY
+    context:
+      <<: *BASE_CONTEXT
+    image_size_baseline: 11000
+    device_type: &DEVICE_TYPE gpu
+    cuda_version: &CUDA_VERSION cu129
+    python_version: &DOCKER_PYTHON_VERSION py3
+    tag_python_version: &TAG_PYTHON_VERSION py312
+    os_version: &OS_VERSION ubuntu22.04
+    tag: !join [ *VERSION, "-", *DEVICE_TYPE, "-", *TAG_PYTHON_VERSION, "-", *CUDA_VERSION, "-", *OS_VERSION, "-ec2" ]
+    latest_release_tag: !join [ *VERSION, "-", *DEVICE_TYPE, "-", *TAG_PYTHON_VERSION, "-", *CUDA_VERSION, "-", *OS_VERSION, "-ec2" ]
+    docker_file: !join [ *FRAMEWORK, /, *ARCH_TYPE, /, *DEVICE_TYPE, /, *CUDA_VERSION, /, *OS_VERSION, /Dockerfile ]
+    target: final
+    build: true
+    enable_common_stage_build: false
+    test_configs:
+      test_platforms:
+        - sanity
+        - security

base/buildspec.yml

@@ -1,54 +1 @@
-account_id: &ACCOUNT_ID <set-$ACCOUNT_ID-in-environment>
-prod_account_id: &PROD_ACCOUNT_ID 763104351884
-region: &REGION <set-$REGION-in-environment>
-framework: &FRAMEWORK base
-version: &VERSION 12.8.1
-short_version: &SHORT_VERSION "12.8"
-arch_type: &ARCH_TYPE x86_64
-autopatch_build: "False"
-
-repository_info:
-  base_repository: &BASE_REPOSITORY
-    image_type: &IMAGE_TYPE gpu
-    root: .
-    repository_name: &REPOSITORY_NAME !join [ pr, "-", *FRAMEWORK ]
-    repository: &REPOSITORY !join [ *ACCOUNT_ID, .dkr.ecr., *REGION, .amazonaws.com/, *REPOSITORY_NAME ]
-    release_repository_name: &RELEASE_REPOSITORY_NAME !join [ *FRAMEWORK ]
-    release_repository: &RELEASE_REPOSITORY !join [ *PROD_ACCOUNT_ID, .dkr.ecr., *REGION, .amazonaws.com/, *RELEASE_REPOSITORY_NAME ]
-
-context:
-  base_context: &BASE_CONTEXT
-    deep_learning_container:
-      source: src/deep_learning_container.py
-      target: deep_learning_container.py
-    install_python:
-      source: scripts/install_python.sh
-      target: install_python.sh
-    install_cuda:
-      source: scripts/install_cuda.sh
-      target: install_cuda.sh
-    install_efa:
-      source: scripts/install_efa.sh
-      target: install_efa.sh
-
-images:
-  base_x86_64_gpu_cuda128:
-    <<: *BASE_REPOSITORY
-    context:
-      <<: *BASE_CONTEXT
-    image_size_baseline: 11000
-    device_type: &DEVICE_TYPE gpu
-    cuda_version: &CUDA_VERSION cu128
-    python_version: &DOCKER_PYTHON_VERSION py3
-    tag_python_version: &TAG_PYTHON_VERSION py312
-    os_version: &OS_VERSION ubuntu24.04
-    tag: !join [ *VERSION, "-", *DEVICE_TYPE, "-", *TAG_PYTHON_VERSION, "-", *CUDA_VERSION, "-", *OS_VERSION, "-ec2" ]
-    latest_release_tag: !join [ *VERSION, "-", *DEVICE_TYPE, "-", *TAG_PYTHON_VERSION, "-", *CUDA_VERSION, "-", *OS_VERSION, "-ec2" ]
-    docker_file: !join [ *FRAMEWORK, /, *ARCH_TYPE, /, *DEVICE_TYPE, /, *CUDA_VERSION, /Dockerfile ]
-    target: final
-    build: true
-    enable_common_stage_build: false
-    test_configs:
-      test_platforms:
-        - sanity
-        - security
+buildspec_pointer: buildspec-cu129-ubuntu22.yml

base/x86_64/gpu/cu128/Dockerfile …/x86_64/gpu/cu128/ubuntu24.04/Dockerfilebase/x86_64/gpu/cu128/Dockerfile renamed to base/x86_64/gpu/cu128/ubuntu24.04/Dockerfile base/x86_64/gpu/cu128/Dockerfile renamed to base/x86_64/gpu/cu128/ubuntu24.04/Dockerfile

@@ -0,0 +1,125 @@
+ARG PYTHON="python3"
+ARG PYTHON_VERSION="3.12.10"
+ARG PYTHON_SHORT_VERSION="3.12"
+ARG CUDA_MAJOR="12"
+ARG CUDA_MINOR="9"
+ARG EFA_VERSION="1.43.1"
+FROM nvidia/cuda:12.9.1-base-ubuntu22.04 AS base-builder
+
+
+RUN mv /usr/local/cuda/compat /usr/local \
+    && apt-get update \
+    && apt-get -y upgrade --only-upgrade systemd \
+    && apt-get install -y --allow-change-held-packages --no-install-recommends \
+        automake \
+        build-essential \
+        ca-certificates \
+        cmake \
+        curl \
+        emacs \
+        git \
+        jq \
+        libcurl4-openssl-dev \
+        libglib2.0-0 \
+        libegl1 \
+        libgl1 \
+        libsm6 \
+        libssl-dev \
+        libxext6 \
+        libxrender-dev \
+        zlib1g-dev \
+        unzip \
+        vim \
+        wget \
+        libhwloc-dev \
+        libgomp1 \
+        libibverbs-dev \
+        libnuma1 \
+        libnuma-dev \
+        libtool \
+        openssl \
+        python3-dev \
+        autoconf \
+        pkg-config \
+        check \
+        libsubunit0 \
+        libsubunit-dev \
+        libffi-dev \
+        libbz2-dev \
+        liblzma-dev \
+    && apt-get autoremove -y \
+    && apt-get clean \
+    && rm -rf /var/lib/apt/lists/*
+
+##############################################################################
+FROM base-builder AS python-builder
+ARG PYTHON_VERSION
+COPY install_python.sh install_python.sh
+RUN bash install_python.sh ${PYTHON_VERSION} && rm install_python.sh
+
+##############################################################################
+FROM base-builder AS cuda-builder
+ARG CUDA_MAJOR
+ARG CUDA_MINOR
+COPY install_cuda.sh install_cuda.sh
+RUN bash install_cuda.sh "${CUDA_MAJOR}.${CUDA_MINOR}" && rm install_cuda.sh
+
+##############################################################################
+FROM nvidia/cuda:12.9.1-base-ubuntu22.04 AS final 
+ARG PYTHON
+ARG PYTHON_SHORT_VERSION
+ARG CUDA_MAJOR
+ARG CUDA_MINOR
+ARG EFA_VERSION
+LABEL maintainer="Amazon AI"
+LABEL dlc_major_version="1"
+ENV DEBIAN_FRONTEND=noninteractive \
+    LANG=C.UTF-8 \
+    LC_ALL=C.UTF-8 \
+    DLC_CONTAINER_TYPE=base \
+    # Python won’t try to write .pyc or .pyo files on the import of source modules
+    # Force stdin, stdout and stderr to be totally unbuffered. Good for logging
+    PYTHONDONTWRITEBYTECODE=1 \
+    PYTHONUNBUFFERED=1 \
+    PYTHONIOENCODING=UTF-8 \
+    CUDA_HOME="/usr/local/cuda" \
+    PATH="/opt/amazon/openmpi/bin:/opt/amazon/efa/bin:/usr/local/cuda/bin:${PATH}" \
+    LD_LIBRARY_PATH="/usr/local/lib:/usr/local/cuda/lib64:/opt/amazon/ofi-nccl/lib:/opt/amazon/efa/lib:/opt/amazon/openmpi/lib:${LD_LIBRARY_PATH}"
+
+WORKDIR /
+
+# + python and pip packages (awscli, boto3, requests)
+COPY --from=python-builder /usr/local/lib/python${PYTHON_SHORT_VERSION} /usr/local/lib/python${PYTHON_SHORT_VERSION}
+COPY --from=python-builder /usr/local/include/python${PYTHON_SHORT_VERSION} /usr/local/include/python${PYTHON_SHORT_VERSION}
+COPY --from=python-builder /usr/local/bin /usr/local/bin
+# + cuda-toolkit, cudnn, nccl
+COPY --from=cuda-builder /usr/local/cuda-${CUDA_MAJOR}.${CUDA_MINOR} /usr/local/cuda-${CUDA_MAJOR}.${CUDA_MINOR}
+COPY install_efa.sh install_efa.sh 
+COPY deep_learning_container.py /usr/local/bin/deep_learning_container.py
+COPY bash_telemetry.sh /usr/local/bin/bash_telemetry.sh
+RUN chmod +x /usr/local/bin/deep_learning_container.py && \
+    chmod +x /usr/local/bin/bash_telemetry.sh && \
+    echo 'source /usr/local/bin/bash_telemetry.sh' >> /etc/bash.bashrc && \
+    # Install EFA
+    bash install_efa.sh ${EFA_VERSION} && \
+    rm install_efa.sh && \
+    # OSS compliance
+    apt-get update && \
+    apt-get upgrade -y && \
+    apt-get install -y --allow-change-held-packages --no-install-recommends \
+    unzip \
+    wget && \
+    apt-get clean && \
+    HOME_DIR=/root && \
+    curl -o ${HOME_DIR}/oss_compliance.zip https://aws-dlinfra-utilities.s3.amazonaws.com/oss_compliance.zip && \
+    unzip ${HOME_DIR}/oss_compliance.zip -d ${HOME_DIR}/ && \
+    cp ${HOME_DIR}/oss_compliance/test/testOSSCompliance /usr/local/bin/testOSSCompliance && \
+    chmod +x /usr/local/bin/testOSSCompliance && \
+    chmod +x ${HOME_DIR}/oss_compliance/generate_oss_compliance.sh && \
+    ${HOME_DIR}/oss_compliance/generate_oss_compliance.sh ${HOME_DIR} ${PYTHON} && \
+    rm -rf ${HOME_DIR}/oss_compliance* && \
+    rm -rf /tmp/tmp* && \
+    rm -rf /var/lib/apt/lists/* && \
+    rm -rf /root/.cache | true
+    
+CMD ["/bin/bash"]