Huggingface pytorch inference latest (#4686)

* Update ['dlc_developer_config.toml']

dlc_developer_config.toml:
{   'build': {   'build_frameworks': ['huggingface_pytorch'],
                 'build_inference': False,
                 'build_training': True},
    'buildspec_override': {   'dlc-pr-huggingface-pytorch-training': 'huggingface/pytorch/training/buildspec-2-3-0.yml'},
    'dev': {   'arm64_mode': False,
               'deep_canary_mode': False,
               'graviton_mode': False,
               'neuronx_mode': False},
    'test': {   'ec2_tests': True,
                'ecs_tests': True,
                'eks_tests': True,
                'sagemaker_local_tests': True,
                'sagemaker_remote_tests': True,
                'sanity_tests': True,
                'security_tests': True}}

* chore: pinning cmake for huggingface pytorch training 2.3

* chore: moving patch

* chore: pinning kenlm instead of cmake

* chore: pinning cmake

* chore: fixing syntax

* Revert "Update ['dlc_developer_config.toml']"

This reverts commit c0b1515.

* Update ['dlc_developer_config.toml']

dlc_developer_config.toml:
{   'build': {   'build_frameworks': ['huggingface_pytorch'],
                 'build_inference': True,
                 'build_training': False},
    'buildspec_override': {   'dlc-pr-huggingface-pytorch-inference': 'huggingface/pytorch/inference/buildspec.yml'},
    'dev': {   'arm64_mode': False,
               'deep_canary_mode': False,
               'graviton_mode': False,
               'neuronx_mode': False},
    'test': {   'ec2_tests': True,
                'ecs_tests': True,
                'eks_tests': True,
                'sagemaker_local_tests': True,
                'sagemaker_remote_tests': True,
                'sanity_tests': True,
                'security_tests': True}}

* chore: pinning cmake in conda install for huggingface pytorch inference gpu and cpu latest

* chore: selectively install kenln prior to other installs

* chore: pinning cmake

* Revert "Update ['dlc_developer_config.toml']"

This reverts commit 37a13b3.

---------

Co-authored-by: shaernev <shaernev@amazon.com>

Author: ErnevSharma

huggingface/pytorch/inference/docker/2.6/py3/Dockerfile.cpu

@@ -118,7 +118,7 @@ RUN curl -L -o ~/miniforge3.sh https://github.com/conda-forge/miniforge/releases
     h5py \
     requests \
     libgcc \
-    cmake \
+    "cmake<=3.24.4" \
     packaging \
     "awscli<2" \
     boto3 \
@@ -195,6 +195,9 @@ RUN chmod +x /usr/local/bin/deep_learning_container.py
 # Hugging Face specific section #
 #################################
 
+# Pre-install kenlm without build isolation so it uses system cmake
+RUN pip install --no-cache-dir --no-build-isolation kenlm
+
 # install Hugging Face libraries and its dependencies
 RUN pip install --no-cache-dir \
     # hf_transfer will be a built-in feature, remove the extra then

huggingface/pytorch/inference/docker/2.6/py3/cu124/Dockerfile.gpu

@@ -168,7 +168,7 @@ RUN /opt/conda/bin/conda install -y -c conda-forge \
     typing \
     h5py \
     libgcc \
-    cmake \
+    "cmake<=3.24.4" \
     packaging \
     "awscli<2" \
     boto3 \
@@ -253,6 +253,9 @@ RUN chmod +x /usr/local/bin/cuda-compatibility-lib.sh
 # Hugging Face specific section #
 #################################
 
+# Pre-install kenlm without build isolation so it uses system cmake
+RUN pip install --no-cache-dir --no-build-isolation kenlm
+
 # install Hugging Face libraries and its dependencies
 RUN pip install --no-cache-dir \
     # hf_transfer will be a built-in feature, remove the extra then