Merge branch 'main' into nsarka/ddlb-integration

Author: nsarka

README.md

@@ -3,31 +3,16 @@
 CloudAI benchmark framework aims to develop an industry standard benchmark focused on grading Data Center (DC) scale AI systems in the Cloud. The primary motivation is to provide automated benchmarking on various systems.
 
 ## Get Started
-**Note**: instructions for installing a custom python version are available [here](#install-custom-python-version).
-
 **Note**: instructions for setting up access for `enroot` are available [here](#set-up-access-to-the-private-ngc-registry).
 
-1. Clone the CloudAI repository to your local machine:
-    ```bash
-    git clone git@github.com:NVIDIA/cloudai.git
-    cd cloudai
-    ```
-
-2. Create a virtual environment:
-    ```bash
-    python -m venv venv
-    source venv/bin/activate
-    ```
-
-3. Next, install the required packages:
-    ```bash
-    pip install .
-    ```
+Using `uv` tool allows users to run CloudAI without manually managing required Python versions and dependencies.
+```bash
+git clone git@github.com:NVIDIA/cloudai.git
+cd cloudai
+uv run cloudai --help
+```
 
-    For development please use the following command:
-    ```bash
-    pip install -e '.[dev]'  
-    ```
+For details and `pip`-based installation, please refer to the [documentation](https://nvidia.github.io/cloudai/#get-started).
 
 ## Key Concepts
 CloudAI operates on four main schemas:
@@ -75,17 +60,6 @@ machine nvcr.io login $oauthtoken password <api-key>
 Replace `<api-key>` with your respective credentials. Keep `$oauthtoken` as is.
 
 
-### Install custom python version
-If your system python version is not supported, you can install a custom version using [uv](https://docs.astral.sh/uv/getting-started/installation/) tool:
-```bash
-curl -LsSf https://astral.sh/uv/install.sh | sh
-source $HOME/.local/bin/env
-uv venv -p 3.10
-source .venv/bin/activate
-# optionally you might need to install pip which is not installed by default:
-uv pip install -U pip
-```
-
 ## CloudAI Modes Usage Examples
 
 CloudAI supports five modes:

doc/USER_GUIDE.md

@@ -78,7 +78,7 @@ scheduler = "slurm"
 
 install_path = "./install"
 output_path = "./results"
-cache_docker_images_locally = "True"
+cache_docker_images_locally = true
 default_partition = "<YOUR PARTITION NAME>"
 
 mpi = "pmix"
@@ -104,15 +104,15 @@ Test Configuration describes a particular test configuration to be run. It is ba
 name = "nccl_test_all_reduce_single_node"
 description = "all_reduce"
 test_template_name = "NcclTest"
-extra_cmd_args = "--stepfactor 2"
 
 [cmd_args]
-"subtest_name" = "all_reduce_perf_mpi"
-"ngpus" = "1"
-"minbytes" = "8M"
-"maxbytes" = "16G"
-"iters" = "5"
-"warmup_iters" = "3"
+subtest_name = "all_reduce_perf_mpi"
+ngpus = 1
+minbytes = "8M"
+maxbytes = "16G"
+iters = 5
+warmup_iters = 3
+stepfactor = 2
 ```
 You can find more examples under `conf/common/test`. In a test schema file, you can adjust arguments as shown above. In the `cmd_args` section, you can provide different values other than the default values for each argument. In `extra_cmd_args`, you can provide additional arguments that will be appended after the NCCL test command. You can specify additional environment variables in the `extra_env_vars` section.
 
@@ -122,12 +122,14 @@ Test Scenario uses Test description from step 5. Below is the `myconfig/scenario
 name = "nccl-test"
 
 [[Tests]]
-id = "Tests.1"
+id = "allreduce.1"
+num_nodes = 1
 test_name = "nccl_test_all_reduce_single_node"
 time_limit = "00:20:00"
 
 [[Tests]]
-id = "Tests.2"
+id = "allreduce.2"
+num_nodes = 1
 test_name = "nccl_test_all_reduce_single_node"
 time_limit = "00:20:00"
   [[Tests.dependencies]]
@@ -178,7 +180,7 @@ cloudai generate-report \
 # Describing a System in the System Schema
 In this section, we introduce the concept of the system schema, explain the meaning of each field, and describe how the fields should be used. The system schema is a TOML file that allows users to define a system's configuration.
 
-```
+```toml
 name = "example-cluster"
 scheduler = "slurm"
 
@@ -207,14 +209,14 @@ name = "partition_1"
 name = "partition_2"
 
 [global_env_vars]
-  # NCCL Specific Configurations
-  NCCL_IB_GID_INDEX = "3"
-  NCCL_IB_TIMEOUT = "20"
-  NCCL_IB_QPS_PER_CONNECTION = "4"
-
-  # Device Visibility Configuration
-  MELLANOX_VISIBLE_DEVICES = "0,3,4,5,6,9,10,11"
-  CUDA_VISIBLE_DEVICES = "0,1,2,3,4,5,6,7"
+# NCCL Specific Configurations
+NCCL_IB_GID_INDEX = "3"
+NCCL_IB_TIMEOUT = "20"
+NCCL_IB_QPS_PER_CONNECTION = "4"
+
+# Device Visibility Configuration
+MELLANOX_VISIBLE_DEVICES = "0,3,4,5,6,9,10,11"
+CUDA_VISIBLE_DEVICES = "0,1,2,3,4,5,6,7"
 ```
 
 ## Field Descriptions

doc/index.md

@@ -3,31 +3,35 @@
 CloudAI benchmark framework aims to develop an industry standard benchmark focused on grading Data Center (DC) scale AI systems in the Cloud. The primary motivation is to provide automated benchmarking on various systems.
 
 ## Get Started
-**Note**: instructions for installing a custom python version are available [here](install-custom-python-version).
-
 **Note**: instructions for setting up access for `enroot` are available [here](set-up-access-to-the-private-ngc-registry).
 
-1. Clone the CloudAI repository to your local machine:
-    ```bash
-    git clone git@github.com:NVIDIA/cloudai.git
-    cd cloudai
-    ```
+```bash
+git clone git@github.com:NVIDIA/cloudai.git
+cd cloudai
+uv run cloudai --help
+```
 
-2. Create a virtual environment:
-    ```bash
-    python -m venv venv
-    source venv/bin/activate
-    ```
+### `pip`-based installation
+See required Python version in the `.python-version` file, please ensure you have it installed (see how a custom python version [can be installed](#install-custom-python-version)). Follow these steps:
+```bash
+git clone git@github.com:NVIDIA/cloudai.git
+cd cloudai
+python -m venv venv
+source venv/bin/activate
+pip install -e .
+```
 
-3. Next, install the required packages:
-    ```bash
-    pip install .
-    ```
+(install-custom-python-version)=
+### Install custom python version
+If your system python version is not supported, you can install a custom version using [uv](https://docs.astral.sh/uv/getting-started/installation/) tool:
+```bash
+curl -LsSf https://astral.sh/uv/install.sh | sh
+source $HOME/.local/bin/env
+uv venv --seed  # this will pick up the python version from .python-version file
+                # --seed will install pip and setuptools
+source .venv/bin/activate
+```
 
-    For development please use the following command:
-    ```bash
-    pip install -e '.[dev]'  
-    ```
 
 ## Key Concepts
 CloudAI operates on four main schemas:
@@ -73,19 +77,6 @@ machine nvcr.io login $oauthtoken password <api-key>
 ```
 Replace `<api-key>` with your respective credentials. Keep `$oauthtoken` as is.
 
-
-(install-custom-python-version)=
-### Install custom python version
-If your system python version is not supported, you can install a custom version using [uv](https://docs.astral.sh/uv/getting-started/installation/) tool:
-```bash
-curl -LsSf https://astral.sh/uv/install.sh | sh
-source $HOME/.local/bin/env
-uv venv -p 3.10
-source .venv/bin/activate
-# optionally you might need to install pip which is not installed by default:
-uv pip install -U pip
-```
-
 ## CloudAI Modes Usage Examples
 
 CloudAI supports five modes: