Update README and environment configuration for Azure ML

- Simplified instructions in README regarding environment updates by removing redundant information about dependencies in requirements.txt.
- Adjusted the build context in create-env.yaml to use the repository root for Dockerfile compatibility, ensuring correct path resolution for dependencies.

Author: fujikosu

.azuredevops/pull_request_template.md

@@ -12,7 +12,7 @@
 
 * [ ] No PII in logs or output
 * [ ] Made corresponding changes to the documentation
-* [ ] All new packages used are included in requirements.txt
+* [ ] All new packages used are included in pyproject.toml
 * [ ] Functions use type hints, and there are no type hint errors
 
 ## Pull Request Type

README.md

@@ -12,6 +12,7 @@ A machine learning and data science project template that makes it easy to work
     - [Features](#features)
   - [Getting Started](#getting-started)
     - [How to setup dev environment?](#how-to-setup-dev-environment)
+  - [How to create a new directory under src with a new environment](#how-to-create-a-new-directory-under-src-with-a-new-environment)
   - [How to update python packages in the dev container](#how-to-update-python-packages-in-the-dev-container)
   - [Directory Structure](#directory-structure)
     - [`notebooks` directory vs `src` directory](#notebooks-directory-vs-src-directory)
@@ -30,7 +31,7 @@ A machine learning and data science project template that makes it easy to work
 
 This repository provides a [VSCode Dev Container](https://code.visualstudio.com/docs/devcontainers/containers) based project template that can help accelerate your Machine Learning inner-loop development phase. The template covers the phases from early ML experimentation (local training/testing) until production oriented ML model training (cloud based training/testing with bigger CPUs and GPUs).
 
-During the early phase of Machine Learning project, you may face challenges such as each data scientist creating various different python environments that span across CPU and GPU that tend to have different setup procedures. With the power of Dev Containers, you can automate environment setup process across the team and every data scientist will get the exact same environment automatically. This template provides both CPU and GPU Dev Container setup as examples. To support multiple different ML approaches with different python environments to be experimented in one project, this solution allows multiple different Dev Containers to be used in one repository while having a "common" module that will be installed into all Dev Container to enable code reuse across different Dev Containers.
+During the early phase of Machine Learning project, you may face challenges such as each data scientist creating various different python environments that span across CPU and GPU that tend to have different setup procedures. With the power of Dev Containers, you can automate environment setup process across the team and every data scientist will get the exact same environment automatically. This template provides both CPU and GPU Dev Container setup as examples. To support multiple different ML approaches with different python environments to be experimented in one project, this solution allows multiple different Dev Containers to be used in one repository.
 
 Another challenge you may face is each data scientist creating a low quality codebase. That is fine during the experimentation stage to keep the team agility high and maximize your team’s experimentation throughput. But when you move to the model productionization stage, you experience the burden of bringing code quality up to production level. With the power of python tools and VSCode extensions configured for this template on top of Dev Containers, you can keep the code quality high automatically without losing your team’s agility and experimentation throughput and ease the transition to the productionization phase.
 
@@ -102,17 +103,17 @@ This section gives you overview of the directory structure of this template. Onl
 │   ├── .devcontainer              # dev container related configuration files goes to here following VSCode convention
 │   │   ├── devcontainer.json      # dev container configuration and VS Code settings, extensions etc.
 │   │   ├── Dockerfile             # referred in devcontainer.json
-│   │   └── pyproject.toml       # includes python package list for notebooks. used in Dockerfile
+│   │   ├── pyproject.toml         # includes python package list for notebooks. used in Dockerfile
+│   │   └── uv.lock                # lock file for python packages. used in Dockerfile
 │   └── sample_notebook.py         # example of interactive python script
 ├── pyproject.toml                 # Setting file for ruff, pytest and pytest-cov
 └── src
-    ├── common                     # this module is accessible from all modules under src. put functions  you want to import across the projects here
-    │   └── requirements.txt       # python package list for common module. installed in all Dockerfile under src. python tools for src goes to here too
     ├── sample_cpu_project         # cpu project example. Setup process is covered in Section: How to setup dev environment?
     │   ├── .devcontainer          # dev container related configuration files goes to here following VSCode convention
     │   │   ├── devcontainer.json  # dev container configuration and VS Code settings, extensions etc.
     │   │   ├── Dockerfile         # referred in devcontainer.json. Supports only CPU
-    │   │   └── pyproject.toml   # includes python package list for sample_cpu_project. used in Dockerfile
+    │   │   ├── pyproject.toml     # includes python package list for sample_cpu_project. used in Dockerfile
+    │   │   └── uv.lock            # lock file for python packages. used in Dockerfile
     │   ├── sample_main.py         
     │   └── tests                  # pytest scripts for sample_cpu_project goes here
     │       └── test_dummy.py      # pytest script example
@@ -121,7 +122,8 @@ This section gives you overview of the directory structure of this template. Onl
         ├── .devcontainer          # dev container related configuration files goes to here following VSCode convention
         │   ├── devcontainer.json  # dev container configuration and VS Code settings, extensions etc.
         │   ├── Dockerfile         # referred in devcontainer.json. Supports GPU
-        │   └── pyproject.toml   # includes python package list for sample_pytorch_gpu_project. used in Dockerfile
+        │   ├── pyproject.toml     # includes python package list for sample_pytorch_gpu_project. used in Dockerfile
+        │   └── uv.lock            # lock file for python packages. used in Dockerfile
         ├── aml_example/           # Sample AML CLI v2 Components-based pipeline, including setup YAML. See sample_pytorch_gpu_project/README for full details of files in this directory.
         ├── sample_main.py        
         ├── inference.py           # Example pytorch inference/eval script that also works with aml_example
@@ -224,7 +226,6 @@ ssh-add
 ## Future Roadmap
 
 - Add Docker build caching to Azure DevOps MS hosted CI pipeline
-- Investigate making `src/common` installed with `pip -e`
 
 ## Contributing
 

notebooks/.devcontainer/Dockerfile

@@ -26,7 +26,8 @@ ENV UV_PROJECT_FILE=.devcontainer/pyproject.toml
 
 # Changing the default UV_LINK_MODE silences warnings about not being able to use hard links since the cache and sync target are on separate file systems
 ENV UV_LINK_MODE=copy
-# Install dependencies (as root for simplicity; devuser can still use installed packages)
+# Install dependencies using bind mounts instead of COPY to avoid extra layers
+# This is the recommended approach by uv: https://docs.astral.sh/uv/guides/integration/docker/#installing-a-project
 RUN --mount=type=cache,target=/root/.cache/uv \
     --mount=type=bind,source=notebooks/.devcontainer/uv.lock,target=uv.lock \
     --mount=type=bind,source=notebooks/.devcontainer/pyproject.toml,target=pyproject.toml \

notebooks/.devcontainer/devcontainer.json

@@ -1,7 +1,7 @@
 {
     "name": "DSToolkit Notebooks Dev Container",
     "build": {
-        // use root directory as build context so that requirements-dev.txt is accessible during build
+        // use root directory as build context so that pyproject.toml and uv.lock are accessible during build
         "context": "../../",
         "dockerfile": "Dockerfile"
     },

src/common/__init__.py

@@ -26,18 +26,13 @@ ENV UV_PROJECT_FILE=.devcontainer/pyproject.toml
 
 # Changing the default UV_LINK_MODE silences warnings about not being able to use hard links since the cache and sync target are on separate file systems
 ENV UV_LINK_MODE=copy
-# Install dependencies (as root for simplicity; devuser can still use installed packages)
+# Install dependencies using bind mounts instead of COPY to avoid extra layers
+# This is the recommended approach by uv: https://docs.astral.sh/uv/guides/integration/docker/#installing-a-project
 RUN --mount=type=cache,target=/root/.cache/uv \
     --mount=type=bind,source=src/sample_cpu_project/.devcontainer/uv.lock,target=uv.lock \
     --mount=type=bind,source=src/sample_cpu_project/.devcontainer/pyproject.toml,target=pyproject.toml \
     uv sync --locked --project $UV_PROJECT_FILE
 
-# install common module related packages
-# This part can be potentially improved by https://docs.astral.sh/uv/concepts/projects/workspaces/#when-not-to-use-workspaces to move away from requirements.txt and gets its own lock file
-COPY src/common/requirements.txt .
-RUN --mount=type=cache,target=/root/.cache/uv \
-    uv pip install -r requirements.txt --system
-
 # Allow devuser to manage packages at runtime without sudo (e.g. uv add)
 RUN chown -R $USERNAME:$USERNAME /usr/local
 USER $USERNAME

src/common/requirements.txt

@@ -1,7 +1,7 @@
 {
     "name": "Sample CPU Project Dev Container",
     "build": {
-        // use root directory as build context so that requirements-dev.txt is accessible during build
+        // use root directory as build context so that pyproject.toml and uv.lock are accessible during build
         "context": "../../../",
         "dockerfile": "Dockerfile"
     },

src/sample_cpu_project/.devcontainer/Dockerfile

@@ -5,6 +5,7 @@ requires-python = ">=3.11"
 
 [dependency-groups]
 dev = [
+    "ipykernel==7.2.0",
     "mypy==1.20.0",
     "pytest==9.0.2",
     "pre-commit==4.5.1",