Minor updates (open-edge-platform#1248)

Co-authored-by: Hoong Tee, Yeoh <[email protected]>
Co-authored-by: Raghu Bhat <[email protected]>

Author: 3 people

microservices/model-download/README.md

@@ -1,93 +1,20 @@
-# Model Download Microservice Overview
+# Model Download Microservice
 
-The Model Download Microservice provides a unified solution for downloading AI/ML models from various model hubs while ensuring consistency and simplicity across applications. This service acts as a centralized model management system that handles model downloads, storage, and optional format conversions.
+The Model Download Microservice offers a streamlined approach for acquiring AI/ML models from multiple model hubs, promoting consistency and ease of use across different applications. It serves as a centralized system for managing model downloads, storage, and optional format conversions.
 
-## Architecture
+Below, you'll find links to detailed documentation to help you get started, configure, and deploy the microservice.
 
-The diagram below illustrates the high-level architecture of the Model Download Microservice, showcasing its core components and their interactions with external systems.
+## Documentation
 
-<p align="center">
-    <img src="./docs/user-guide/images/architecture.png" alt="Architecture" />
-</p>
+- **Overview**
+  - [Overview](docs/user-guide/Overview.md): A high-level introduction to the microservice.
 
-## Components
+- **Getting Started**
+  - [Get Started](docs/user-guide/get-started.md): Step-by-step guide to getting started with the microservice.
+  - [System Requirements](docs/user-guide/system-requirements.md): Hardware and software requirements for running the microservice.
 
-The service follows a plugin-based microservice architecture with the following key components:
+- **Deployment**
+  - [How to Build from Source](docs/user-guide/build-from-source.md): Instructions for building the microservice from source code.
 
-### Core Components
-
-1. **FastAPI Service Layer**
-   - **Description**: The FastAPI Service Layer serves as the primary entry point for all client interactions. It exposes a RESTful API for downloading, converting, and managing models.
-   - **Responsibilities**:
-     - Provides RESTful API endpoints for all service operations.
-     - Handles incoming request validation, serialization, and routing to the appropriate components.
-     - Generates and serves OpenAPI (Swagger) documentation for clear, interactive API specifications.
-
-2. **Model Manager**
-   - **Description**: The Model Manager is the central orchestration component that directs the model download and conversion processes. It acts as the brain of the service, coordinating actions between the API layer and the plugin system.
-   - **Responsibilities**:
-     - Orchestrates end-to-end model download and conversion workflows.
-     - Manages model storage, including organizing file paths and handling caching.
-     - Interfaces with the Plugin Registry to delegate tasks to the appropriate plugins.
-
-3. **Plugin Registry**
-   - **Description**: The Plugin Registry is responsible for the discovery, registration, and management of all available plugins. It provides a flexible mechanism for extending the service's capabilities without modifying the core application logic.
-   - **Responsibilities**:
-     - Dynamically discovers and registers plugins at startup.
-     - Manages the lifecycle of each plugin.
-     - Provides a consistent abstraction layer that decouples the Model Manager from concrete plugin implementations.
-
-### Plugin System
-
-The service's functionality is extended through a modular plugin system that handles interactions with different model sources and conversion tasks.
-
-**Model Hub Plugins:**
-- **HuggingFace Hub Plugin**: Manages model downloads from the Hugging Face Hub, including handling authentication for private or gated models.
-- **Ollama Hub Plugin**: Interfaces with Ollama to pull and manage models from the Ollama model library.
-- **Ultralytics Hub Plugin**: Downloads computer vision models, such as YOLO, from the Ultralytics framework.
-
-**Conversion Plugins:**
-- **OpenVINO Model Conversion Plugin**: Provides functionality to convert downloaded models (e.g., from Hugging Face) into the OpenVINO™ Intermediate Representation (IR) format for optimized inference on Intel hardware.
-
-### Storage
-
-- **Downloaded Models Storage**: This component represents the physical storage location for all downloaded and converted models. It is a configurable file system path that acts as a centralized repository and cache.
-  - **Responsibilities**:
-    - Provides a persistent location for storing model files.
-    - Enables caching to avoid redundant downloads of the same model.
-    - Organizes models in a structured directory format for easy access.
-
-## Key Features
-
-- **Multi-Hub Support**: Download models from multiple sources (HuggingFace, Ollama, Ultralytics)
-- **Format Conversion**: Convert models to OpenVINO format for optimization
-- **Parallel Downloads**: Optional concurrent model downloads
-- **Precision Control**: Support for various model precisions (INT8, FP16, FP32)
-- **Device Targeting**: Optimization for different compute devices (CPU, GPU)
-- **Caching**: Configurable model caching for improved performance
-
-## Integration
-
-The service can be integrated into applications through:
-- REST API calls
-- Docker container deployment
-- Docker Compose orchestration
-
-## Use Cases
-
-This microservice is ideal for:
-- Edge AI applications requiring model downloads
-- Development and testing environments
-- Sample applications demonstrating AI capabilities
-- Automated model deployment pipelines
-
-## Limitations
-
-This service is not intended to replace full model registry solutions and has the following limitations:
-- Basic model versioning
-- Limited model metadata management
-- No built-in model serving capabilities
-
-## Supporting Resources
-- [**Get Started Guide**](./docs/user-guide/get-started.md)
-- [**API Reference**](./docs/user-guide/api-docs/openapi.yaml)
+- **Release Notes**
+  - [Release Notes](docs/user-guide/release-notes.md): Information on the latest updates, improvements, and bug fixes.

microservices/model-download/chart/Chart.yaml

@@ -2,5 +2,5 @@ apiVersion: v2
 name: model-download
 description: A Helm chart for deploying the model-download FastAPI microservice
 type: application
-version: 1.0.0
-appVersion: "1.0.0"
+version: 1.0.1
+appVersion: "1.0.1"

microservices/model-download/chart/templates/deployment.yaml

@@ -18,7 +18,6 @@ spec:
         fsGroup: 1001
         runAsUser: 1001
         runAsGroup: 1001
-        allowPrivilegeEscalation: false
       containers:
         - name: model-download
           image: "{{ .Values.modeldownload.image.registry }}model-download:{{ .Values.modeldownload.image.tag }}"
@@ -51,6 +50,8 @@ spec:
           volumeMounts:
             - name: models
               mountPath: /opt/models
+          securityContext:
+            allowPrivilegeEscalation: false
       volumes:
         - name: models
           persistentVolumeClaim:

microservices/model-download/chart/values.yaml

@@ -11,7 +11,7 @@ modeldownload:
   name: model-download
   image:
     registry: ""  #provide your registry info here, replace 'registry' with actual registry URL
-    tag: 1.0.0
+    tag: 1.0.1
     pullPolicy: IfNotPresent
 
   readinessProbe:

microservices/model-download/docker/Dockerfile

@@ -4,9 +4,6 @@
 # ---- Stage 1: Build dependencies ----
 FROM python:3.11-slim AS python-base
 
-# Add uv to PATH
-ENV PATH="/opt/.local/bin:$PATH"
-
 # Set the working directory in the container
 WORKDIR /opt
 
@@ -21,7 +18,7 @@ RUN apt-get update && apt-get install -y \
     rm -rf /var/lib/apt/lists/*
 
 # Install uv
-RUN curl -LsSf https://astral.sh/uv/install.sh | UV_INSTALL_DIR=/opt/.local/bin sh
+RUN pip install uv==0.9.10
 
 # Copy project files
 COPY pyproject.toml uv.lock /opt/
@@ -53,7 +50,6 @@ RUN groupadd -g ${GID} appuser && \
 # Copy installed dependencies from python-base
 COPY --from=python-base /usr/local/lib/python3.11/site-packages /usr/local/lib/python3.11/site-packages
 COPY --from=python-base /usr/local/bin /usr/local/bin
-COPY --from=python-base /opt/.local/bin/uv /usr/local/bin/uv
 
 COPY src /opt/src
 COPY scripts /opt/scripts

microservices/model-download/docker/entrypoint.sh

@@ -103,52 +103,48 @@ while [[ $# -gt 0 ]]; do
     esac
 done
 
-# Define all available plugins in the application
+
 AVAILABLE_PLUGINS=("openvino" "huggingface" "ollama" "ultralytics")
+PLUGINS_LOWER=$(echo "$PLUGINS" | tr '[:upper:]' '[:lower:]')
 
-# Install plugin-specific dependencies
+# Determine which plugins to activate
 print_header "Installing plugin dependencies"
-if [ "$PLUGINS" = "all" ]; then
+if [ "$PLUGINS_LOWER" = "all" ]; then
+    PLUGIN_LIST=("${AVAILABLE_PLUGINS[@]}")
     print_info "Installing ALL plugins"
-    
-    # Install dependencies for all available plugins
-    for plugin in "${AVAILABLE_PLUGINS[@]}"; do
-        install_dependencies "$plugin"
-    done
-
-    echo "ACTIVATED_PLUGINS=all" > "$PLUGINS_ENV_FILE"
-    print_success "All plugins are activated"
 else
-    # Split comma-separated plugins and install dependencies for each
-    IFS=',' read -ra PLUGIN_LIST <<< "$PLUGINS"
-    echo "ACTIVATED_PLUGINS=$PLUGINS" > "$PLUGINS_ENV_FILE"
-    
-    for plugin in "${PLUGIN_LIST[@]}"; do
-        install_dependencies "$plugin"
+    # Split comma-separated plugins into array and convert to lowercase
+    IFS=',' read -ra PLUGIN_LIST_RAW <<< "$PLUGINS_LOWER"
+    PLUGIN_LIST=()
+    for plugin in "${PLUGIN_LIST_RAW[@]}"; do
+        # Trim whitespace and add to array
+        plugin=$(echo "$plugin" | xargs)
+        PLUGIN_LIST+=("$plugin")
     done
-    
-    print_success "Activated plugins: $PLUGINS"
 fi
 
+# Install plugin-specific dependencies
+for plugin in "${PLUGIN_LIST[@]}"; do
+    install_dependencies "$plugin"
+done
+
+# Save activated plugins to env file
+echo "ACTIVATED_PLUGINS=$PLUGINS" > "$PLUGINS_ENV_FILE"
+print_success "Activated plugins: ${PLUGIN_LIST[*]}"
+
+# Build the list of --extra arguments from the activated plugins
+EXTRA_ARGS=()
+for plugin in "${PLUGIN_LIST[@]}"; do
+    EXTRA_ARGS+=(--extra "$plugin")
+done
+
 # Sync dependencies using UV
 print_header "Syncing dependencies with UV"
 cd /opt
 print_info "Installing dependencies from pyproject.toml..."
 
-# Add UV and ollama to PATH if it's not already there
-export PATH="/usr/local/bin:$HOME/.local/bin:/opt/bin/:$PATH"
-
-# Build the list of --extra arguments from the activated plugins
-EXTRA_ARGS=()
-if [ "$PLUGINS" = "all" ]; then
-    for plugin in "${AVAILABLE_PLUGINS[@]}"; do
-        EXTRA_ARGS+=(--extra "$plugin")
-    done
-else
-    for plugin in "${PLUGIN_LIST[@]}"; do
-        EXTRA_ARGS+=(--extra "$plugin")
-    done
-fi
+# ollama to PATH if it's not already there
+export PATH="/opt/bin/:$PATH"
 
 if uv sync "${EXTRA_ARGS[@]}"; then
     print_success "Dependencies synced successfully"
@@ -174,4 +170,4 @@ if [ "$START_SERVICE" = true ]; then
 else
     print_warning "Service start skipped due to --no-start flag"
     exec "$@"
-fi
+fi

microservices/model-download/docs/user-guide/Overview.md

@@ -0,0 +1,94 @@
+## Model Download
+
+The Model Download Microservice provides a unified solution for downloading AI/ML models from various model hubs while ensuring consistency and simplicity across applications. This service acts as a centralized model management system that handles model downloads, storage, and optional format conversions.
+
+
+## Architecture
+
+The diagram below illustrates the high-level architecture of the Model Download Microservice, showcasing its core components and their interactions with external systems.
+
+<p align="center">
+    <img src="./images/architecture.png" alt="Architecture" />
+</p>
+
+## Components
+
+The service follows a plugin-based microservice architecture with the following key components:
+
+### Core Components
+
+1. **FastAPI Service Layer**
+   - **Description**: The FastAPI Service Layer serves as the primary entry point for all client interactions. It exposes a RESTful API for downloading, converting, and managing models.
+   - **Responsibilities**:
+     - Provides RESTful API endpoints for all service operations.
+     - Handles incoming request validation, serialization, and routing to the appropriate components.
+     - Generates and serves OpenAPI (Swagger) documentation for clear, interactive API specifications.
+
+2. **Model Manager**
+   - **Description**: The Model Manager is the central orchestration component that directs the model download and conversion processes. It acts as the brain of the service, coordinating actions between the API layer and the plugin system.
+   - **Responsibilities**:
+     - Orchestrates end-to-end model download and conversion workflows.
+     - Manages model storage, including organizing file paths and handling caching.
+     - Interfaces with the Plugin Registry to delegate tasks to the appropriate plugins.
+
+3. **Plugin Registry**
+   - **Description**: The Plugin Registry is responsible for the discovery, registration, and management of all available plugins. It provides a flexible mechanism for extending the service's capabilities without modifying the core application logic.
+   - **Responsibilities**:
+     - Dynamically discovers and registers plugins at startup.
+     - Manages the lifecycle of each plugin.
+     - Provides a consistent abstraction layer that decouples the Model Manager from concrete plugin implementations.
+
+### Plugin System
+
+The service's functionality is extended through a modular plugin system that handles interactions with different model sources and conversion tasks.
+
+**Model Hub Plugins:**
+- **HuggingFace Hub Plugin**: Manages model downloads from the Hugging Face Hub, including handling authentication for private or gated models.
+- **Ollama Hub Plugin**: Interfaces with Ollama to pull and manage models from the Ollama model library.
+- **Ultralytics Hub Plugin**: Downloads computer vision models, such as YOLO, from the Ultralytics framework.
+
+**Conversion Plugins:**
+- **OpenVINO Model Conversion Plugin**: Provides functionality to convert downloaded models (e.g., from Hugging Face) into the OpenVINO™ Intermediate Representation (IR) format for optimized inference on Intel hardware.
+
+### Storage
+
+- **Downloaded Models Storage**: This component represents the physical storage location for all downloaded and converted models. It is a configurable file system path that acts as a centralized repository and cache.
+  - **Responsibilities**:
+    - Provides a persistent location for storing model files.
+    - Enables caching to avoid redundant downloads of the same model.
+    - Organizes models in a structured directory format for easy access.
+
+## Key Features
+
+- **Multi-Hub Support**: Download models from multiple sources (HuggingFace, Ollama, Ultralytics)
+- **Format Conversion**: Convert models to OpenVINO format for optimization
+- **Parallel Downloads**: Optional concurrent model downloads
+- **Precision Control**: Support for various model precisions (INT8, FP16, FP32)
+- **Device Targeting**: Optimization for different compute devices (CPU, GPU)
+- **Caching**: Configurable model caching for improved performance
+
+## Integration
+
+The service can be integrated into applications through:
+- REST API calls
+- Docker container deployment
+- Docker Compose orchestration
+
+## Use Cases
+
+This microservice is ideal for:
+- Edge AI applications requiring model downloads
+- Development and testing environments
+- Sample applications demonstrating AI capabilities
+- Automated model deployment pipelines
+
+## Limitations
+
+This service is not intended to replace full model registry solutions and has the following limitations:
+- Basic model versioning
+- Limited model metadata management
+- No built-in model serving capabilities
+
+## Supporting Resources
+- [**Get Started Guide**](./get-started.md)
+- [**API Reference**](./api-docs/openapi.yaml)

microservices/model-download/docs/user-guide/deploy-with-helm.md

@@ -101,7 +101,7 @@ kubectl get services -n <your-namespace>
 
 ### Step 6: Access the Application
 
-Open the application swagger documentation in a browser at `http://\<node-ip\>:\<node-port\>/api/v1/docs`
+Open the application swagger documentation in a browser at `http://<node-ip>:<node-port>/api/v1/docs`
 
 ### Step 7: Uninstall Helm chart
 

microservices/model-download/docs/user-guide/get-started.md

@@ -154,7 +154,7 @@ curl -X POST "http://<host-ip>:8200/api/v1/models/download?download_path=ovms_mo
         "is_ovms": true,
         "config": {
           "precision": "fp32",
-          "device": "cpu",
+          "device": "CPU",
           "cache_size": 10
         }
       }

microservices/model-download/src/api/main.py

@@ -132,6 +132,7 @@ async def download_models(
                     hub=model.hub,
                     output_dir=model_download_path,
                     plugin_name=model.hub,
+                    model_type=model.type,
                 )
 
                 # Add to job_ids for response
@@ -176,7 +177,8 @@ async def download_models(
                     model_name=model.name,
                     hub=model.hub,
                     output_dir=convert_output_dir,
-                    plugin_name="openvino"
+                    plugin_name="openvino",
+                    model_type=model.type,
                 )
 
                 # Add to job_ids for response