Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ To learn about cross stream batching feature, refer this [doc](./cross-stream-ba
To enable Open Telemetry and capture various runtime statistics, refer this [doc](./enable-open-telemetry.md)

## Working with other services
To learn how DL Streamer Pipeline Server interacts with other microservices such as Model Registry, refer this [doc](./work-with-other-services.md)
To learn how DL Streamer Pipeline Server interacts with other microservices such as Model Download, refer this [doc](./work-with-other-services.md)

<!--hide_directive
```{toctree}
Expand Down
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
# Model Update

## Model Registry service integration
DL Streamer Pipeline Server supports MLOps capabilities through integration with the Model Registry service. This enhancement streamlines the model management process, enabling dynamic updates and ensuring that DL Streamer Pipeline Server and ensuring that DL Streamer Pipeline Server utilizes the most accurate and up-to-date models. Currently it supports following features:
## Model Download service integration
DL Streamer Pipeline Server supports MLOps capabilities through integration with the Model Download service. This enhancement streamlines the model management process, enabling dynamic updates and ensuring that DL Streamer Pipeline Server and ensuring that DL Streamer Pipeline Server utilizes the most accurate and up-to-date models. Currently it supports following features:

**Model Initialization**: Upon initialization, DL Streamer Pipeline Server can retrieve and deploy machine learning models directly from a Model Registry. To leverage this feature, users must specify the necessary model details in the configuration file.
**Model Initialization**: Upon initialization, DL Streamer Pipeline Server can retrieve and deploy machine learning models directly from Model Download. To leverage this feature, users must specify the necessary model details in the configuration file.

**Runtime Model Update**: DL Streamer Pipeline Server offers a REST endpoint that facilitates on-demand model updates during runtime. This endpoint allows users to pull the latest model version from the Model Registry and download it for immediate use within the existing DL Streamer Pipeline Server pipeline. This endpoint also provides options to dynamically restart pipeline with new model.
**Runtime Model Update**: DL Streamer Pipeline Server offers a REST endpoint that facilitates on-demand model updates during runtime. This endpoint allows users to download the latest model version from Model Download for immediate use within the existing DL Streamer Pipeline Server pipeline. This endpoint also provides options to dynamically restart pipeline with new model.

Refer the detailed [documentation](../work-with-other-services.md#model-registry-mraas) for steps on launching the model registry microservice and configurations to interact with it during DL Streamer Pipeline Server's startup and runtime.
Refer the detailed [documentation](../work-with-other-services.md#model-registry-mraas) for steps on launching the Model Download microservice and configurations to interact with it during DL Streamer Pipeline Server's startup and runtime.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what about this link? it goes to model-registry ...

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ The following REST API endpoints are currently supported.
| `GET /pipelines/{instance_id}` | Return pipeline instance summary. |
| `POST /pipelines/{name}/{version}/{instance_id}` | Send request to an already queued pipeline. Supported only for source of type `"image-ingestor"` |
| `DELETE /pipelines/{instance_id}` | Stops a running pipeline. |
| `POST /pipelines/{name}/{version}/{instance_id}/models` | Download files from the model registry microservice associated with a specific model. |
| `POST /pipelines/{name}/{version}/{instance_id}/models` | Download files from the Model Download microservice associated with a specific model. |

Learn how to access and utilize the RESTful endpoints provided by the Pipeline Server for efficient pipeline management [here](../../../api-reference.md).

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ The RESTful API has a default maximum body size of 10 KB, this can be changed by
| [`GET` /pipelines/{instance_id}](#get-pipelinesinstance_id) | Return pipeline instance summary. |
| [`POST` /pipelines/{name}/{version}/{instance_id}](#post-pipelinesnameversioninstance_id) | Send request to an already queued pipeline. Supported only for source of type "image_ingestor". |
| [`DELETE` /pipelines/{instance_id}](#delete-pipelinesinstance_id) | Stops a running pipeline or cancels a queued pipeline. |
| [`POST` /pipelines/{name}/{version}/{instance_id}/models](#post-pipelinesnameversioninstance_idmodels) | Download files from the model registry microservice associated with a specific model and deploy it in pipeline. |
| [`POST` /pipelines/{name}/{version}/{instance_id}/models](#post-pipelinesnameversioninstance_idmodels) | Download files from the Model Download microservice associated with a specific model and deploy it in pipeline. |

### `GET` /pipelines

Expand Down Expand Up @@ -254,11 +254,11 @@ Accepted values: any

### `POST` /pipelines/{name}/{version}/{instance_id}/models

Download files from the model registry microservice associated with a specific model and deploy the newly downloaded model in the pipeline.
- Properties are supported: `name` (string), `project_name` (string), `version` (string), `category` (string), `architecture` (string), `precision` (string), `deploy` (boolean), and `pipeline_element_name`(string). These properties allow DL Streamer Pipeline Server to query for a model stored in the model registry microservice.
Download files from the Model Download microservice associated with a specific model and deploy the newly downloaded model in the pipeline.
- Properties are supported: `name` (string), `project_name` (string), `version` (string), `category` (string), `architecture` (string), `precision` (string), `deploy` (boolean), and `pipeline_element_name`(string). These properties allow DL Streamer Pipeline Server to query for a model stored in the Model Download microservice.
- If `deploy` is set to `true`, the specified pipeline instance will be stopped and a new instance will be started using the downloaded model. The default value for `deploy` is `false`.
- When `deploy` is set to true, `pipeline_element_name` should be provided to update the newly downloaded model for this element in the pipeline. For example, if pipeline has `gvadetect name=detection`, `pipeline_element_name` would be `detection`.
- If a response from the model registry microservice is not received within 300 seconds by default, this request will time out and return a relevant response.
- If a response from the Model Download microservice is not received within 300 seconds by default, this request will time out and return a relevant response.


#### Path parameters
Expand Down Expand Up @@ -289,14 +289,14 @@ Accepted values: any
##### Example 1
Download model:

If the user is aware that there is only one model in model registry with a given name then user can provide only `name` in the request body to retrieve the model from the registry.
If the user is aware that there is only one model in Model Download with a given name then user can provide only `name` in the request body to retrieve the model from the registry.
```json
{
"name": "PalletDetection_YOLO_v1",
}
```

Other model properties can be provided to retrieve a specific model the model registry.
Other model properties can be provided to retrieve a specific model the Model Download.

```json
{
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -108,24 +108,24 @@ DL Streamer Pipeline Server microservice's configuration is defined via environm
- **HTTPS (Boolean)** - Make it `true` to enable SSL/TLS secure mode, mount the generated certificates
- Example: `HTTPS=true`
- Example: `HTTPS=false`
- **MTLS_VERIFICATION (Boolean)** - Enable/disable client certificate verification for mTLS Model Registry Microservice
- **MTLS_VERIFICATION (Boolean)** - Enable/disable client certificate verification for mTLS Model Download Microservice
- Example: `MTLS_VERIFICATION=true`
- Example: `MTLS_VERIFICATION=false`
- **MR_URL (String)** - Sets the URL where the model registry microservice is accessible
- **MR_URL (String)** - Sets the URL where the Model Download microservice is accessible
- Example: MR_URL=`http://10.100.10.100:32002`
- Example: MR_URL=`http://model-registry:32002`
- In order to connect to the model registry using its hostname, the DL Streamer Pipeline Server and model registry has to belong to the same shared network.
- If not set or left empty, the DL Streamer Pipeline Server will not be able to connect to the model registry successfully.
- **MR_SAVED_MODELS_DIR (String)** - Sets the directory path where the DL Streamer Pipeline Server stores models downloaded from the model registry microservice.
- To connect to the Model Download using its hostname, the DL Streamer Pipeline Server and Model Download has to belong to the same shared network.
- If not set or left empty, the DL Streamer Pipeline Server will not be able to connect to the Model Download successfully.
- **MR_SAVED_MODELS_DIR (String)** - Sets the directory path where the DL Streamer Pipeline Server stores models downloaded from the Model Download microservice.
- Example: `MR_SAVED_MODELS_DIR=./mr_models`
- The `.` (dot) refers to the current working directory inside the DL Streamer Pipeline Server container.
- For example, if the container's working directory is `/home/pipeline-server`, then `./mr_models` means `/home/pipeline-server/mr_models`.
- You can configure the volume mount for this directory in your respective `docker-compose.yml` file.
- If not set, it defaults to `./mr_models`.
- **MR_REQUEST_TIMEOUT (Integer)** - Sets the timeout for requests sent to the model registry microservice.
- **MR_REQUEST_TIMEOUT (Integer)** - Sets the timeout for requests sent to the Model Download microservice.
- Example: `MR_REQUEST_TIMEOUT=300`
- If not set, it defaults to `300`.
- **MR_VERIFY_CERT (String)** - Specifies how SSL certificate verification is handled when communicating with the model registry microservice.
- **MR_VERIFY_CERT (String)** - Specifies how SSL certificate verification is handled when communicating with the Model Download microservice.
- Example: `MR_VERIFY_CERT=/run/secrets/ModelRegistry_Server/ca-bundle.crt`
- Example: `MR_VERIFY_CERT=yes`
- This variable is only used if `MR_URL` contains `https`
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ ntplib==0.4.0
# OPCUA
asyncua==1.1.5

# Model Registry Client
# Model Download Client
pydantic==2.8.2

# Pipeline Server
Expand Down
2 changes: 1 addition & 1 deletion microservices/dlstreamer-pipeline-server/utils/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@
* The provided IP address is included in the OpenSSL configuration (\*.cnf) file for the client and server. It is used in the creation of their certificates. This allows for the client's certificate to be identified by a server with the IP address "192.168.1.100".

# int_mr_dir.sh
**Description**: This shell script makes the `mr_models` directory in the project's root directory, creates a new user based on the value of the `PIPELINE_SERVER_USER` environment variable, and changes the ownership of the `mr_models` directory to the new user. This directory is used as the location for model artifacts downloaded from the model registry microservice by DL Streamer Pipeline Server.
**Description**: This shell script makes the `mr_models` directory in the project's root directory, creates a new user based on the value of the `PIPELINE_SERVER_USER` environment variable, and changes the ownership of the `mr_models` directory to the new user. This directory is used as the location for model artifacts downloaded from the Model Download microservice by DL Streamer Pipeline Server.

**Usage**:
```bash
Expand Down
Loading