Merge branch 'open-edge-platform:main' into main

Author: unarayan

.github/workflows/[HLS]-Multi-Modal-Patient-Monitoring.yaml

@@ -32,10 +32,10 @@ The main features are as follows:
 :::{toctree}
 :hidden:
 
-get-started
-how-it-works
-application-flow
-release-notes
+./get-started
+./how-it-works
+./application-flow
+Release Notes <./release-notes>
 
 :::
 hide_directive-->

education-ai-suite/smart-classroom/docs/user-guide/index.md

@@ -1,25 +1,51 @@
-# Release Notes
+# Release Notes: Smart Classroom
 
-## Release EAS-2025.2.2
+## Version 2026.0
+
+**April 1, 2026**
+
+The Smart Classroom application now offers a series after-class summary enhancements in the form of next‑generation real-time audio and visual analytics, giving teachers and schools a better understanding of classroom dynamics through AI‑driven summaries and engagement metrics.
+
+The Education AI Suite now also includes built-in telemetry hooks and benchmarking.
 
 **New**
 
-- Video Pipeline Integration:
-  - Real-time video processing for classroom monitoring.
-  - Bounding boxes and pose estimation for teachers and students.
-- **Room View Tab**: Displays live classroom video with captions and engagement metrics.
-- **Board Captures Tab**: Chronological snapshots of smart board content.
-- **Mind Map Tab**: Keyword-indexed graph of class content for visual learning.
-- **Class Engagement Panel**: Video-based engagement scoring for students
+- **Speaker Diarization** (via the Audio Pipeline):
+  - identifies teacher and student speakers using NPU-accelerated diarization
+  - generates an interactive audio timeline for replay and analysis
+  - enables time-coded navigation within class video recordings
 
-## Release EAS-2025.2.0
+- **Class Engagement Metrics – Audio**:
+  - measure teacher and student speech duration
+  - track questions asked and answered
+  - track student-teacher interaction frequency
 
-This is the first release of Intel® Education AI Suite EAS-2025.2.
+- **Class Engagement Metrics – Video**:
+  - track student hand raises
+  - track posture changes (stand up/sit down)
+  - track teacher movement
 
-**New**
+- **Built‑in telemetry** to measure classroom workloads across Intel platforms (CPU core utilization, iGPU load, NPU load, memory usage, workload-specific performance counters)
+
+- **Benchmarking scripts** to reproduce Intel internal performance measurements, and validate XPU performance
+
+**Improved**
 
-In this release, the **Smart Classroom** application is added. It is an extensible framework for processing, analyzing, and summarizing classroom sessions using advanced multimodal AI.
+- **Knowledge Graph UI** readability and formatting, and increased clarity when visualizing topic relationships
 
 ## Documentation and Source Code
 
 - [GitHub](https://github.com/open-edge-platform/edge-ai-suites/tree/main/education-ai-suite)
+
+## Previous releases
+
+- [Release notes 2025](./release-notes/release-notes-2025.md)
+
+<!--hide_directive
+:::{toctree}
+:hidden:
+
+Release Notes 2025 <./release-notes/release-notes-2025.md>
+
+:::
+hide_directive-->

education-ai-suite/smart-classroom/docs/user-guide/release-notes.md

@@ -0,0 +1,25 @@
+# Release Notes: Smart Classroom 2025
+
+## Release EAS-2025.2.2
+
+**New**
+
+- Video Pipeline Integration:
+  - Real-time video processing for classroom monitoring.
+  - Bounding boxes and pose estimation for teachers and students.
+- **Room View Tab**: Displays live classroom video with captions and engagement metrics.
+- **Board Captures Tab**: Chronological snapshots of smart board content.
+- **Mind Map Tab**: Keyword-indexed graph of class content for visual learning.
+- **Class Engagement Panel**: Video-based engagement scoring for students
+
+## Release EAS-2025.2.0
+
+This is the first release of Intel® Education AI Suite EAS-2025.2.
+
+**New**
+
+In this release, the **Smart Classroom** application is added. It is an extensible framework for processing, analyzing, and summarizing classroom sessions using advanced multimodal AI.
+
+## Documentation and Source Code
+
+- [GitHub](https://github.com/open-edge-platform/edge-ai-suites/tree/main/education-ai-suite)

education-ai-suite/smart-classroom/docs/user-guide/release-notes/release-notes-2025.md

@@ -38,7 +38,7 @@ services:
     environment:
       - WORKLOAD_TYPE=mdpnp
       - GRPC_PORT=50051
-      - HTTP_PORT=8000    
+      - HTTP_PORT=8001
     network_mode: "host"
 
   metrics-collector:
@@ -99,7 +99,7 @@ services:
     environment:
       - GRPC_HOST=localhost
       - GRPC_PORT=50051
-      - MDPNP_DEVICE=${MDPNP_DEVICE}
+      - MDPNP_DEVICE=${MDPNP_DEVICE:-}
     env_file:
         - ./configs/device.env   
     depends_on:
@@ -127,7 +127,7 @@ services:
       patient-monitoring-aggregator:
         condition: service_started
     environment:
-      - ECG_DEVICE=${ECG_DEVICE}  
+      - ECG_DEVICE=${ECG_DEVICE:-}  
     env_file:
       - ./configs/device.env  
     volumes:
@@ -191,7 +191,7 @@ services:
       patient-monitoring-aggregator:
           condition: service_started  
     environment:
-      - RPPG_DEVICE=${RPPG_DEVICE}  
+      - RPPG_DEVICE=${RPPG_DEVICE:-}  
     env_file:
       - ./configs/device.env  
     volumes:
@@ -207,6 +207,8 @@ services:
       context: ./services/ui
       dockerfile: Dockerfile
       args:
+        VITE_API_BASE_URL: http://${HOST_IP:-localhost}:8001
+        VITE_POSE_STREAM_URL: http://${HOST_IP:-localhost}:8085/video_feed
         HTTP_PROXY: ${HTTP_PROXY}
         HTTPS_PROXY: ${HTTPS_PROXY}
         NO_PROXY: ${NO_PROXY}

health-and-life-sciences-ai-suite/multi_modal_patient_monitoring/docker-compose.yaml

@@ -1,29 +1,78 @@
-# Health AI Suite – Helm Deployment
+# Multi modal patient monitoring – Helm Deployment
 
-This Helm chart deploys the **Health & Life Sciences AI Suite** on Kubernetes.
+This Helm chart deploys the **Multi modal patient monitoring app** on Kubernetes.
 
 
 ## Prerequisites
 
 - Kubernetes cluster (Minikube / Kind / Bare-metal)
 - `kubectl`
 - `helm` (v3+)
-- Docker images built locally
+- A working PersistentVolume provisioner (required for PVC binding)
 
-## Required Docker Images
+### Storage prerequisite (required)
 
-The following images **must exist locally** before deploying the Helm chart.
+This chart creates PVCs (`models-pvc`, `videos-pvc`, `health-ai-assets-pvc`) and expects your
+cluster to provide PersistentVolumes through a StorageClass.
+
+If your cluster has no dynamic provisioner, PVCs will remain `Pending` and workloads will not
+schedule.
+
+- For single-node/local clusters, install a dynamic provisioner (for example,
+  `local-path-provisioner`) before installing this chart.
+- Or pre-create matching static PersistentVolumes for all claims.
+
+> `local-path-provisioner` does **not** support `ReadWriteMany`.
+> Use `ReadWriteOnce` (this chart default) unless you use a RWX-capable storage backend.
+
+## Optional: Proxy configuration 
+
+Configure Proxy Settings (If behind a proxy)
+
+If you are deploying in a proxy environment, also update the proxy settings in the same values.yaml file:
+```bash
+http_proxy: "http://your-proxy-server:port"
+https_proxy: "http://your-proxy-server:port"
+no_proxy: "localhost,127.0.0.1,.local,.cluster.local"
+```
+Replace your-proxy-server:port with your actual proxy server details.
+ 
+
+Set via CLI if needed:
+
+```bash
+--set assets.proxy.enabled=true \
+--set assets.proxy.httpProxy=http://your-proxy-server:port\
+--set assets.proxy.httpsProxy=http://your-proxy-server:port\
+--set assets.proxy.noProxy=localhost,127.0.0.1,.svc,.cluster.local
+```
+
+## Setup Storage Provisioner (For Single-Node Clusters)
+Check if your cluster has a default storage class with dynamic provisioning. If not, install a storage provisioner:
 
-Check available images:
 ```bash
-docker images | grep intel/hl-ai
+# Check for existing storage classes
+kubectl get storageclass
+
+# If no storage classes exist or none are marked as default, install local-path-provisioner
+# This step is typically needed for single-node bare Kubernetes installations
+# (Managed clusters like EKS/GKE/AKS already have storage classes configured)
+
+# Install local-path-provisioner for automatic storage provisioning
+kubectl apply -f https://raw.githubusercontent.com/rancher/local-path-provisioner/master/deploy/local-path-storage.yaml
+
+# Set it as default storage class
+kubectl patch storageclass local-path -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'
+
+# Verify storage class is ready
+kubectl get storageclass
 ```
 
 
 ## Install
 
 ```bash
-cd health-and-life-sciences-ai-suite/helm/multi_modal_patient_monitoring
+cd health-and-life-sciences-ai-suite/multi_modal_patient_monitoring/helm/multi_modal_patient_monitoring
 
 helm install multi-modal-patient-monitoring . \
   --namespace multi-modal-patient-monitoring \
@@ -53,7 +102,7 @@ kubectl get svc -n multi-modal-patient-monitoring
 
 ## Check Logs (recommended)
 ```bash
-kubectl logs -n multi-modal-patient-monitoringi deploy/mdpnp
+kubectl logs -n multi-modal-patient-monitoring deploy/mdpnp
 kubectl logs -n multi-modal-patient-monitoring deploy/dds-bridge
 kubectl logs -n multi-modal-patient-monitoring deploy/aggregator
 kubectl logs -n multi-modal-patient-monitoring deploy/ai-ecg
@@ -70,7 +119,9 @@ Healthy services will show:
 
 
 ## Access the Frontend UI
-Check Ingress resource:
+### 1. Check the Ingress Resource
+
+Run the following command to view the ingress configuration:
 
 ```bash
 kubectl get ingress -n multi-modal-patient-monitoring
@@ -82,24 +133,42 @@ Example output:
 NAME       HOSTS               PATHS   ADDRESS         PORTS
 multi-modal-patient-monitoring  multi-modal-patient-monitoring.local       /       xx.xx.xx.xx   80
 ```
+### 2. If an IP Address Appears in ADDRESS
 
-Add an entry on your Linux host (replace <IP> with the one you found):
+Add the hostname mapping to your local machine:
 ```bash
 echo "<IP> multi-modal-patient-monitoring.local" | sudo tee -a /etc/hosts
 ```
+Replace <IP> with the value shown in the ADDRESS column.
+
+### 3. If the ADDRESS Field is Empty (Common in Minikube)
+Some local Kubernetes environments (such as Minikube) do not automatically populate the ingress IP.
+
+Retrieve the Minikube cluster IP:
+```bash
+minikube ip
+```
+Then map the hostname to the IP:
+```bash
+echo "$(minikube ip) multi-modal-patient-monitoring.local" | sudo tee -a /etc/hosts
+```
 
-Open your browser and go to:
+### 4. Enable Ingress in Minikube (if not already enabled)
+```bash
+minikube addons enable ingress
+```
+Wait a few moments for the ingress controller to start.
+
+### 5.Open the Application
+Open your browser and navigate to:
 ```bash
 http://<host-or-ip>/
 ``` 
 Example:
 ```bash
 http://multi-modal-patient-monitoring.local/
 ``` 
-If using Minikube, you may need to enable the ingress addon:
-```bash
-minikube addons enable ingress
-``` 
+
 This will open the Health AI Suite frontend dashboard.
 
 From here you can access:

health-and-life-sciences-ai-suite/multi_modal_patient_monitoring/helm/README.md

@@ -18,6 +18,8 @@ spec:
           imagePullPolicy: {{ .Values.aggregator.image.pullPolicy }}
           ports:
             - containerPort: 8001
+              hostPort: 8001
+              protocol: TCP
             - containerPort: 50051
           env:
             - name: METRICS_SERVICE_URL

health-and-life-sciences-ai-suite/multi_modal_patient_monitoring/helm/multi_modal_patient_monitoring/templates/aggregator-deployment.yaml

@@ -13,16 +13,19 @@ spec:
         app: ai-ecg
     spec:
       initContainers:
-        - name: copy-models
-          image: "{{ .Values.assets.image.repository }}/{{ .Values.assets.image.name }}:{{ .Values.assets.image.tag }}"
-          imagePullPolicy: {{ .Values.assets.image.pullPolicy }}
-          command: ["/bin/sh", "-c"]
-          args:
-            - cp -r /opt/assets/models/* /models/
+        - name: wait-for-ecg-model
+          image: busybox:1.36
+          command:
+            - sh
+            - -c
+            - >-
+              until [ -f /models/ai-ecg/hubert_ecg_small_fp16.xml ]; do
+                echo "Waiting for HuBERT-ECG model asset...";
+                sleep 5;
+              done
           volumeMounts:
             - name: models
               mountPath: /models
-
       containers:
         - name: ai-ecg
           image: "{{ .Values.aiEcg.image.repository }}/{{ .Values.aiEcg.image.name }}:{{ .Values.aiEcg.image.tag }}"
@@ -32,22 +35,46 @@ spec:
           envFrom:
             - configMapRef:
                 name: device-config
+          startupProbe:
+            tcpSocket:
+              port: 8000
+            periodSeconds: 5
+            failureThreshold: 24
+          readinessProbe:
+            httpGet:
+              path: /health
+              port: 8000
+            initialDelaySeconds: 5
+            periodSeconds: 5
+            failureThreshold: 6
+          livenessProbe:
+            httpGet:
+              path: /health
+              port: 8000
+            initialDelaySeconds: 15
+            periodSeconds: 15
+            failureThreshold: 3
           ports:
             - containerPort: 8000
           volumeMounts:
             - name: models
               mountPath: /models
+            - name: videos
+              mountPath: /videos
             - name: dri
               mountPath: /dev/dri
             - name: dev
               mountPath: /dev
-
       volumes:
         - name: models
-          emptyDir: {}
+          persistentVolumeClaim:
+            claimName: models-pvc
+        - name: videos
+          persistentVolumeClaim:
+            claimName: videos-pvc
         - name: dri
           hostPath:
             path: /dev/dri
         - name: dev
           hostPath:
-            path: /dev
+            path: /dev