Merge branch 'main' into nathsudi/npu-profile-main

Author: nathsudi

.github/workflows/docs-reusable-workflow.yaml

@@ -2,9 +2,9 @@
 # SPDX-FileCopyrightText: (C) 2025 Intel Corporation
 # SPDX-License-Identifier: Apache-2.0
 
-name: 'Build Documentation'
+name: "Build Documentation"
 
-on:  # yamllint disable-line rule:truthy rule:line-length
+on: # yamllint disable-line rule:truthy rule:line-length
   workflow_call:
     inputs:
       docs_directory:
@@ -33,14 +33,13 @@ permissions:
 jobs:
   build-documentation:
     permissions:
-      contents: read        # minimal privilege required
+      contents: read # minimal privilege required
     runs-on: ubuntu-latest
     env:
       DOCS_DIR: ${{ inputs.docs_directory }}
     steps:
-
       - name: Checkout code
-        uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8  # v5.0.0
+        uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
         with:
           # Fetch all history, otherwise sporadic issue with missing tags
           fetch-depth: 0

.github/workflows/documentation-check.yaml

@@ -1,7 +1,7 @@
 
 # Edge-AI-Suites Contributor Guide
 
-The following are guidelines for contributing to the Edge AI Suites project, including the code of conduct, submitting issues, and contributing code.  
+The following are guidelines for contributing to the Edge AI Suites project, including the code of conduct, submitting issues, and contributing code.
 
 # Table of Contents
 
@@ -15,37 +15,44 @@ The following are guidelines for contributing to the Edge AI Suites project, inc
 
 # Code of Conduct
 
-This project and everyone participating in it are governed by the [`CODE_OF_CONDUCT`](CODE_OF_CONDUCT.md) document. By participating, you are expected to adhere to this code.  
+This project and everyone participating in it are governed by the [`CODE_OF_CONDUCT`](CODE_OF_CONDUCT.md) document. By participating, you are expected to adhere to this code.
 
-# Security 
+# Security
 
-Read the [`Security Policy`](SECURITY.md).  
+Read the [`Security Policy`](SECURITY.md).
 
 # Get Started
 
-Clone the repository and follow the [`README`](README.md) to get started with the sample applications of interest.  
+Clone the repository and follow the [`README`](README.md) or
+[documentation's](https://docs.openedgeplatform.intel.com/) "Get Started" of the chosen
+applications.
 
 ```
     git clone https://github.com/open-edge-platform/edge-ai-suites.git
     cd edge-ai-suites
 ```
+Note that you do not need to clone the entire repository. You can clone just the portion you
+are interested with. To see how ti do it, check out the
+[Contributing to Open Edge Platform](https://docs.openedgeplatform.intel.com/canonical/OEP-articles/contribution-guide.html#repository-cloning-partial-cloning)
+article.
+
 
 # How to Contribute
 
 ## Contribute Code Changes
 
-> If you want to help improve Edge AI Suites, choose one of the issues reported in [`GitHub Issues`](https://github.com/open-edge-platform/edge-ai-suites/issues) and create a [`Pull Request`](https://github.com/open-edge-platform/edge-ai-suites/pulls) to address it.  
+> If you want to help improve Edge AI Suites, choose one of the issues reported in [`GitHub Issues`](https://github.com/open-edge-platform/edge-ai-suites/issues) and create a [`Pull Request`](https://github.com/open-edge-platform/edge-ai-suites/pulls) to address it.
 > Note: Please check that the change hasn't been implemented before you start working on it.
 
 ## Improve Documentation
 
 The easiest way to help with the `Developer Guide` and `User Guide` is to review it and provide feedback on the
-existing articles. Whether you notice a mistake, see the possibility of improving the text, or think more 
+existing articles. Whether you notice a mistake, see the possibility of improving the text, or think more
 information should be added, you can reach out to discuss the potential changes.
 
 ## Report Bugs
 
-If you encounter a bug, open an issue in [`Github Issues`](https://github.com/open-edge-platform/edge-ai-suites/issues). Provide the following information to help us 
+If you encounter a bug, open an issue in [`Github Issues`](https://github.com/open-edge-platform/edge-ai-suites/issues). Provide the following information to help us
 understand and resolve the issue quickly:
 
 - A clear and descriptive title
@@ -59,7 +66,7 @@ understand and resolve the issue quickly:
 
 Intel welcomes suggestions for new features and improvements. Follow these steps to make a suggestion:
 
-- Check if there's already a similar suggestion in [`Github Issues`](https://github.com/open-edge-platform/edge-ai-suites/issues).  
+- Check if there's already a similar suggestion in [`Github Issues`](https://github.com/open-edge-platform/edge-ai-suites/issues).
 - If not, open a new issue and provide the following information:
    - A clear and descriptive title
    - A detailed description of the enhancement
@@ -75,7 +82,7 @@ Before submitting a pull request, ensure you follow these guidelines:
 - Test your changes thoroughly.
 - Document your changes (in code, readme, etc.).
 - Submit your pull request, detailing the changes and linking to any relevant issues.
-- Wait for a review. Intel will review your pull request as soon as possible and provide you with feedback. 
+- Wait for a review. Intel will review your pull request as soon as possible and provide you with feedback.
 You can expect a merge once your changes are validated with automatic tests and approved by maintainers.
 
 # Development Guidelines
@@ -95,7 +102,7 @@ Clear and informative commit messages make it easier to understand the history o
 - Capitalize the first letter
 - Keep the message concise, ideally under 50 characters
 
-Please fill in the details as per the [pull request template](./.github/PULL_REQUEST_TEMPLATE.md) while submitting the 
+Please fill in the details as per the [pull request template](./.github/PULL_REQUEST_TEMPLATE.md) while submitting the
 pull request.
 
 ## Testing

CONTRIBUTING.md

@@ -187,8 +187,6 @@ def process(self, input_generator):
                 if os.path.exists(chunk_path) and DELETE_CHUNK_AFTER_USE:
                     os.remove(chunk_path)
 
-                StorageManager.save_async(transcript_path, transcribed_text, append=True)
-
                 yield {
                     **chunk_data,
                     "text": transcribed_text,
@@ -208,8 +206,9 @@ def process(self, input_generator):
                 teacher_speaker = max(self.speaker_text_len, key=self.speaker_text_len.get)
 
             if teacher_speaker:
-                teacher_lines_with_time = []
+                teacher_lines = []
                 full_updated_lines = []
+                full_timestamped_lines = []
 
                 for seg in self.all_segments:
                     spk = seg["speaker"]
@@ -219,8 +218,8 @@ def process(self, input_generator):
 
                     if spk == teacher_speaker:
                         speaker_label = LABEL_TEACHER
-                        teacher_lines_with_time.append(
-                            f"[{start} - {end}] {speaker_label}: {text}"
+                        teacher_lines.append(
+                            f"{text}"
                         )
                     else:
                         if spk.startswith(f"{LABEL_SPEAKER}_"):
@@ -233,7 +232,11 @@ def process(self, input_generator):
                             speaker_label = spk
 
                     full_updated_lines.append(
-                        f"[{start} - {end}] {speaker_label}: {text}"
+                        f"{speaker_label}: {text}"
+                    )
+
+                    full_timestamped_lines.append(
+                        f"[{start} - {end}]: {text}"
                     )
 
                 StorageManager.save(
@@ -242,9 +245,15 @@ def process(self, input_generator):
                     append=False
                 )
 
+                StorageManager.save(
+                    os.path.join(project_path, "content_segmentation_transcription.txt"),
+                    "\n".join(full_timestamped_lines) + "\n",
+                    append=False
+                )
+
                 StorageManager.save(
                     os.path.join(project_path, "teacher_transcription.txt"),
-                    "\n".join(teacher_lines_with_time) + "\n",
+                    "\n".join(teacher_lines) + "\n",
                     append=False
                 )
 
@@ -269,4 +278,4 @@ def process(self, input_generator):
                 }
             )
 
-            logger.info(f"Transcription Complete: {self.session_id}")
+            logger.info(f"Transcription Complete: {self.session_id}")

education-ai-suite/smart-classroom/components/asr_component.py

@@ -1,99 +1,135 @@
 from components.llm.base_summarizer import BaseSummarizer
-import logging
+import logging, threading, gc
 from transformers import AutoTokenizer, TextIteratorStreamer
 from optimum.intel.openvino import OVModelForCausalLM
 from utils import ensure_model
 from utils.config_loader import config
 from utils.locks import audio_pipeline_lock
-import threading
 
 logger = logging.getLogger(__name__)
 
 
 class Summarizer(BaseSummarizer):
     def __init__(self, model_name, device, temperature=0.7, revision=None):
         self.model_name = model_name
-        self.device = device.upper()  # OpenVINO uses "GPU" or "CPU"
+        self.device = device.upper()
         self.temperature = temperature
 
-        model_path = ensure_model.get_model_path()
-        logger.info(f"Loading Model: model name={self.model_name}, model path={model_path}, device={self.device}")
-        
+        self.model_path = ensure_model.get_model_path()
+
+        logger.info(
+            f"Summarizer initialized (lazy load). "
+            f"model={self.model_name}, path={self.model_path}, device={self.device}"
+        )
+
         self.tokenizer = AutoTokenizer.from_pretrained(
-            model_path, 
-            trust_remote_code=True, 
-            fix_mistral_regex=True
+            self.model_path,
+            trust_remote_code=True,
+            fix_mistral_regex=True,
         )
 
         if self.tokenizer.pad_token is None:
             self.tokenizer.pad_token = self.tokenizer.eos_token
-        
-        self.model = OVModelForCausalLM.from_pretrained(
-            model_path, 
-            device=self.device, 
-            use_cache=True
+
+    def _load_model(self):
+        logger.info("Loading OVModelForCausalLM instance...")
+        return OVModelForCausalLM.from_pretrained(
+            self.model_path,
+            device=self.device,
+            use_cache=True,
         )
 
+    def _destroy_model(self, model):
+        try:
+            del model
+            gc.collect()
+            logger.info("OV model instance destroyed")
+        except Exception as e:
+            logger.warning(f"Failed to destroy OV model cleanly: {e}")
+
     def generate(self, prompt: str, stream: bool = True):
         max_new_tokens = config.models.summarizer.max_new_tokens
         inputs = self.tokenizer(prompt, return_tensors="pt")
 
         if stream:
             class CountingTextIteratorStreamer(TextIteratorStreamer):
-                    def __init__(self, tokenizer, skip_special_tokens=True, skip_prompt=True):
-                        super().__init__(tokenizer, skip_special_tokens=skip_special_tokens, skip_prompt=skip_prompt)
-                        self.total_tokens = 0
+                def __init__(self, tokenizer, skip_special_tokens=True, skip_prompt=True):
+                    super().__init__(
+                        tokenizer,
+                        skip_special_tokens=skip_special_tokens,
+                        skip_prompt=skip_prompt,
+                    )
+                    self.total_tokens = 0
 
-                    def put(self, value):
-                        if value is not None:
-                            self.total_tokens += 1
-                        super().put(value)
+                def put(self, value):
+                    if value is not None:
+                        self.total_tokens += 1
+                    super().put(value)
 
             streamer = CountingTextIteratorStreamer(
-                self.tokenizer, 
-                skip_special_tokens=True, 
-                skip_prompt=True
+                self.tokenizer,
+                skip_special_tokens=True,
+                skip_prompt=True,
             )
-                
+
             def run_generation():
-                with audio_pipeline_lock:
-                    generation_kwargs = {
-                        "input_ids": inputs.input_ids,
-                        "max_new_tokens": max_new_tokens,
-
-                        # 🔑 sampling safety
-                        "do_sample": True,
-                        "temperature": max(self.temperature, 0.1), 
-                        "top_p": 0.9,
-                        "top_k": 50,  
-
-                        # tokens
-                        "pad_token_id": self.tokenizer.eos_token_id,
-                        "eos_token_id": self.tokenizer.eos_token_id,
-
-                        # streaming
-                        "streamer": streamer,
-                    }
-                    self.model.generate(**generation_kwargs)
-            
-            thread = threading.Thread(target=run_generation, daemon=True)
-            thread.start()
-            
+                model = None
+                try:
+                    with audio_pipeline_lock:
+                        model = self._load_model()
+                        model.generate(
+                            input_ids=inputs.input_ids,
+                            max_new_tokens=max_new_tokens,
+
+                            # sampling
+                            do_sample=True,
+                            temperature=max(self.temperature, 0.1),
+                            top_p=0.9,
+                            top_k=50,
+
+                            # tokens
+                            pad_token_id=self.tokenizer.eos_token_id,
+                            eos_token_id=self.tokenizer.eos_token_id,
+
+                            # streaming
+                            streamer=streamer,
+                        )
+
+                except Exception:
+                    logger.error(
+                        "Exception occurred in OV streaming generation",
+                        exc_info=True,
+                    )
+                    if hasattr(streamer, "_queue"):
+                        streamer._queue.put(
+                            "[ERROR]: Summary generation failed due to resource constraints."
+                        )
+
+                finally:
+                    if model is not None:
+                        self._destroy_model(model)
+                    streamer.end()
+
+            threading.Thread(target=run_generation, daemon=True).start()
             return streamer
+
         else:
-            with audio_pipeline_lock:
-                generation_kwargs = {
-                    "input_ids": inputs.input_ids,
-                    "max_new_tokens": max_new_tokens,
-
-                    # 🔑 sampling safety
-                    "do_sample": True,
-                    "temperature": max(self.temperature, 0.1), 
-                    "top_p": 0.9,
-                    "top_k": 50,     
-
-                    # tokens
-                    "pad_token_id": self.tokenizer.eos_token_id,
-                    "eos_token_id": self.tokenizer.eos_token_id,
-                }
-            return self.model.generate(**generation_kwargs)
+            model = None
+            try:
+                with audio_pipeline_lock:
+                    model = self._load_model()
+                    return model.generate(
+                        input_ids=inputs.input_ids,
+                        max_new_tokens=max_new_tokens,
+
+                        do_sample=True,
+                        temperature=max(self.temperature, 0.1),
+                        top_p=0.9,
+                        top_k=50,
+
+                        pad_token_id=self.tokenizer.eos_token_id,
+                        eos_token_id=self.tokenizer.eos_token_id,
+                    )
+            finally:
+                if model is not None:
+                    self._destroy_model(model)