Video Search and Summary Sample Application and microservices (#349)

Signed-off-by: Krishna <krishna.murti@intel.com>
Signed-off-by: Krishna Murti <krishna.murti@intel.com>
Signed-off-by: Vellaisamy, Sathyendran <sathyendran.vellaisamy@intel.com>
Signed-off-by: Pooja Kumbharkar <pooja.kumbharkar@intel.com>
Signed-off-by: B, Vinod K <vinod.k.b@intel.com>
Signed-off-by: Sudarshana Panda <sudarshana.panda@intel.com>
Signed-off-by: Vinod K B <vinod.k.b@intel.com>
Signed-off-by: Yeoh, Hoong Tee <hoong.tee.yeoh@intel.com>
Signed-off-by: dmichalo <dawid.michalowski@intel.com>
Co-authored-by: bhardwaj-nakul <nakul.bhardwaj@intel.com>
Co-authored-by: Krishna Murti <krishna.murti@intel.com>
Co-authored-by: AnshulWagadre <107472813+anshul-wagadre@users.noreply.github.com>
Co-authored-by: Wagadre, Anshul <anshul.wagadre@intel.com>
Co-authored-by: Yogesh <yogeshpandey@intel.com>
Co-authored-by: Raghavendra Bhat <raghavendra.bhat@intel.com>
Co-authored-by: Tomasz Janczak <Tomasz.Janczak@intel.com>
Co-authored-by: msmiatac <153737147+msmiatac@users.noreply.github.com>
Co-authored-by: nszczygl9 <118973656+nszczygl9@users.noreply.github.com>
Co-authored-by: Vinod Kumar B <vinod.k.b@intel.com>
Co-authored-by: Vellaisamy, Sathyendran <sathyendran.vellaisamy@intel.com>
Co-authored-by: Pooja Kumbharkar <pooja.kumbharkar@intel.com>
Co-authored-by: sathyendranv <84972945+sathyendranv@users.noreply.github.com>
Co-authored-by: SudarshanaPanda <sudarshana.panda@intel.com>
Co-authored-by: Basak Caprak Senzeybek <basak.caprak@intel.com>
Co-authored-by: Nicolas Oliver <dario.n.oliver@intel.com>
Co-authored-by: Hoong Tee, Yeoh <hoong.tee.yeoh@intel.com>
Co-authored-by: ganesanintel <ganesan.v@intel.com>
Co-authored-by: Tomasz Bujewski <tomasz.bujewski@intel.com>
Co-authored-by: Dawid Michalowski <dawid.michalowski@intel.com>
Co-authored-by: Michal Holownia <michal.holownia@intel.com>
Co-authored-by: oommensy <steffy.a.oommen@intel.com>
Co-authored-by: saikiransayabugari <saikiran.sayabugari@intel.com>
Co-authored-by: Elroy Ashtian, Jr. <elroy.ashtian@intel.com>
Co-authored-by: marcin-wadolkowski <106673332+marcin-wadolkowski@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Author: 25 people

README.md

@@ -21,19 +21,22 @@ Key components of the **Edge AI Libraries**:
 | [OpenVINO™ toolkit](https://github.com/openvinotoolkit/openvino) | Library | [Link](https://docs.openvino.ai/2025/index.html) | [API Reference](https://docs.openvino.ai/2025/api/api_reference.html) |
 | [OpenVINO™ Training Extensions](https://github.com/open-edge-platform/training_extensions) | Library | [Link](https://github.com/open-edge-platform/training_extensions?tab=readme-ov-file#introduction) | [API Reference](https://github.com/open-edge-platform/training_extensions?tab=readme-ov-file#quick-start) |
 | [OpenVINO™ Model API](https://github.com/open-edge-platform/model_api) | Library | [Link](https://github.com/open-edge-platform/model_api?tab=readme-ov-file#installation) | [API Reference](https://github.com/open-edge-platform/model_api?tab=readme-ov-file#usage) |
+| [Audio Intelligence](microservices/audio-intelligence) | Microservice | [Link](microservices/audio-intelligence/docs/user-guide/get-started.md) | [API Reference](microservices/audio-intelligence/docs/user-guide/api-reference.md) |
 | [Deep Learning Streamer Pipeline Server](microservices/dlstreamer-pipeline-server) | Microservice | [Link](microservices/dlstreamer-pipeline-server#quick-try-out) | [API Reference](microservices/dlstreamer-pipeline-server/docs/user-guide/api-docs/pipeline-server.yaml) |
 | [Document Ingestion](microservices/document-ingestion) | Microservice | [Link](microservices/document-ingestion/pgvector/docs/get-started.md) | [API Reference](microservices/document-ingestion/pgvector/docs/dataprep-api.yml) |
 | [Model Registry](microservices/model-registry) | Microservice | [Link](microservices/model-registry/docs/user-guide/get-started.md) | [API Reference](microservices/model-registry/docs/user-guide/api-docs/openapi.yaml) |
+| [Multimodal Embedding Serving](microservices/multimodal-embedding-serving) | Microservice | [Link](microservices/multimodal-embedding-serving/docs/user-guide/get-started.md) | [API Reference](microservices/multimodal-embedding-serving/docs/user-guide/api-docs/openapi.yaml) |
 | [Time Series Analytics Microservice](microservices/time-series-analytics) | Microservice |  [Link](microservices/time-series-analytics/docs/user-guide/Overview.md) | [Usage](microservices/time-series-analytics/docs/user-guide/get-started.md) |
 | [Vector Retriever (with Milvus)](microservices/vector-retriever/milvus/) | Microservice | [Link](microservices/vector-retriever/milvus/docs/user-guide/get-started.md) | [API Reference](microservices/vector-retriever/milvus/docs/user-guide/api-reference.md) |
 | [Visual-Data Preparation for Retrieval (with Milvus)](microservices/visual-data-preparation-for-retrieval/milvus/) | Microservice | [Link](microservices/visual-data-preparation-for-retrieval/milvus/docs/user-guide/get-started.md) | [API Reference](microservices/visual-data-preparation-for-retrieval/milvus/docs/user-guide/api-reference.md) |
+| [Visual-Data Preparation for Retrieval (with VDMS)](microservices/visual-data-preparation-for-retrieval/vdms/) | Microservice | [Link](microservices/visual-data-preparation-for-retrieval/vdms/docs/user-guide/get-started.md) | [API Reference](microservices/visual-data-preparation-for-retrieval/vdms/docs/user-guide/api-reference.md) |
 | [VLM Inference Serving](microservices/vlm-openvino-serving) | Microservice |  [Link](microservices/vlm-openvino-serving/README.md) | [Usage](microservices/vlm-openvino-serving/README.md) |
 | [Intel® Geti™](https://github.com/open-edge-platform/geti)[`*`](#license) | Tool | [Link](https://geti.intel.com/) | [Docs](https://docs.geti.intel.com) |
 | [Intel® SceneScape](https://github.com/open-edge-platform/scenescape)[`*`](#license) | Tool | [Link](https://docs.openedgeplatform.intel.com/scenescape/main/user-guide/Getting-Started-Guide.html) | [Docs](https://docs.openedgeplatform.intel.com/scenescape/main/toc.html) |
 | [Visual Pipeline and Platform Evaluation Tool](tools/visual-pipeline-and-platform-evaluation-tool) | Tool | [Link](tools/visual-pipeline-and-platform-evaluation-tool/docs/user-guide/get-started.md) | [Build](tools/visual-pipeline-and-platform-evaluation-tool/docs/user-guide/how-to-build-source.md) instructions |
 | [Chat Question and Answer](sample-applications/chat-question-and-answer) | Sample Application |  [Link](sample-applications/chat-question-and-answer/docs/user-guide/get-started.md) | [Build](sample-applications/chat-question-and-answer/docs/user-guide/build-from-source.md) instructions |
 | [Chat Question and Answer Core](sample-applications/chat-question-and-answer-core) | Sample Application | [Link](sample-applications/chat-question-and-answer-core/docs/user-guide/get-started.md) | [Build](sample-applications/chat-question-and-answer-core/docs/user-guide/build-from-source.md) instructions |
-
+| [Video Search and Summarization](sample-applications/video-search-and-summarization) | Sample Application | [Link](sample-applications/video-search-and-summarization/docs/user-guide/get-started.md) | [Build](sample-applications/video-search-and-summarization/docs/user-guide/build-from-source.md) instructions |
 
 > Intel, the Intel logo, OpenVINO, and the OpenVINO logo are trademarks of Intel Corporation or its subsidiaries.
 

microservices/audio-intelligence/.dockerignore

@@ -0,0 +1,61 @@
+# Python bytecode and cache
+__pycache__/
+*.py[cod]
+*$py.class
+.pytest_cache/
+.coverage
+htmlcov/
+
+# Poetry and Python virtual environments
+.venv/
+venv/
+ENV/
+.python-version
+
+# Development and editor files
+.git/
+.github/
+.gitignore
+.idea/
+.vscode/
+*.swp
+*.swo
+
+# Docker files (no need to copy these)
+docker/
+.dockerignore
+
+# Documentation
+docs/
+*.md
+!README.md
+
+# Testing
+tests/
+test_*.py
+*_test.py
+
+# Local development artifacts
+data/
+models/
+logs/
+.env
+.env.*
+wget-log*
+
+# Build artifacts
+*.so
+*.dylib
+*.dll
+dist/
+build/
+*.egg-info/
+
+# Temporary files
+tmp/
+temp/
+*.tmp
+*.bak
+setup_docker.sh
+setup.sh
+setup_host.sh

microservices/audio-intelligence/README.md

@@ -0,0 +1,26 @@
+# Audio Intelligence Microservice
+
+This repository provides a FastAPI-based microservice for audio intelligence including speech transcription from video files using pywhispercpp or OpenVINO with openvino-genai.
+
+Below, you'll find links to detailed documentation to help you get started, configure, and deploy the microservice.
+
+## Documentation
+
+- **Overview**
+  - [Overview](docs/user-guide/Overview.md): A high-level introduction to the microservice.
+  - [Overview Architecture](docs/user-guide/overview-architecture.md): Detailed architecture.
+
+- **Getting Started**
+  - [Get Started](docs/user-guide/get-started.md): Step-by-step guide to getting started with the microservice.
+  - [System Requirements](docs/user-guide/system-requirements.md): Hardware and software requirements for running the microservice.
+
+- **Deployment**
+  - [How to Build from Source](docs/user-guide/how-to-build-from-source.md): Instructions for building the microservice from source code.
+
+- **API Reference**
+  - [API Reference](docs/user-guide/api-reference.md): Comprehensive reference for the available REST API endpoints.
+
+- **Release Notes**
+  - [Release Notes](docs/user-guide/release-notes.md): Information on the latest updates, improvements, and bug fixes.
+
+

microservices/audio-intelligence/SECURITY.md

@@ -0,0 +1,17 @@
+# Security Policy
+
+## Security practices
+
+[![OpenSSF Best Practices](https://www.bestpractices.dev/projects/<project-id>/badge)](https://www.bestpractices.dev/projects/<project-id>)
+[![Coverity](https://scan.coverity.com/projects/<project-id>/badge.svg)](https://scan.coverity.com/projects/<project-name>)
+
+## Report a Vulnerability
+
+Please report security issues or vulnerabilities to the [Intel® Security Center].
+
+For more information on how Intel® works to resolve security issues, see
+[Vulnerability Handling Guidelines].
+
+[Intel® Security Center]:https://www.intel.com/security
+
+[Vulnerability Handling Guidelines]:https://www.intel.com/content/www/us/en/security-center/vulnerability-handling-guidelines.html

microservices/audio-intelligence/audio_intelligence/__init__.py

@@ -0,0 +1,19 @@
+# Copyright (C) 2025 Intel Corporation
+# SPDX-License-Identifier: Apache-2.0
+
+from fastapi import APIRouter
+
+from audio_intelligence.schemas.transcription import HealthResponse
+
+router = APIRouter()
+
+
+@router.get("/health", response_model=HealthResponse, tags=["Health API"], summary="Health status of API")
+async def health_check() -> HealthResponse:
+    """
+    Health check endpoint.
+    
+    Returns:
+        A response indicating the service status, version and a descriptive message.
+    """
+    return HealthResponse()

microservices/audio-intelligence/audio_intelligence/api/__init__.py

@@ -0,0 +1,45 @@
+# Copyright (C) 2025 Intel Corporation
+# SPDX-License-Identifier: Apache-2.0
+
+from fastapi import APIRouter
+
+from audio_intelligence.core.settings import settings
+from audio_intelligence.schemas.transcription import AvailableModelsResponse, WhisperModelInfo
+from audio_intelligence.utils.logger import logger
+
+router = APIRouter()
+
+
+@router.get(
+    "/models",
+    response_model=AvailableModelsResponse,
+    tags=["Models API"],
+    summary="Get list of models available for use with detailed information",
+)
+async def get_available_models() -> AvailableModelsResponse:
+    """
+    Get a list of available Whisper model variants that can be used for transcription.
+    
+    This endpoint returns all the whisper models that are configured in the service
+    and available for transcription requests, along with detailed information including
+    display names, descriptions, and the default model that is used when no specific 
+    model is requested.
+    
+    Returns:
+        A response with the list of available models with their details and the default model
+    """
+    logger.debug("Getting available models details")
+    
+    # Get the list of enabled models from settings with their detailed information
+    model_info_list = [model.to_dict() for model in settings.ENABLED_WHISPER_MODELS]
+    
+    # Convert dictionaries to ModelInfo objects
+    models = [WhisperModelInfo(**model_info) for model_info in model_info_list]
+    default_model = settings.DEFAULT_WHISPER_MODEL.value
+    
+    logger.debug(f"Available models: {len(models)} models, default: {default_model}")
+    
+    return AvailableModelsResponse(
+        models=models,
+        default_model=default_model
+    )

microservices/audio-intelligence/audio_intelligence/api/endpoints/__init__.py

@@ -0,0 +1,130 @@
+# Copyright (C) 2025 Intel Corporation
+# SPDX-License-Identifier: Apache-2.0
+
+import traceback
+from typing import Annotated
+
+from fastapi import APIRouter, Query, HTTPException, status, Depends
+from pydantic.json_schema import SkipJsonSchema
+
+from audio_intelligence.schemas.transcription import (
+    ErrorResponse,
+    TranscriptionResponse,
+    TranscriptionStatus,
+    TranscriptionFormData
+)
+from audio_intelligence.core.audio_extractor import AudioExtractor
+from audio_intelligence.core.transcriber import TranscriptionService
+from audio_intelligence.utils.file_utils import get_file_duration
+from audio_intelligence.utils.validation import RequestValidation
+from audio_intelligence.utils.transcription_utils import get_video_path, store_transcript_output
+from audio_intelligence.utils.logger import logger
+
+router = APIRouter()
+
+
+@router.post(
+    "/transcriptions",
+    response_model=TranscriptionResponse,
+    responses={
+        status.HTTP_400_BAD_REQUEST: {"model": ErrorResponse},
+        status.HTTP_500_INTERNAL_SERVER_ERROR: {"model": ErrorResponse},
+        status.HTTP_422_UNPROCESSABLE_ENTITY: {"description": "Invalid request body or parameter provided"},
+    },
+    tags=["Transcription API"],
+    summary="Transcribe audio from uploaded video file or a video stored at Minio"
+)
+async def transcribe_video(
+    request: Annotated[TranscriptionFormData, Depends()],
+    language: Annotated[
+        str | SkipJsonSchema[None], 
+        Query(description="_(Optional)_ Language for transcription. If not provided, auto-detection will be used.")
+    ] = None
+) -> TranscriptionResponse:
+    """
+    Transcribe speech from a video file.
+    
+    Upload a video file directly or specify MinIO parameters to transcribe its audio content.
+    
+    Two ways to provide the video:
+    - Upload a video file using form-data
+    - Specify MinIO parameters (minio_bucket, video_id, video_name) to retrieve from storage
+     
+    Args:
+        request: Form data containing the file or MinIO parameters and transcription settings
+        language: Optional language code for transcription
+    
+    Returns:
+        A response with the transcription status and details
+    """
+    
+    try:
+        # Validate the request parameters
+        RequestValidation.validate_form_data(request)
+
+        logger.info(f"Received transcription request for {'file upload' if request.file else 'MinIO video'}")
+        logger.debug(f"Transcription parameters: model={request.model_name}, device={request.device}, language={language}")
+    
+        # Get video path either from direct upload or MinIO
+        video_path, filename = await get_video_path(request)
+        
+        # Extract audio from video
+        audio_path = await AudioExtractor.extract_audio(video_path)
+        logger.debug(f"Audio extracted successfully to: {audio_path}")
+        
+        # Get file duration
+        duration = get_file_duration(video_path)
+        logger.debug(f"File duration: {duration} seconds")
+        
+        logger.info(f"Initializing transcription service with model: {request.model_name}, device: {request.device}")
+        transcriber = TranscriptionService(
+            model_name=request.model_name,
+            device=request.device
+        )
+        
+        # Perform transcription
+        job_id, transcript_path = await transcriber.transcribe(
+            audio_path,
+            language=language,
+            include_timestamps=request.include_timestamps,
+            video_duration=duration  # Pass the video duration to optimize processing
+        )
+        
+        # Store the transcript output using the configured backend
+        output_location = store_transcript_output(
+            transcript_path, 
+            job_id, 
+            filename,
+            minio_bucket=request.minio_bucket,
+            video_id=request.video_id
+        )
+
+        if not output_location:
+            raise Exception("Failed to store transcript output.")
+        
+        logger.info(f"Transcription completed using {transcriber.backend.value} on {transcriber.device_type.value}")
+        
+        return TranscriptionResponse(
+            status=TranscriptionStatus.COMPLETED,
+            message="Transcription completed successfully",
+            job_id=job_id,
+            transcript_path=output_location,
+            video_name=filename,
+            video_duration=duration
+        )
+    
+    except HTTPException as http_exc:
+        raise http_exc
+    
+    except Exception as e:
+        error_details = traceback.format_exc()
+        logger.error(f"Transcription failed: {str(e)}")
+        logger.debug(f"Error details: {error_details}")
+        
+        raise HTTPException(
+            status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
+            detail=ErrorResponse(
+                error_message=f"Transcription failed!",
+                details="An error occurred during transcription. Please check logs for details."
+            ).model_dump()
+        )