GenAI Document Summarization Sample Application — Full Demo Breakdown

[biapalmeiro]

This discussion covers the key concepts demonstrated in the GenAI Document Summarization Sample Application video. The demo walks through the full end-to-end workflow of deploying an AI-powered summarization service on an Intel® Arrow Lake Core Ultra edge platform using the Edge AI Libraries and Edge Microvisor Toolkit (EMT) image.

🎥 Video Demo: https://www.youtube.com/watch?v=L92vUPFhRko

The demo presents a complete edge-ready GenAI document summarization pipeline, including:

• A microservices-based architecture (API, UI, model service)
• Deployment using Docker Compose
• Optimized LLM inference using OpenVINO, Optimum Intel, and INT8 quantization
• A web interface for uploading documents and getting instant summaries

This application represents a turnkey pattern for running text-heavy AI workloads at the edge and answers common challenges in edge GenAI deployments:

Challenge 1: Running LLMs efficiently on edge hardware
➡️ Solution: quantization + OpenVINO + Optimum Intel accelerate inference while minimizing resource use.

Challenge 2: Deploying complex GenAI pipelines
➡️ Solution: microservices + Docker Compose simplify packaging, scaling, and operations.

Challenge 3: Enabling non-technical users to access AI capabilities
➡️ Solution: clean, simple, web-based UI for document uploads and summary outputs.

Challenge 4: Managing model execution reliably on edge nodes
➡️ Solution: EMT image + controlled container environment ensures repeatable deployments.

Key Workflow Steps

1. Preparing the Environment

• Verified Docker service is running
• Cloned the Edge AI Libraries repo
• Navigated to the document-summarization sample folder
• Exported environment variables (model paths, registry configs, etc.)

2. Installing Dependencies

The setup script installs:

• transformers
• sentence-transformers
• OpenVINO optimizations
• Optimum Intel runtime for hardware-accelerated inference

This ensures models run efficiently on Intel’s Arrow Lake platform.

3. Model Download, Optimization & Quantization

• Downloads model configs, tokenizers, and weights
• Applies INT8 quantization and graph optimizations
• Registers the optimized model for deployment
• Reduces memory footprint and speeds up inference while maintaining accuracy

4. Deploying via Docker Compose

• Starts the summarization API, UI, and supporting services
• Builds a production-ready containerized pipeline
• Runs on an EMT-edge provisioned device

5. Accessing the UI

• SSH port-forwarding into the edge device
• Opening the UI via localhost
• Uploading documents (Word, PDF, TXT)
• Generating summarization results in seconds

Main Takeaways

• Purpose-built for edge inference
  Optimized for Intel Arrow Lake Core Ultra with NPU, iGPU, and CPU orchestration.
• End-to-end containerized microservice architecture
  Developers can reuse or extend components (API, UI, model backend).
• Full automation through setup scripts
  No manual dependency or model handling.
• Quantized models = efficient edge performance
  INT8 quantization drastically improves throughput on device.
• Simple user workflow
  Upload → Process → Summary in seconds.

---

👉 Learn more about the GenAI Document Summarization Sample Application and Edge Microvisor Toolkit (EMT)
👉 Visit the Open Edge Platform Playlist for more demos