Add MCP server with image search and carousel display

[pamelafox]

Summary

This PR adds an MCP (Model Context Protocol)
server to the project, enabling AI assistants and agents (e.g. VS Code GitHub Copilot) to search and display images from the Azure AI Search index.

Changes

MCP server (app/backend/mcp_server.py)

• New FastMCP server exposing two tools:
  • image_search – searches the Azure AI Search index using vector embeddings; returns 256×256 thumbnails (via Pillow) to keep LLM context token usage low
  • display_image_files – fetches one or more images from blob storage and renders them in a carousel MCP App iframe

MCP App (app/backend/image-viewer.html)

• New HTML page served as an MCP resource, rendered in an iframe by MCP clients
• Carousel UI with prev/next navigation and an image counter

Infrastructure

• app/backend/Dockerfile.mcp – container image for the MCP server
• infra/aca-mcp.bicep – Azure Container Apps definition for the MCP server
• infra/main.bicep / infra/main.parameters.json – wired up MCP server deployment

Dependencies

• Added Pillow>=11.0.0 to requirements.txt for thumbnail resizing

Other

• app/backend/setup_search_service.py – updated to upload images from pictures/clothes/ and optionally generate image descriptions via ChatCompletionSkill
• app/frontend – search results now display image descriptions
• README.md – updated to describe the MCP server, its tools, local run instructions, and how the indexer pipeline works
• .vscode/mcp.json – MCP server configuration for local development
• .agents/skills/convert-heic-to-jpeg/ – agent skill for converting HEIC photos to JPEG before indexing

[Copilot]

Pull request overview

Adds an MCP (Model Context Protocol) server alongside the existing web app to enable AI clients to search an Azure AI Search image index and display images via an embedded carousel UI, plus infrastructure updates to deploy the MCP server and (optionally) enrich the index with Azure OpenAI-generated image descriptions.

Changes:

• Introduces a FastMCP server with image_search and display_image_files tools, plus an HTML carousel “app” resource.
• Extends the indexing pipeline to optionally generate and store verbalized_image descriptions via ChatCompletionSkill.
• Updates Azure infrastructure (Bicep + azd service definitions) to deploy an MCP Container App and wire permissions/outputs.

Reviewed changes

Copilot reviewed 18 out of 18 changed files in this pull request and generated 10 comments.

Show a summary per file

File	Description
infra/main.parameters.json	Adds MCP/OpenAI-related deployment parameters.
infra/main.bicep	Deploys MCP container app, OpenAI account/deployment, and role assignments; adds new outputs.
infra/core/host/container-app.bicep	Adds support for exposing additional ingress ports.
infra/core/host/container-app-upsert.bicep	Plumbs additionalPorts through the upsert module.
infra/aca-mcp.bicep	New module defining the MCP Container App + managed identity.
azure.yaml	Registers a new mcp service built from Dockerfile.mcp.
app/frontend/src/components/SearchResults/SearchResults.tsx	Loads real image dimensions before rendering the gallery.
app/backend/setup_search_service.py	Adds verbalized_image field and optional OpenAI chat completion skill.
app/backend/requirements.txt	Adds FastMCP + Pillow and bumps Azure/Python deps.
app/backend/mcp_server.py	New FastMCP server implementing search + carousel display tools.
app/backend/image-viewer.html	New carousel UI rendered as an MCP App resource.
app/backend/app.py	Updates vector query parameter name for the Azure SDK.
app/backend/Dockerfile.mcp	New container image definition for the MCP server.
README.md	Documents MCP server usage and indexing pipeline.
.vscode/mcp.json	Adds local and Azure MCP server endpoints for VS Code.
.agents/skills/convert-heic-to-jpeg/convert_heic_to_jpeg.py	Adds a utility script to convert HEIC/HEIF images to JPEG.
.agents/skills/convert-heic-to-jpeg/SKILL.md	Documents running the HEIC→JPEG conversion skill via uv run.

Comments suppressed due to low confidence (2)

app/backend/mcp_server.py:155

• fetch_image_bytes creates a new aiohttp.ClientSession for every image fetch. This is expensive and can exhaust sockets under load; prefer a single shared session (or use the Azure Blob SDK) with timeouts and reuse across requests, closing it on server shutdown.

    """Infer MIME type for supported image formats from blob filename."""
    image_format = get_image_format(filename)
    mime_type = (
        "image/jpeg" if image_format in {"jpg", "jpeg"} else f"image/{image_format}"
    )
    if mime_type in ALLOWED_IMAGE_MIME_TYPES:
        return mime_type

app/backend/Dockerfile.mcp:12

• Build toolchains (gcc, musl-dev, python3-dev, cargo, etc.) are installed in the shared base stage, so they end up in the final runtime image too. Move build-only dependencies into the build stage and keep the final stage minimal to reduce image size and attack surface.

# ------------------- Stage 0: Base Stage ------------------------------
FROM python:3.11-alpine AS base

WORKDIR /code

# Install tini, a tiny init for containers
RUN apk add --update --no-cache tini

# Install required packages for cryptography package
# https://cryptography.io/en/latest/installation/#building-cryptography-on-linux
RUN apk add gcc musl-dev python3-dev libffi-dev openssl-dev cargo pkgconfig

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 18 out of 18 changed files in this pull request and generated 7 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 18 out of 18 changed files in this pull request and generated 2 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.