Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ This document provides guidelines and instructions for AI assistants working on
- ✅ **Automated Quality Assessment** - VLM-based reflection for generated images (FR-9 completed)
- ✅ **Product FAQ Generation** - FAQs from enriched data with optional product manual PDF enhancement (FR-10, FR-12 completed)
- ✅ **Policy Compliance** - PDF policy library with Milvus RAG and compliance classification (FR-11 completed)
- ✅ **Protocol Schema Export** - ACP and UCP schema generation with LLM-extracted structured fields (FR-13 completed)
- ⚠️ **In Development** - 3D Asset Generation (backend complete) and Video Generation in progress

### Key Goals
Expand Down Expand Up @@ -132,6 +133,22 @@ uvicorn --app-dir src backend.main:app --host 0.0.0.0 --port 8000 --reload
- LLM generates 5-8 product-type-specific queries from title + categories (not description)
- Retrieves relevant chunks per query via in-memory cosine similarity

**Protocol Schema Generation:**
- POST `/protocols/generate`
- Request: `multipart/form-data` with fields:
- `title` (string, optional): Enriched product title
- `description` (string, optional): Enriched product description
- `categories` (JSON string, optional): Categories array
- `tags` (JSON string, optional): Tags array
- `colors` (JSON string, optional): Colors array
- `faqs` (JSON string, optional): FAQs array from `/vlm/faqs`
- `locale` (string, optional): Regional locale code (default: "en-US")
- Response: `{ "acp": { ... }, "ucp": { ... } }`
- Calls LLM once to extract brand, material, age_group, gender, product_details, product_highlights, short_title, google_product_category
- Merges LLM-extracted fields + enriched data + deterministic defaults into both ACP and UCP schemas
- ACP: agent-oriented schema with product, pricing, FAQs, agent_actions, fulfillment, campaigns
- UCP: Google Merchant Center-derived schema with structured_title/description using `trained_algorithmic_media`

**3D Asset Generation:**
- POST `/generate/3d`
- Request: `multipart/form-data` with fields:
Expand Down
6 changes: 4 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,8 @@ A GenAI-powered catalog enrichment system that transforms basic product images i
- **3D Asset Generation**: Transform 2D product images into interactive 3D GLB models using Microsoft TRELLIS
- **Product FAQ Generation**: Automatically generate product FAQs from enriched catalog data, with optional product manual PDF upload for richer FAQs (up to 10) via stateless targeted RAG
- **Policy Compliance**: Upload policy PDFs and automatically check product listings against them using RAG + Milvus
- **Modular API**: Separate endpoints for VLM analysis, FAQ generation, image generation, and 3D asset generation
- **Protocol Schema Export**: Export enriched product data as ACP (Agentic Commerce Protocol) and UCP (Unified Commerce Protocol) compliant schemas with LLM-extracted structured attributes
- **Modular API**: Separate endpoints for VLM analysis, FAQ generation, image generation, 3D asset generation, and protocol schema export

## Documentation

Expand Down Expand Up @@ -220,13 +221,14 @@ For complete Docker deployment instructions, see the **[Docker Deployment Guide]

## API Endpoints

The system provides three main endpoints:
The system provides the following endpoints:

- `POST /vlm/analyze` - Fast VLM/LLM analysis
- `POST /vlm/faqs` - Product FAQ generation (supports optional manual knowledge)
- `POST /vlm/manual/extract` - Extract knowledge from a product manual PDF for FAQ enrichment
- `POST /generate/variation` - Image generation with FLUX
- `POST /generate/3d` - 3D asset generation with TRELLIS
- `POST /protocols/generate` - ACP & UCP protocol schema generation

### Image Input Guidance

Expand Down
89 changes: 89 additions & 0 deletions docs/API.md
Original file line number Diff line number Diff line change
Expand Up @@ -573,6 +573,95 @@ curl -X POST \

---

## 6️⃣ Protocol Schema Generation: `/protocols/generate`

Generate ACP (Agentic Commerce Protocol) and UCP (Unified Commerce Protocol) schema instances from enriched product data. Uses an LLM call to extract structured attributes (brand, material, product details, etc.) from the enriched title and description, then merges them into both schema templates.

**`POST /protocols/generate`**

Content-Type: `multipart/form-data`

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `title` | string | No | Enriched product title |
| `description` | string | No | Enriched product description |
| `categories` | JSON string | No | Categories array (default: `[]`) |
| `tags` | JSON string | No | Tags array (default: `[]`) |
| `colors` | JSON string | No | Colors array (default: `[]`) |
| `faqs` | JSON string | No | FAQs array (default: `[]`) |
| `locale` | string | No | Regional locale code (default: `en-US`) |

### Response Schema

```json
{
"acp": {
"product": {
"id": null,
"title": "Nature Made Fish Oil Softgels...",
"description": "Support your heart health...",
"brand": "Nature Made",
"attributes": { "colors": ["brown", "yellow"], "material": null, "condition": "new", ... },
"categories": ["health", "supplements"],
"tags": ["fish oil", "omega-3", ...],
"images": { ... },
"identifiers": { "gtin": null, "mpn": null, "sku": null },
"dimensions": { ... },
"details": [{ "attribute_name": "Omega-3 Content", "attribute_value": "360 mg" }, ...],
"highlights": ["Supports heart health", ...]
},
"pricing": { "availability": "in_stock", "price": null, ... },
"faqs": [{ "question": "...", "answer": "..." }, ...],
"agent_actions": { "discoverable": true, "buyable": true, "returnable": true, ... },
"fulfillment": { ... },
"campaigns": { "short_title": "Nature Made Fish Oil 300ct", ... },
"certifications": [],
"energy_efficiency": { ... },
"bundling": { ... },
"marketplace": { ... },
"metadata": { "enrichment_source": "nvidia-catalog-enrichment", ... }
},
"ucp": {
"structured_title": { "digital_source_type": "trained_algorithmic_media", "content": "..." },
"structured_description": { "digital_source_type": "trained_algorithmic_media", "content": "..." },
"brand": "Nature Made",
"color": "brown / yellow",
"condition": "new",
"product_type": "health > supplements",
"google_product_category": "Health > Vitamins & Supplements > Fish Oil",
"product_detail": [{ "attribute_name": "...", "attribute_value": "..." }],
"product_highlight": ["..."],
"faqs": [{ "question": "...", "answer": "..." }],
"price": { "amount": null, "currency": null },
"shipping": [],
...
}
}
```

### Usage Example

```bash
curl -X POST \
-F "title=Nature Made Fish Oil Softgels, 1200 mg, 300 Count" \
-F "description=Support your heart health with Omega-3 fatty acids." \
-F 'categories=["health","supplements"]' \
-F 'tags=["fish oil","omega-3","heart health"]' \
-F 'colors=["brown","yellow"]' \
-F 'faqs=[{"question":"Is it mercury-free?","answer":"Yes, purified to remove mercury."}]' \
-F "locale=en-US" \
http://localhost:8000/protocols/generate
```

**Notes:**
- Calls the LLM once to extract structured fields (brand, material, age_group, gender, short_title, google_product_category, product_details, product_highlights), then builds both schemas from the same extraction
- ACP schema includes agent actions, fulfillment, and campaigns sections for agentic commerce
- UCP schema follows the Google Merchant Center Product Data Specification with `structured_title`/`structured_description` using `digital_source_type: "trained_algorithmic_media"` for AI-generated content
- Fields not derivable from enriched data are set to `null` for the consumer to fill in
- Deterministic defaults: `availability` = `"in_stock"`, `condition` = `"new"`, `adult` = `false`, `is_bundle` = `false`

---

## Supported Locales

The API supports 10 regional locales for language and cultural context:
Expand Down
12 changes: 12 additions & 0 deletions docs/PRD.md
Original file line number Diff line number Diff line change
Expand Up @@ -159,6 +159,17 @@ A GenAI-powered catalog enrichment system that transforms basic product images i
- Support deduplication of repeated policy uploads by content hash
- Display compliance results in the UI with visual pass/fail indicators

### FR-13: Protocol Schema Export (ACP & UCP)
- Generate ACP (Agentic Commerce Protocol) and UCP (Unified Commerce Protocol) schema instances from enriched product data
- Use LLM to extract structured attributes (brand, material, age_group, gender, product_details, product_highlights, short_title, google_product_category) from enriched title and description
- Merge LLM-extracted fields with enriched data and deterministic defaults (availability, condition, adult, is_bundle) into full schema templates
- ACP schema covers product, pricing, FAQs, agent actions, fulfillment, campaigns, certifications, energy efficiency, bundling, marketplace, and metadata
- UCP schema follows Google Merchant Center Product Data Specification across all 9 sections (basic product data, price/availability, product category, identifiers, detailed description, shopping campaigns, marketplaces, destinations, shipping/returns)
- UCP uses `structured_title` and `structured_description` with `digital_source_type: "trained_algorithmic_media"` for AI-generated content
- Single `/protocols/generate` endpoint calls LLM once and returns both schemas
- UI displays schemas in a Protocols tab with ACP/UCP sub-tabs, syntax-highlighted JSON, and copy-to-clipboard
- Schema generation fires in the background after FAQ generation completes, ensuring FAQs are included in both schemas

## Technical Requirements

### TR-1: Model Integration
Expand Down Expand Up @@ -304,6 +315,7 @@ A GenAI-powered catalog enrichment system that transforms basic product images i
- [x] ~~FR-10: Product FAQ Generation~~ *(Separate /vlm/faqs endpoint with async loading, Kaizen Tabs + Accordion UI)*
- [x] ~~FR-11: Policy Compliance Checking~~ *(PDF policy library with Milvus embeddings, semantic retrieval, compliance classification)*
- [x] ~~FR-12: Product Manual PDF Enhancement for FAQs~~ *(Stateless targeted RAG via /vlm/manual/extract, dynamic query generation, up to 10 manual-enriched FAQs)*
- [x] ~~FR-13: Protocol Schema Export (ACP & UCP)~~ *(Single /protocols/generate endpoint with LLM field extraction, syntax-highlighted UI with ACP/UCP sub-tabs)*

- [ ] TR-1: Model Integration
- [x] ~~NVIDIA Nemotron VLM API integration~~
Expand Down
Loading
Loading