feat: LLM Gateway V2 integration with publication system #256
Conversation
- Add LLM Gateway V2 HTTP/WebSocket client - Job submission with progress tracking via WebSocket - Support for multiple export formats (summary, chapters, etc.) - Real-time job status updates through Gateway WebSocket API - Update environment config for V2 endpoints
- Add version history for export results via LLM Gateway - Restore previous versions endpoint - Generate PDF/DOCX documents from export results - New routes: versions list, get version, restore, generate document
- Add Socket.IO rooms for LLM job broadcasts (llm/{orgId}, llm/{orgId}/{convId})
- Add llm:join/llm:leave handlers in studio-api IoHandler
- Add webhook receiver in studio-websocket for Gateway callbacks
- Add llmJobController for job subscription management
- Security: verify organization membership before joining any room
- Security: add auth checks on watch_organization_media/session events
- Rework ConversationsPublish view for LLM Gateway V2 - Add real-time job progress via WebSocket (llm:job:update events) - Add inline markdown editing with WYSIWYG editor - Add version history panel with restore capability - Add PDF/DOCX document generation - Add job regeneration with format selection - i18n support (en-US, fr-FR)
- Add publication routes for template management - Add publication controller with LLM Gateway integration - Update export controllers for template-based generation - Add axios utility with timeout configuration - Update LLM controller for document export
- Add ConversationGenerations model for tracking AI generations - Add generations controller with CRUD operations - Add routes for generation history endpoints - Add database migration 1.6.2
- Add PublicationSection component with template upload - Add PublicationTemplateCard for template display - Add publication API client - Update ConversationsPublish view with template support - Update publish content component and styles
- Add AIServiceMenu component for service selection - Add GenerationTimeline for tracking generation history - Add PdfViewer component for document preview - Update Markdown WYSIWYG editor - Update conversation and service API clients
- Add 18 English translation keys for publication section - Add 3 French translation keys for delete confirmation - Remove hardcoded French fallback strings
- Add publication API tests - Add LLM integration tests (services, versioning, websocket) - Update frontend and API dependencies
Show regenerate button only when verbatim has been modified or when user is organization admin.
When a job completes, the timeline was showing stale data (old versions from previous generation) until page refresh. Now properly calls loadGenerations() and loadVersions() in handleJobComplete to refresh the timeline and versions list in real-time.
- Top bar mode toggle: edit, preview, publication icons - Template cards: scope badges (pin-on, work, profile), delete icon - Upload modal: upload and document icons - Added proper icon colors for different states (hover, active)
LLM Gateway is now the single source of truth for job content. Changes: - Remove local data storage in conversationExport.data - Add GET /export/:jobId/content endpoint to fetch from gateway - Add health check before acting on 404 (avoid deleting refs when gateway is down) - Filter orphan exports/generations when listing (validate against gateway) - Auto-regenerate only when NO valid generations exist for a service - Mark conversationExport.data field as deprecated This fixes cache coherence issues when jobs are edited/deleted in LLM Gateway.
- Add apiGetExportContent() to fetch job content from gateway - Update getPreview() to always fetch fresh content for editable outputs - Handle job_not_found by refreshing generations list (no auto-regen) - Handle gateway_unavailable with appropriate error message - Add i18n key job_removed_refreshing for deleted job notification
- getExportContent.test.js: Test new endpoint with various scenarios (success, job_not_found, gateway_unavailable, processing state) - autoRegeneration.test.js: Test conditional auto-regeneration logic (only when no valid generations exist for service)
Add Raw JSON (WhisperX) export option to verbatim download menu. The export includes word-level timing, confidence scores, and speaker attribution. Closes #253
studio-api/components/MongoMigration/version/1.6.2/conversationGenerations.js
Outdated
Show resolved
Hide resolved
studio-api/components/MongoMigration/version/1.6.2/conversationGenerations.js
Outdated
Show resolved
Hide resolved
studio-api/components/MongoMigration/version/1.6.2/conversationGenerations.js
Outdated
Show resolved
Hide resolved
| ) | ||
|
|
||
| const { exportConversation, listExport } = require( | ||
| const { |
There was a problem hiding this comment.
Refactor studio-api/components/WebServer/controllers/llm/index.js to behave like a proxy, limiting responsibilities to only what is required, such as storing the request or related data.
Reference similar logic in:
components/WebServer/routes/router.js(around line 177:createProxyRoutes)components/WebServer/routes/proxy/sessions/session.js
There was a problem hiding this comment.
After detailed analysis, I decided not to implement the proxy pattern for LLM routes.
The existing proxy pattern (used for session routes) works well for simple passthrough with before/after hooks. However, the LLM controllers require:
- Complex conditional logic based on service resolution
- WebSocket room management (
io.to().emit()) - Multi-step orchestration (create generation record → submit job → poll status → emit updates)
- Error handling with specific business logic
These don't fit the proxy model where we just forward requests and optionally process responses. The current controller approach provides the necessary flexibility.
There was a problem hiding this comment.
I did a thorough analysis to implement the proxy pattern, and encountered several technical issues that make this approach not viable:
1. HTTP Method Incompatibility
Studio API uses PUT for /conversations/:id/export/:jobId/result, but LLM Gateway expects PATCH. The http-proxy-middleware forwards the original HTTP method, which would cause 405 errors.
2. Complex Path Transformations
Routes require non-trivial transformations:
/conversations/:conversationId/export/:jobId/result→/jobs/:jobId/result
The existing proxy pattern (sessions) uses scrapPath + rewrite, but the execution order is problematic: rewrite applies before scrapPath, requiring complex regex with capture groups.
3. Route Conflicts
Cannot register both proxy routes AND existing API routes for the same endpoints. Would need to remove current API routes, breaking existing business logic (validation, user context, ownership checks).
4. Business Logic in Hooks
The proxy pattern's executeBeforeResult and executeAfterResult hooks are designed for simple operations. LLM controllers require:
- WebSocket room management (
io.to(room).emit()) - MongoDB record creation/updates before AND after requests
- Conditional service resolution
- Status polling with retry logic
Conclusion
I created the proxy files as an exercise (then deleted them), but the pattern doesn't fit. Session routes work as proxy because they're true passthrough. LLM routes are orchestration, not proxying.
There was a problem hiding this comment.
Note: The HTTP method mismatch (issue 1) could be fixed by changing frontend + API to use PATCH. However, issue 4 (WebSocket management, transactional MongoDB, retry logic) is the real blocker - proxy hooks can't handle orchestration logic.
There was a problem hiding this comment.
The mismatch is on Studio side (frontend calls PUT, API defines PUT route) while LLM Gateway correctly expects PATCH for partial updates. Semantically PATCH is correct here - could be worth fixing in a future cleanup PR. @tjiho
studio-api/components/WebServer/routecontrollers/conversation/generations.js
Show resolved
Hide resolved
studio-api/components/WebServer/routecontrollers/publication/publication.js
Show resolved
Hide resolved
Create checkSocketOrganizationAccess helper function in middlewares/access/organization.js and use it in IoHandler for: - watch_organization_session - watch_organization_media - llm:join Also adds public session token support for llm:join handler, allowing anonymous users with valid public tokens to receive LLM job updates for their session's organization.
- Remove all debug statements from migration file - Create schema helper in controllers/schema/conversationGenerations.js following the pattern from tokens.js - Simplify migration to use helper for index creation
LLM Gateway generates documents directly, so this duplicate function is not needed. Removes generateDocxFromMarkdown and parseMarkdownInline helper, along with unused imports.
Include organizationId in public session JWT tokens to enable validation of organization membership for anonymous users. This fixes the security regression where public session users couldn't receive WebSocket updates for their session. Changes: - public_generator.js: Accept optional organizationId parameter - session.js: Pass session.organizationId when generating tokens
After analysis, the proxy pattern is not suitable for LLM Gateway routes because they require orchestration logic (callbacks, state management, WebSocket coordination) that simple passthrough cannot replicate. Removed the commented-out proxy registration code.
studio-api/components/WebServer/routecontrollers/publication/publication.js
Outdated
Show resolved
Hide resolved
- Create publication.js exception file with 8 custom error classes - Refactor controller to throw custom errors and use next(err) - Remove duplicate error logging from catch blocks - Let global error handler manage propagation and logging
…ollers - Add 5 new error classes to conversation.js: ExportError, ExportNotConfigured, ExportJobNotFound, ExportGatewayError, GenerationNotFound - Refactor export.js to use custom errors and next(err) pattern - Refactor generations.js to use custom errors and next(err) pattern - Update tests to verify new error handling behavior
|
Extended the custom error class pattern to Added 5 new error classes to
All LLM Gateway-related controllers now use the global error handler pattern. |
Summary
This PR integrates LLM Gateway V2 with a complete publication system for AI-generated document management.
Key Features
Changes
API (studio-api)
Frontend (studio-frontend)
WebSocket (studio-websocket)
Tests
Comprehensive test coverage added:
Stats