Phase5/dev

[namtroi]

PR Type

Enhancement, Tests, Documentation

---

Description

Phase 5 Production Infrastructure Implementation

• Qdrant Vector Database Integration: Added QdrantService for hybrid search with dense + sparse vectors using RRF fusion, replacing pgvector-based search

• Hybrid Embeddings: Implemented HybridEmbedder in AI Worker generating both dense (384-dim BGE-small) and sparse (BM25) vectors with new /embed/query endpoint

• Vector Sync Infrastructure: Created QdrantSyncProcessor with outbox pattern, batch processing, and exponential backoff to sync chunks from PostgreSQL to Qdrant

• Per-User OAuth for Google Drive: Added UserDriveService with AES-256-GCM encrypted token storage, Google OAuth routes, and Drive Picker integration for folder selection

• Encryption Service: Implemented EncryptionService for authenticated encryption of sensitive data (OAuth refresh tokens)

• Schema Updates: Renamed DriveConfig to DriveFolder, added ChunkSyncStatus enum (PENDING/SYNCED/FAILED), added vector fields to chunks, new DriveOAuth model

• System Health Dashboard: Added /api/system endpoint and SettingsPage component showing Qdrant status, AI Worker health, encryption config, and OAuth setup

• Analytics Enhancement: Added sync queue statistics (PENDING/SYNCED/FAILED counts) to analytics overview

• Frontend Components: New GoogleOAuthSection, SyncFrequencySelect, AddFolderModal with Drive Picker, and sync status badges on chunk cards

• Test Coverage: Added Qdrant service integration tests, hybrid embedder tests, encryption service tests; skipped search/query tests pending Qdrant availability

• Documentation: Updated architecture, API contracts, data flow, and roadmap; added detailed Phase 5 implementation plan and Phase 6 SaaS roadmap

• Removed Legacy Code: Deleted old HybridSearchService, Embedder, and hybrid search initialization code

---

Diagram Walkthrough

flowchart LR
  A["AI Worker<br/>HybridEmbedder"] -->|"dense + sparse<br/>vectors"| B["PostgreSQL<br/>Chunks Table"]
  B -->|"outbox pattern<br/>batch sync"| C["QdrantSyncProcessor<br/>BullMQ Queue"]
  C -->|"upsert vectors"| D["Qdrant Cloud<br/>Hybrid Search"]
  E["Google OAuth<br/>Flow"] -->|"encrypted tokens<br/>AES-256-GCM"| F["DriveOAuth<br/>Model"]
  F -->|"decrypt & use"| G["UserDriveService<br/>Per-User OAuth"]
  G -->|"sync files"| B
  H["Search Request"] -->|"query embeddings"| A
  A -->|"dense + sparse"| D
  D -->|"RRF fusion"| I["Search Results<br/>with Qdrant Badge"]
  J["Settings Dashboard"] -->|"health check"| K["System Status<br/>Endpoint"]

Loading

File Walkthrough

	Relevant files
Tests	11 files search-route.test.ts Skip search tests, add Qdrant unavailability tests              apps/backend/tests/integration/routes/search-route.test.ts Skipped all search route tests with describe.skip() due to Phase 5 Qdrant requirement Removed 245+ lines of integration tests for semantic/hybrid search, filtering, and modes Added new test suite for Qdrant unavailability (503 SEARCH_UNAVAILABLE) behavior Kept only validation error tests that execute before Qdrant check +52/-297 query-flow.test.ts Skip E2E query tests, add Qdrant unavailability test          apps/backend/tests/e2e/query-flow.test.ts Skipped all E2E query tests with describe.skip() due to Qdrant requirement Removed 120+ lines of query validation tests (topK, ordering, metadata) Added new test suite verifying 503 SEARCH_UNAVAILABLE when Qdrant not configured Simplified document content in remaining test +36/-125 qdrant.service.test.ts Add Qdrant service integration tests                                          apps/backend/tests/integration/qdrant.service.test.ts New integration tests for QdrantService against live Qdrant Cloud instance Tests collection creation, upsert, hybrid/dense search, filtering, and deletion Uses test collection with timestamp to avoid production data conflicts Skipped if QDRANT_URL not configured +209/-0  encryption.service.test.ts Add encryption service unit tests                                                apps/backend/tests/unit/encryption.service.test.ts New unit tests for AES-256-GCM encryption service Tests encryption/decryption, tamper detection, key validation Covers edge cases: empty strings, unicode, long strings, ciphertext tampering Validates encrypted payload structure (IV, auth tag lengths) +112/-0  multi-format-flow.test.ts Remove query tests from multi-format E2E flows                      apps/backend/tests/e2e/multi-format-flow.test.ts Removed query tests from Markdown, TXT, and JSON upload flows Renamed tests from "Upload → Callback → Query" to "Upload → Callback → Completed" Kept document upload and completion verification, removed search assertions +8/-37    sync-service-relink.test.ts Update sync service tests for UserDriveService                      apps/backend/tests/unit/services/sync-service-relink.test.ts Updated mocks from getDriveService to UserDriveService.create() Changed all driveConfigId references to driveFolderId Updated mock setup to use async UserDriveService.create() pattern +7/-7      pdf-upload-flow.test.ts Remove query test from PDF upload E2E flow                              apps/backend/tests/e2e/pdf-upload-flow.test.ts Added documentation comment explaining Phase 5 Qdrant requirement Removed query test from PDF upload flow Renamed test from "Upload → Queue → Callback → Chunks → Query" to "Upload → Queue → Callback → Chunks" +8/-21    database.ts Update test database cleanup for DriveFolder                          apps/backend/tests/helpers/database.ts Updated database cleanup to use driveFolder.deleteMany() instead of driveConfig.deleteMany() +1/-1      test_hybrid_embedder.py Add comprehensive tests for hybrid embedder                            apps/ai-worker/tests/test_hybrid_embedder.py Comprehensive test suite for HybridEmbedder functionality Tests dense vector dimensions (384), sparse vector structure, batch processing Validates token count estimation and unicode handling Includes 11 test cases covering edge cases and data integrity +116/-0  test_existing_formats.py Update regression tests for hybrid embedder                            apps/ai-worker/tests/regression/test_existing_formats.py Updated regression test to use HybridEmbedder instead of Embedder Changed test to verify dense vector dimensions from HybridVector objects Maintains backward compatibility check for 384-dimension embeddings +7/-5      test_main.py Update main.py tests for hybrid embedder mocking                  apps/ai-worker/tests/test_main.py Updated mock to patch HybridEmbedder at module level in main.py Changed mock method from embed() to embed_dense_only() Reflects new endpoint implementation using backward-compatible dense-only method +3/-2
Enhancement	35 files sync-service.ts Migrate to UserDriveService with OAuth support                      apps/backend/src/services/sync-service.ts Replaced DriveService with UserDriveService for per-user OAuth support Changed driveConfig to driveFolder throughout (schema rename) Added lazy initialization of UserDriveService via getDrive() method Updated all Drive API calls to use async UserDriveService.create() +31/-23  qdrant.service.ts Add Qdrant Cloud vector database service                                  apps/backend/src/services/qdrant.service.ts New service for Qdrant Cloud vector database integration with hybrid search Implements dense + sparse vector support with RRF fusion via Query API Provides methods for upsert, delete, hybrid/dense search, and collection management Includes singleton pattern and configuration validation +358/-0  user-drive-service.ts Add per-user OAuth Google Drive service                                    apps/backend/src/services/user-drive-service.ts New service for Google Drive with per-user OAuth credentials (Phase 5F) Decrypts stored refresh tokens using AES-256-GCM encryption Falls back to service account if OAuth not configured Supports recursive file listing, downloads, and change tracking +295/-0  google.route.ts Add Google OAuth routes with encrypted token storage          apps/backend/src/routes/oauth/google.route.ts New OAuth routes for Google Drive integration (Phase 5F) Implements OAuth flow: start → callback → status check → access token → disconnect Encrypts refresh tokens with AES-256-GCM before storage Provides access token endpoint for Google Drive Picker +248/-0  qdrant-sync.processor.ts Add Qdrant sync queue processor with batching                        apps/backend/src/queue/qdrant-sync.processor.ts New BullMQ processor for syncing chunks from PostgreSQL to Qdrant (Phase 5D) Implements outbox pattern with batch processing and exponential backoff Nullifies vectors in PostgreSQL after successful sync to save space Handles continuation jobs for large document sets +237/-0  config-routes.ts Rename driveConfig to driveFolder, use UserDriveService    apps/backend/src/routes/drive/config-routes.ts Renamed driveConfig to driveFolder throughout (schema alignment) Updated to use UserDriveService.create() instead of getDriveService() Added optional folderName parameter from Google Picker to avoid extra API call Updated event bus emit to use driveFolder:deleted +32/-26  callback-route.ts Add batch chunk insertion and Qdrant sync job enqueue        apps/backend/src/routes/internal/callback-route.ts Changed chunk insertion from serial to batch processing (20 concurrent inserts) Added support for Phase 5 hybrid vectors (dense + sparse) alongside legacy embeddings Stores sync_status, dense_vector, sparse_indices, sparse_values in chunks table Enqueues Qdrant sync job after chunk insertion if configured +85/-35  qdrant-hybrid-search.ts Add Qdrant hybrid search service with AI Worker integration apps/backend/src/services/qdrant-hybrid-search.ts New service for Qdrant-based hybrid search (Phase 5E) Gets query embeddings from AI Worker, performs Qdrant search Supports semantic (dense-only) and hybrid (dense+sparse) modes Maps Qdrant results to application format with metadata +137/-0  search-route.ts Replace pgvector search with Qdrant-only implementation    apps/backend/src/routes/query/search-route.ts Replaced pgvector-based search with Qdrant-only implementation Removed EmbeddingClient and HybridSearchService dependencies Returns 503 SEARCH_UNAVAILABLE if Qdrant not configured Simplified response format, removed alpha parameter +18/-69  useDrivePicker.ts Add Google Drive Picker React hook                                              apps/frontend/src/hooks/useDrivePicker.ts New React hook for Google Drive Picker integration Loads Google Picker API dynamically, handles folder selection Gets access token from backend OAuth endpoint Returns folder ID and name for Drive config creation +130/-0  encryption.service.ts Add AES-256-GCM encryption service                                              apps/backend/src/services/encryption.service.ts New AES-256-GCM encryption service for sensitive data (OAuth tokens) Implements authenticated encryption with random IV per encryption Validates 32-byte hex key format, provides singleton instance Includes tamper detection via auth tag verification +110/-0  endpoints.ts Update API types for Qdrant and sync queue support              apps/frontend/src/api/endpoints.ts Removed alpha parameter from SearchParams and SearchResponse Updated SearchResponse.mode to include qdrant_hybrid option Added optional provider field to indicate Qdrant vs pgvector backend Updated driveApi.createConfig to accept optional folderName parameter Added syncQueue stats to AnalyticsOverview interface Added syncStatus field to ChunkListItem interface +10/-5    health-route.ts Add system status endpoint for Settings dashboard                apps/backend/src/routes/health-route.ts Added new /api/system endpoint for Settings dashboard (Phase 5I) Reports vector DB provider, Qdrant connection status, encryption config Checks AI Worker health and hybrid embeddings support Includes OAuth configuration status and system uptime +62/-0    overview-route.ts Add sync queue statistics to analytics overview                    apps/backend/src/routes/analytics/overview-route.ts Added Phase 5J sync queue statistics to overview response Queries chunk sync status counts (PENDING, SYNCED, FAILED) Includes syncQueue object in analytics response +26/-0    worker-init.ts Initialize Qdrant sync worker and queue                                    apps/backend/src/queue/worker-init.ts Added Qdrant sync worker and queue initialization Creates qdrantQueue and qdrantWorker if Qdrant configured Exports getQdrantSyncQueue() function for job enqueueing Includes cleanup in shutdown handler +24/-1    app.ts Register OAuth routes, remove hybrid search init                  apps/backend/src/app.ts Added OAuth routes registration via oauthRoutes(app) Removed hybrid search initialization (initializeHybridSearch) Kept Drive sync cron job initialization +6/-7      callback-validator.ts Add hybrid vector schema support to callback validator      apps/backend/src/validators/callback-validator.ts Added SparseVectorSchema for sparse vector indices/values (Phase 5B) Added HybridVectorSchema combining dense + sparse vectors Updated ProcessingResultSchema to support both legacy embedding and new vector formats Maintains backward compatibility with Phase 4 format +15/-1    sync-routes.ts Rename driveConfig to driveFolder in sync routes                  apps/backend/src/routes/drive/sync-routes.ts Renamed driveConfig to driveFolder in two route handlers Updated Prisma queries to use driveFolder model +2/-2      index.ts Update service exports for Qdrant and encryption                  apps/backend/src/services/index.ts Replaced hybrid search exports with Qdrant service exports Added EncryptionService and getEncryptionService exports Added QdrantService, QdrantHybridSearchService exports with type definitions Removed HybridSearchService and initializeHybridSearch exports +7/-3      oauth.ts OAuth API client for Google Drive integration                        apps/frontend/src/api/oauth.ts New OAuth API client module for Google Drive integration Exports OAuthStatus interface and oauthApi object with methods Provides endpoints for status check, disconnect, token retrieval, and OAuth flow initiation +34/-0    chunks-route.ts Add syncStatus field to chunk responses                                    apps/backend/src/routes/chunks/chunks-route.ts Added syncStatus field to chunk selection in database query Includes syncStatus in response payload mapping +2/-0      hybrid_embedder.py Implement hybrid embedder with dense and sparse vectors    apps/ai-worker/src/hybrid_embedder.py New HybridEmbedder class implementing dense + sparse vector generation Uses fastembed for both BGE-small (384d dense) and BM25 (sparse) embeddings Provides singleton pattern with methods for hybrid embedding and token counting Includes SparseVector and HybridVector dataclasses for structured output +152/-0  pipeline.py Integrate hybrid embedder into processing pipeline              apps/ai-worker/src/pipeline.py Replaced Embedder import with HybridEmbedder Updated embedding call to use embed() returning HybridVector objects Changed chunk vector format to store dense and sparse vectors separately Added Phase 5 comment explaining hybrid vector format for Qdrant +13/-5    main.py Add hybrid embedding endpoints to AI worker API                    apps/ai-worker/src/main.py Added HybridEmbedder import at module level Updated /embed endpoint to use embed_dense_only() for backward compatibility New /embed/query endpoint for hybrid embeddings with dense + sparse vectors Returns HybridEmbedResponse with both vector types for Qdrant search +36/-4    models.py Add hybrid embedding response models                                          apps/ai-worker/src/models.py New Phase 5 models for hybrid embedding responses Added SparseVectorModel, HybridVectorModel, HybridEmbedRequest, HybridEmbedResponse Supports both dense (384 floats) and sparse (indices/values) vector formats +30/-0    SettingsPage.tsx Add system settings and health dashboard                                  apps/frontend/src/components/settings/SettingsPage.tsx New settings dashboard showing system health and configuration status Displays vector database provider, AI worker status, encryption, and OAuth configuration Real-time status monitoring with 30-second refresh interval Shows uptime, version, and Qdrant connection status +229/-0  GoogleOAuthSection.tsx Add Google OAuth connection UI component                                  apps/frontend/src/components/drive/GoogleOAuthSection.tsx New component for Google OAuth connection management Displays connection status and user email when connected Provides connect/disconnect buttons with loading states Handles OAuth callback detection via URL parameters +146/-0  AddFolderModal.tsx Integrate Google Drive Picker and sync frequency dropdown apps/frontend/src/components/drive/AddFolderModal.tsx Replaced manual folder ID input with Google Drive Picker integration Added useDrivePicker hook for native folder selection Replaced cron input with SyncFrequencySelect dropdown component Improved UX with visual feedback for selected folder and error handling +91/-42  results-list.tsx Add Qdrant provider badge to search results                            apps/frontend/src/components/query/results-list.tsx Added provider prop to display Qdrant badge when applicable Added "Powered by Qdrant" badge with rocket icon for Qdrant searches Updated score display to use percentage format (0-100) consistently Removed alpha parameter from hybrid search display +34/-26  search-form.tsx Remove manual alpha tuning from search form                            apps/frontend/src/components/query/search-form.tsx Removed alpha slider for hybrid search balance control Removed alpha state management and parameter passing Simplified search mode toggle (semantic/hybrid without manual tuning) Updated ResultsList to pass provider instead of alpha +2/-28    SyncFrequencySelect.tsx New sync frequency dropdown component                                        apps/frontend/src/components/drive/SyncFrequencySelect.tsx New component for selecting sync frequency with preset cron intervals Provides dropdown options ranging from 15 minutes to daily, plus manual-only option Includes accessible label and helper text describing the sync frequency purpose Styled with Tailwind CSS for consistent UI appearance +46/-0    AnalyticsPage.tsx Add Qdrant sync queue analytics display                                    apps/frontend/src/components/analytics/AnalyticsPage.tsx Added Phase 5J Qdrant Sync Queue analytics section to overview dashboard Displays three metrics in grid layout: PENDING, SYNCED, and FAILED counts Uses color-coded badges (yellow, green, red) for visual status indication Includes icon and formatted number display for each sync queue status +30/-0    App.tsx Integrate dedicated settings page component                            apps/frontend/src/App.tsx Imported new SettingsPage component from settings module Replaced inline placeholder settings UI with dedicated SettingsPage component Removed hardcoded "No settings configured yet" message and Phase 5 placeholder text +2/-14    DriveSyncTab.tsx Add Google OAuth section and folder name support                  apps/frontend/src/components/drive/DriveSyncTab.tsx Added import for new GoogleOAuthSection component Updated handleCreate function to accept and pass folderName parameter Integrated GoogleOAuthSection component at top of drive sync tab Enhanced folder creation to include folder name from picker +6/-1      ChunkCard.tsx Add sync status badge to chunk cards                                          apps/frontend/src/components/chunks/ChunkCard.tsx Added Phase 5H sync status badge display in chunk card footer Shows status indicators for SYNCED, PENDING, and FAILED states Uses color-coded styling with appropriate icons and text labels Conditionally renders badge only when syncStatus is present +13/-0
Refactoring	2 files event-bus.ts Rename drive config event to drive folder                                apps/backend/src/services/event-bus.ts Renamed event type from driveConfig:deleted to driveFolder:deleted Updated comment to reflect folder terminology +1/-1      cron.ts Update cron jobs to use DriveFolder model                                apps/backend/src/jobs/cron.ts Updated comment from "DriveConfigs" to "DriveFolders" Changed Prisma query from driveConfig.findMany() to driveFolder.findMany() +2/-2
Configuration changes	3 files vite-env.d.ts Add Vite environment variable type definitions                      apps/frontend/src/vite-env.d.ts New TypeScript environment variable definitions for Vite Defines VITE_API_URL, VITE_GOOGLE_PICKER_API_KEY, and VITE_GOOGLE_CLIENT_ID +11/-0    schema.prisma Add Qdrant sync status and OAuth encryption to schema        apps/backend/prisma/schema.prisma Renamed DriveConfig model to DriveFolder with updated field mappings Added ChunkSyncStatus enum (PENDING, SYNCED, FAILED) for Qdrant sync tracking Added vector fields to Chunk: syncStatus, denseVector, sparseIndices, sparseValues New DriveOAuth model for encrypted OAuth token storage with AES-256-GCM fields Updated relationships and indexes to reflect new model names +34/-10  .env.example Add frontend environment configuration template                    apps/frontend/.env.example New environment configuration template file for frontend Defines VITE_API_URL for backend API endpoint Includes Google Drive Picker API key and Google Client ID configuration Provides example values with instructions for setup +8/-0
Documentation	7 files roadmap-phase5.md Refactor Phase 5 roadmap to production infrastructure        docs/roadmap-phase5.md Complete rewrite from Phase 5 SaaS focus to Phase 5 Production Infrastructure New focus on Qdrant hybrid search (dense + sparse vectors) and AES-256-GCM encryption Detailed implementation of outbox pattern for vector sync Moved SaaS features (Supabase, Stripe) to Phase 6 +149/-363 roadmap-phase6.md Add Phase 6 multi-tenant SaaS roadmap                                        docs/roadmap-phase6.md New file containing Phase 6 SaaS platform roadmap Includes Supabase auth, Stripe billing, multi-tenant architecture Per-user Drive OAuth with Phase 5 encryption prerequisites Comprehensive scope, pricing tiers, and implementation order +416/-0  detailed-plan-phase5.md Add detailed Phase 5 implementation plan                                  docs/detailed-plan-phase5.md New comprehensive implementation guide for Phase 5 Covers 12 parts: AES encryption, sparse embeddings, Qdrant integration, outbox pattern Includes data flow diagrams, schema changes, environment variables Details OAuth flow, folder picker, sync frequency UI, and success criteria +382/-0  roadmap.md Update roadmap with Phase 5 and Phase 6                                    docs/roadmap.md Updated phase count from 4 to 6 phases Added Phase 5 (Production Infrastructure) and Phase 6 (SaaS) to roadmap Updated technology stack to include Qdrant and AES-256-GCM Updated format support versions from Phase 3 to Phase 4 +61/-32  architecture.md Update architecture documentation for Phase 5                        docs/architecture.md Updated to Phase 5 complete status (Dec 31, 2025) Added Qdrant Cloud as new container in architecture Updated vector storage section to describe hybrid search (dense + sparse) Explained outbox pattern for vector sync and nullification Updated Drive sync to use OAuth 2.0 instead of service account +19/-14  api.md Update API contracts for Phase 5 changes                                  docs/api.md Updated to Phase 5 complete status (Dec 31, 2025) Added syncStatus field to Chunk interface Renamed DriveConfig to DriveFolder in API contracts New DriveOAuth interface for encrypted token storage Updated search documentation to describe Qdrant hybrid search with RRF +25/-7    data-flow.md Update data flow documentation for Qdrant search                  docs/data-flow.md Updated retrieval section to describe Qdrant hybrid search flow Replaced SQL-based search examples with Qdrant query structure Explained RRF fusion with prefetch sparse + dense query Removed alpha parameter documentation (now handled by Qdrant) +23/-19
Dependencies	3 files pnpm-lock.yaml Add Qdrant client library dependencies                                      pnpm-lock.yaml Added @qdrant/js-client-rest@1.16.2 dependency for Qdrant integration Added @qdrant/openapi-typescript-fetch@1.2.6 as transitive dependency Added undici@6.22.0 for HTTP client support +27/-0    package.json Add Qdrant JavaScript client dependency                                    apps/backend/package.json Added @qdrant/js-client-rest dependency version ^1.16.2 Enables Qdrant vector database integration for backend services +1/-0      requirements.txt Add fastembed dependency for hybrid embeddings                      apps/ai-worker/requirements.txt Added fastembed>=0.4.0 dependency for Phase 5 hybrid embeddings Enables dense and sparse embedding support for AI worker +3/-0
Bug fix	1 files document-list.tsx Update document list filter parameter naming                          apps/frontend/src/components/documents/document-list.tsx Changed query parameter from driveConfigId to driveFolderId Updates document list filtering to use folder ID instead of config ID +1/-1
Additional files	8 files embedder.py +0/-60    test_embedder.py +0/-76    hybrid-search-init.ts +0/-88    hybrid-search.ts +0/-215  query-validator.ts +0/-5      hybrid-search.test.ts +0/-205  query-validator.test.ts +0/-26    extension-hybrid-search.md +0/-207

---

[qodo-code-review]

PR Compliance Guide 🔍

Below is a summary of compliance checks for this PR:

Security Compliance
⚪	OAuth CSRF risk Description: The Google OAuth flow is initiated without an anti-CSRF state parameter (and no PKCE), enabling realistic OAuth CSRF/login-swapping where an attacker can trick a victim into completing the callback with the attacker's authorization code, causing the server to store the attacker-controlled refresh token under the shared driveOAuth record. google.route.ts [45-136] Referred Code fastify.get('/api/oauth/google/start', async (request, reply) => { try { const oauth2Client = getOAuth2Client(); const authUrl = oauth2Client.generateAuthUrl({ access_type: 'offline', scope: SCOPES, prompt: 'consent', // Force consent to get refresh token }); logger.info('oauth_start_redirect'); return reply.redirect(authUrl); } catch (error: any) { logger.error({ error: error.message }, 'oauth_start_failed'); return reply.status(500).send({ error: 'OAUTH_CONFIG_ERROR', message: error.message, }); } }); ... (clipped 71 lines)
	Token endpoint exposure Description: The /api/oauth/google/access-token endpoint returns a valid Google access token derived from the stored refresh token without any visible authentication/authorization checks in this route, which could allow any caller who can reach the endpoint to mint access tokens and access the connected Google Drive scope. google.route.ts [178-223] Referred Code fastify.get('/api/oauth/google/access-token', async (request, reply) => { try { const oauth = await prisma.driveOAuth.findUnique({ where: { id: 'system' }, }); if (!oauth || !oauth.encryptedRefreshToken) { return reply.status(401).send({ error: 'NOT_CONNECTED', message: 'Google account not connected. Please connect first.', }); } // Decrypt refresh token const encryption = getEncryptionService(); const refreshToken = encryption.decrypt({ ciphertext: oauth.encryptedRefreshToken, iv: oauth.tokenIv, authTag: oauth.tokenAuthTag, }); ... (clipped 25 lines)
Ticket Compliance
⚪	🎫 No ticket provided Create ticket/issue
Codebase Duplication Compliance
⚪	Codebase context is not defined Follow the guide to enable codebase context checks.
Custom Compliance
🟢	Generic: Meaningful Naming and Self-Documenting Code Objective: Ensure all identifiers clearly express their purpose and intent, making code self-documenting Status: Passed Learn more about managing compliance generic rules or creating your own custom rules
🔴	Generic: Comprehensive Audit Trails Objective: To create a detailed and reliable record of critical system actions for security analysis and compliance. Status: Missing user context: Critical OAuth actions are logged without any user identifier (and are stored under a shared driveOAuth record), preventing reconstruction of which user initiated connect/disconnect or token issuance. Referred Code fastify.get('/api/oauth/google/start', async (request, reply) => { try { const oauth2Client = getOAuth2Client(); const authUrl = oauth2Client.generateAuthUrl({ access_type: 'offline', scope: SCOPES, prompt: 'consent', // Force consent to get refresh token }); logger.info('oauth_start_redirect'); return reply.redirect(authUrl); } catch (error: any) { logger.error({ error: error.message }, 'oauth_start_failed'); return reply.status(500).send({ error: 'OAUTH_CONFIG_ERROR', message: error.message, }); } }); ... (clipped 181 lines) Learn more about managing compliance generic rules or creating your own custom rules
	Generic: Robust Error Handling and Edge Case Management Objective: Ensure comprehensive error handling that provides meaningful context and graceful degradation Status: Swallowed exceptions: The Drive folder lookup silently swallows all exceptions and returns null, removing actionable error context and hindering production debugging/monitoring. Referred Code async getFolder(folderId: string): Promise<{ id: string; name: string } | null> { try { const response = await this.drive.files.get({ fileId: folderId, fields: 'id, name, mimeType', }); if (response.data.mimeType !== 'application/vnd.google-apps.folder') { return null; } return { id: response.data.id!, name: response.data.name!, }; } catch { return null; } } Learn more about managing compliance generic rules or creating your own custom rules
	Generic: Secure Error Handling Objective: To prevent the leakage of sensitive system information through error messages while providing sufficient detail for internal debugging. Status: Leaky user errors: User-facing 503 responses include raw upstream error.message strings (e.g., Qdrant search failed: ...), potentially exposing internal system details to clients. Referred Code const { query, topK, mode } = input.data; // Phase 5: Qdrant-only search (mandatory) if (!shouldUseQdrantSearch()) { return reply.status(503).send({ error: 'SEARCH_UNAVAILABLE', message: 'Qdrant is not configured. Please set QDRANT_URL and QDRANT_API_KEY.', }); } try { const qdrantResults = await qdrantHybridSearchService.search({ queryText: query, topK, mode: mode as 'semantic' | 'hybrid', }); return reply.send({ mode: mode === 'hybrid' ? 'qdrant_hybrid' : 'qdrant_semantic', provider: 'qdrant', results: qdrantResults.map(r => ({ ... (clipped 13 lines) Learn more about managing compliance generic rules or creating your own custom rules
	Generic: Secure Logging Practices Objective: To ensure logs are useful for debugging and auditing without exposing sensitive information like PII, PHI, or cardholder data. Status: Sensitive query logging: Logs emit the full Drive query string (including folder identifiers) which can leak sensitive resource identifiers into logs and should be minimized or redacted. Referred Code do { const query = `'${currentFolderId}' in parents and trashed = false`; logger.debug({ query }, 'listAllFiles_query'); Learn more about managing compliance generic rules or creating your own custom rules
	Generic: Security-First Input Validation and Data Handling Objective: Ensure all data inputs are validated, sanitized, and handled securely to prevent vulnerabilities Status: Missing authz checks: OAuth connect/status/access-token/disconnect endpoints operate on a shared system credential without any authentication/authorization guard, enabling any caller to connect/disconnect or request access tokens. Referred Code export async function oauthRoutes(fastify: FastifyInstance): Promise<void> { const prisma = getPrismaClient(); /** * GET /api/oauth/google/start - Initiate OAuth flow * * Redirects user to Google consent screen. * After consent, Google redirects back to /api/oauth/google/callback. */ fastify.get('/api/oauth/google/start', async (request, reply) => { try { const oauth2Client = getOAuth2Client(); const authUrl = oauth2Client.generateAuthUrl({ access_type: 'offline', scope: SCOPES, prompt: 'consent', // Force consent to get refresh token }); logger.info('oauth_start_redirect'); return reply.redirect(authUrl); ... (clipped 191 lines) Learn more about managing compliance generic rules or creating your own custom rules

Compliance status legend

🟢 - Fully Compliant
🟡 - Partial Compliant
🔴 - Not Compliant
⚪ - Requires Further Human Verification
🏷️ - Compliance label

[qodo-code-review]

PR Code Suggestions ✨

Explore these optional code suggestions:

Category	Suggestion	Impact
High-level	Restore search tests with testcontainers The PR disables all integration and E2E tests for the search feature due to the lack of a Qdrant instance in the test environment. It is suggested to use Testcontainers to create a temporary Qdrant instance during testing, which would allow for the re-enabling and continuous validation of these crucial tests. Examples: apps/backend/tests/integration/routes/search-route.test.ts [18] describe.skip('POST /api/query (Requires Qdrant)', () => { apps/backend/tests/e2e/query-flow.test.ts [20] describe.skip('E2E: Query Flow (Requires Qdrant)', () => { Solution Walkthrough: Before: // File: apps/backend/tests/integration/routes/search-route.test.ts /** * NOTE: These tests are SKIPPED because search now requires Qdrant (Phase 5). * Qdrant is not available in the test environment... */ describe.skip('POST /api/query (Requires Qdrant)', () => { let app: any; beforeAll(async () => { app = await createTestApp(); }); // ... all tests are skipped it('should return results for a valid query', async () => { // ... test logic is present but not executed }); }); After: // File: apps/backend/tests/integration/routes/search-route.test.ts import { QdrantContainer, StartedQdrantContainer } from "@testcontainers/qdrant"; // Re-enable the test suite describe('POST /api/query (Requires Qdrant)', () => { let app: any; let qdrantContainer: StartedQdrantContainer; beforeAll(async () => { // Start a Qdrant container for the test run qdrantContainer = await new QdrantContainer().start(); process.env.QDRANT_URL = qdrantContainer.getHttpUrl(); app = await createTestApp(); }, 60000); // ... all tests are now executed against the temporary Qdrant instance }); Suggestion importance[1-10]: 9 __ Why: This suggestion addresses a critical quality assurance gap by proposing a viable solution (Testcontainers) to re-enable the entire disabled search test suite, which is crucial for a core, rewritten feature.	High
Security	Revoke Google token on disconnect Revoke the Google OAuth token via the revokeToken endpoint when a user disconnects, instead of only deleting it from the local database, to fully terminate access and mitigate security risks. apps/backend/src/routes/oauth/google.route.ts [230-247] fastify.post('/api/oauth/google/disconnect', async (request, reply) => { try { + const oauth = await prisma.driveOAuth.findUnique({ where: { id: 'system' } }); + if (oauth?.encryptedRefreshToken) { + try { + const encryption = getEncryptionService(); + const refreshToken = encryption.decrypt({ + ciphertext: oauth.encryptedRefreshToken, + iv: oauth.tokenIv, + authTag: oauth.tokenAuthTag, + }); + const oauth2Client = getOAuth2Client(); + await oauth2Client.revokeToken(refreshToken); + logger.info('oauth_token_revoked'); + } catch (revokeError: any) { + logger.warn({ error: revokeError.message }, 'oauth_revoke_failed_continuing_disconnect'); + } + } + await prisma.driveOAuth.delete({ where: { id: 'system' }, }).catch(() => { // Ignore if not found }); logger.info('oauth_disconnect_success'); return reply.send({ success: true }); } catch (error: any) { logger.error({ error: error.message }, 'oauth_disconnect_failed'); return reply.status(500).send({ error: 'OAUTH_DISCONNECT_ERROR', message: error.message, }); } }); Apply / Chat Suggestion importance[1-10]: 9 __ Why: This suggestion addresses a security vulnerability by ensuring the Google OAuth token is revoked, not just deleted locally, which prevents dangling access to the user's account.	High
Possible issue	Correct non-recursive file listing Fix the non-recursive sync logic by calling listFiles instead of listAllFiles to fetch only the immediate contents of a folder, respecting the recursive flag. apps/backend/src/services/sync-service.ts [148-150] const driveFiles = recursive ? await drive.listAllFiles(folderId) - : await drive.listAllFiles(folderId); // Use listAllFiles for both cases + : (await drive.listFiles(folderId, undefined, false)).files; Apply / Chat Suggestion importance[1-10]: 9 __ Why: This suggestion correctly identifies and fixes a regression where the non-recursive sync option was broken, ensuring the system respects the user's configuration.	High
	Avoid redundant database query execution Remove the redundant database query for syncStatusCounts and the incorrect type casting of formatCounts. Instead, capture and use the result from the existing Promise.all call. apps/backend/src/routes/analytics/overview-route.ts [111-128] // Build sync queue stats (Phase 5J) -const syncQueueRaw = formatCounts as unknown as Array<{ syncStatus: string | null; _count: number }>; -// Re-query to get actual sync stats from the last query const syncQueue = { PENDING: 0, SYNCED: 0, FAILED: 0, }; // Get from the last element of the Promise.all result (syncStatusCounts) -const syncStatusCounts = await prisma.chunk.groupBy({ - by: ['syncStatus'], - _count: true, -}); for (const item of syncStatusCounts) { if (item.syncStatus && item.syncStatus in syncQueue) { syncQueue[item.syncStatus as keyof typeof syncQueue] = item._count; } } Apply / Chat Suggestion importance[1-10]: 7 __ Why: The suggestion correctly identifies a redundant database query and an incorrect type cast, improving performance and correctness, but another suggestion provides a more complete code fix.	Medium
	Align cron job to predictable intervals *Change the cron string for the 15-minute sync frequency to 0,15,30,45 * * * to ensure it runs at predictable quarter-hour intervals. apps/frontend/src/components/drive/SyncFrequencySelect.tsx [13-20] const FREQUENCY_OPTIONS = [ - { label: 'Every 15 minutes', value: '*/15 * * * *' }, + { label: 'Every 15 minutes', value: '0,15,30,45 * * * *' }, { label: 'Every hour', value: '0 * * * *' }, { label: 'Every 6 hours', value: '0 */6 * * *' }, { label: 'Every 12 hours', value: '0 */12 * * *' }, { label: 'Daily', value: '0 0 * * *' }, { label: 'Manual only', value: '' }, ]; [To ensure code accuracy, apply this suggestion manually] Suggestion importance[1-10]: 5 __ Why: The suggestion correctly identifies that the cron string */15 * * * * leads to unpredictable job start times and proposes a change for more intuitive, clock-aligned scheduling, which improves the feature's usability.	Low
General	Use NULL to clear vector data Use NULL instead of empty arrays ([]) to clear vector data in the chunk table after syncing to Qdrant, which more effectively frees up disk space in PostgreSQL. apps/backend/src/queue/qdrant-sync.processor.ts [176-184] await prisma.chunk.updateMany({ where: { id: { in: validChunks.map((c) => c.id) } }, data: { syncStatus: 'SYNCED', - denseVector: [], // Clear to save space - sparseIndices: [], - sparseValues: [], + denseVector: null, // Clear to save space + sparseIndices: null, + sparseValues: null, }, }); Apply / Chat Suggestion importance[1-10]: 6 __ Why: The suggestion correctly points out that using NULL is better for reclaiming storage in PostgreSQL than using empty arrays, which improves database space management.	Low
	Require embedding or vector Add a .refine() rule to the ProcessingResultSchema to enforce that every item in the chunks array contains either an embedding or a vector property. apps/backend/src/validators/callback-validator.ts [46-57] const ProcessingResultSchema = z.object({ processedContent: z.string().optional(), chunks: z.array(z.object({ content: z.string(), index: z.number(), embedding: z.array(z.number()).optional(), // Legacy: 384d dense only vector: HybridVectorSchema.optional(), // Phase 5: dense + sparse metadata: ChunkMetadataSchema.optional() })).optional(), pageCount: z.number().int().nonnegative(), }) +.refine(data => + data.chunks?.every(chunk => chunk.embedding !== undefined || chunk.vector !== undefined), + { + message: "Each chunk must include either `embedding` or `vector`", + path: ["chunks"], + } +); [To ensure code accuracy, apply this suggestion manually] Suggestion importance[1-10]: 6 __ Why: The suggestion improves data validation by ensuring that every chunk contains at least one of the two possible embedding formats, which prevents processing chunks without vector data.	Low
	Improve token count estimation accuracy Improve the accuracy of the get_token_counts method by using re.split(r'\s+', text.strip()) instead of text.split() to handle various whitespace patterns more reliably. apps/ai-worker/src/hybrid_embedder.py [123-140] def get_token_counts(self, texts: List[str]) -> List[int]: """ Estimate token counts for texts. Uses a simple heuristic based on word count. For more accurate counts, use the dense model's tokenizer. """ + import re + if not texts: return [] # Simple estimation: ~0.75 tokens per word (typical for English) # This is faster than loading tokenizer for each call counts = [] for text in texts: - word_count = len(text.split()) + word_count = len(re.split(r'\s+', text.strip())) # Cap at 512 (model max) counts.append(min(int(word_count * 1.3), 512)) return counts Apply / Chat Suggestion importance[1-10]: 4 __ Why: The suggestion correctly points out a minor inaccuracy in the token counting heuristic and proposes a more robust method using regex, which slightly improves the estimation's reliability.	Low
	Display search score as a percentage Append a percentage symbol ('%') to the displayed search score to improve clarity for the user. apps/frontend/src/components/query/results-list.tsx [96-98] <span className="text-sm font-medium"> - {(result.score * 100).toFixed(1)} + {`${(result.score * 100).toFixed(1)}%`} </span> [To ensure code accuracy, apply this suggestion manually] Suggestion importance[1-10]: 4 __ Why: This suggestion correctly points out that removing the '%' symbol from the score display makes it less clear for users; adding it back improves UI clarity and consistency.	Low
More