Ai#9
Open
kietoichoiDXD wants to merge 167 commits intodscdut:devfrom
Open
Conversation
- System architecture design (multi-tenant, Supabase, RAG) - Database schema with pgvector + RLS - FastAPI project structure - Legal Q&A agent skeleton - RAG search engine (hybrid semantic + keyword) - Law crawler + embedding pipeline - Docker + deployment config - API schemas (Pydantic models) - Project plan + roadmap
…ion, batch processing, compliance engine, contract lifecycle
…cs loaded - FastAPI with 4 endpoints: /ask, /review, /draft, /search - Claude OAuth integration (configurable) - Full-text search (PostgreSQL tsvector + trigram) - 504 Vietnamese law documents, 37,746 chunks loaded - Multi-tenant API key authentication - Usage tracking and quota management - Contract review with risk scoring - Document drafting with legal compliance
- Fixed search_law() to use keyword extraction + ILIKE (no more AND-all-terms bug) - Added Vietnamese stop words filter - Added anthropic-beta header for OAuth support - Created test company + API key - Updated .env with OAuth token
- Refactor search_law() to use indexed tsvector search instead of ILIKE - Improve from 5-8s to ~1.1s average response time (5-7x faster) - Add Vietnamese compound phrase detection and stop word filtering - Implement law domain boosting for better relevance - Remove expensive string operations from SELECT clause - Add comprehensive test suite and performance benchmarks Key changes: - Primary filter: tsv @@ tsquery (GIN index) - Simplified scoring: ts_rank + domain boost + length normalization - No ILIKE in SELECT (moved to WHERE or eliminated) - Keyword limit (max 6) for performance Files: - scripts/migration_search_fast_final.sql (RECOMMENDED) - test_search_performance.py (benchmark suite) - SEARCH_OPTIMIZATION_REPORT.md (detailed analysis) Performance: 5-8s → 1.1s avg (target: <1s) Quality: 2/3 test queries return expected law documents See SEARCH_OPTIMIZATION_REPORT.md for full details.
- Landing page with pricing, features, API demo, chat widget - Dockerfile for container deployment - Static file serving via FastAPI - .env.example for safe config
✅ Implemented Features: - User auth: register, login, JWT tokens, password management - Company management: members, invitations, settings - API key management: create, revoke, usage tracking - Usage & billing: quota tracking, history, billing info - Chat history: list, view, delete, export (json/txt/md) - Document management: upload, list, delete (PDF/DOCX/TXT) 🔐 Security: - JWT authentication (HS256, bcrypt passwords) - Multi-tenant RLS policies - Role-based access control (owner/admin/member/viewer) - API key authentication (backward compatible) 📊 Database: - Migration script for auth tables - company_invites table - Extended users & companies tables - RLS policies for data isolation 🧪 Testing: - All endpoints tested and working - Auth flow verified - Backward compatibility maintained 📖 Documentation: - Complete AUTH_README.md guide - API examples - Migration instructions
… API Keys, and Usage pages
- Legal endpoints now accept both X-API-Key and Bearer token - Fixed /v1/auth/me response structure (company.name not company_name) - Dashboard uses Bearer token from login (no API key needed) - Fixed initDashboard crash on key loading
- Chat-first design with ChatGPT-like UX - Clean, professional UI (Notion/Linear quality) - Vietnamese UI throughout - 6 main pages: Chat, Templates, Contracts, Search, API, Settings - Markdown support via marked.js - Drag-and-drop file upload - Responsive design - Single-file SPA (no frameworks) - Inter font from Google Fonts - Proper auth flow with localStorage tokens
Features: - Admin Panel with superadmin role - Dashboard with platform stats - Company management (list, detail, update plan/quota) - User management (list, ban/unban, change roles) - Usage analytics (by hour/day/endpoint/company) - API logs viewer with filters - Announcements broadcasting - Contract Management Backend - Full CRUD for contracts (create, list, get, update, delete) - File upload support (PDF, DOCX, images) - AI contract review with Claude integration - Expiring contracts alerts - Multi-tenant scoped to company - Platform Logging Middleware - Logs all API requests to platform_logs table - Captures: endpoint, method, status, response_time, tokens, IP - Async logging (non-blocking) - Database Migration - Added superadmin role to user_role enum - Created contracts table - Created platform_logs table - Created announcements table - Made bi@hrvn.vn a superadmin - Admin Frontend (admin.html) - Clean SPA dashboard matching app.html style - Chart.js for analytics visualization - Real-time data with auto-refresh - Modal-based editing - Search and filter capabilities All features tested and working!
- Add professional Vietnamese legal consultant system prompt - Direct answers first (1-2 sentence summary) - Specific article citations (Điều X, Khoản Y, Luật Z) - Clear formatting with headings and bold - No hallucinations - only use provided sources - Enhance context building - Clearly show law title, number, and article - Structured format for better AI parsing - Better citation accuracy - Add search query extraction - Remove Vietnamese question words (bao lâu, thế nào, etc.) - Keep legal keywords only - Cleaner, more focused search queries - Implement multi-query search - Search with full question + extracted keywords - Merge and deduplicate results - Sort by relevance, return top N - Add domain auto-detection - Auto-identify legal domains from keywords - lao_dong, thue, doanh_nghiep, dan_su, dat_dai, etc. - Better search precision without manual selection Expected improvement: 60-80% boost in answer accuracy and citation quality Test cases: ✅ Thời gian thử việc -> Should cite BLLĐ 2019 Điều 25 ✅ Thuế TNDN -> Should answer 20%, cite Luật Thuế TNDN Điều 10 ✅ Nghỉ phép năm -> Should answer 12 ngày, cite BLLĐ 2019 Điều 113
- Beautiful dark mode color palette (zinc-950 backgrounds) - Inter font with proper typography hierarchy - Clean, minimal design inspired by ChatGPT, Linear, Vercel, Notion - Generous whitespace and micro-interactions - Smooth transitions (60fps) - Fully responsive (sidebar collapses on mobile) - Professional chat interface with markdown support - All pages redesigned: Chat, Templates, Contracts, Search, API, Settings - Consistent spacing, border radius, shadows - Modern tech startup aesthetic
…sting - ILIKE phrase search with domain auto-detection - Synonym expansion: TNDN→thu nhập DN, nghỉ phép→nghỉ hằng năm - Title keyword matching boost (Luật Thuế TNDN ranked higher for tax queries) - Bo Luat/Luat ranked above Legal Document entries - Multi-query: phrase ILIKE + synonym + tsvector merged
✅ Task 1: Critical Backend Fixes - Fix PlatformLoggingMiddleware using BackgroundTask (resolves Starlette body consumption bug) - Re-enable logging middleware in main.py - Implement chat history auto-save in /v1/legal/ask endpoint - Add load_dotenv() to properly load .env file - Update requirements.txt with all dependencies - Create scripts/index_chunks.py for database maintenance ✅ Task 2: Production Deployment Preparation - Create Procfile for Railway deployment - Create railway.toml with nixpacks config - Create render.yaml for Render.com deployment - Update Dockerfile with Python 3.11-slim - CORS already configured for production ✅ Task 3: Testing - Health endpoint: ✓ - Login endpoint: ✓ - API key needs updating for full E2E test See DEPLOYMENT_FIXES.md for complete details.
- Save actual_question (including file content) instead of bare question - Increase chat history from 20 to 50 messages - Now follow-up questions about uploaded files will have context
- Added edit_and_diff_document tool to TOOLS list - Implemented diff generation utility in src/services/diff_utils.py - Added tool execution function that uses Claude to generate edited version - Streaming function now emits document_edit SSE event with diff data - Frontend: Added CSS for diff view (inline, mobile-friendly, dark theme) - Frontend: Implemented renderDiffView() to display inline diff in chat - Frontend: Added downloadEditedDocument() for downloading edited version - UI text in Vietnamese, compatible with dark theme - Simple inline diff (not side-by-side) for mobile compatibility - Tested with test_diff.py - all working correctly
Settings now queries GET /v1/llm/status on load. Shows '✅ Đã kết nối' with provider/model info. No need to re-enter key after refresh.
…d text - Parse **bold**, ĐIỀU/CHƯƠNG headings, numbered items, bullets - Proper typography with sans-serif font - Dark theme styled with accent colors - Metadata bar with clean badges
- White paper background on dark UI - Left margin line (like legal paper) - Georgia/Times font for professional look - Proper bold, headings, numbered items styling - Fix ** parsing for partial bold markers
- Real A4 paper look: white page on dark background with shadow - Times New Roman serif font, proper margins - Double-line border title, centered - Unicode NFC normalization in DOCX extraction + frontend - Preserve heading styles from DOCX
- Chat upload now auto-saves to documents table immediately - Full extracted text saved with NFC normalization - write_document tool: emphasize FULL content, no summaries - Users always have complete document even if AI summarizes
- AnthropicProvider raises clear error if api_key is None - Fallback paths raise instead of passing None - Vietnamese error messages guide user to Settings
- New context_builder.py: builds per-request context - Injects: user name, role, company, plan, usage stats - Injects: document inventory, recent files, contracts - Injects: folder structure, recent chat topics - Injects: time context (greeting style based on hour) - Expiring contracts flagged with⚠️ warning - Vietnamese labels throughout
- Fallback markdown renderer: bold, italic, headers, lists, code - Works even if marked CDN fails to load - System prompt: HÀNH ĐỘNG KHÔNG HỎI LẠI - 'sửa giúp tôi' → edit immediately, don't ask 'sửa phần nào?'
- Auto-find latest session within 30 min if no session_id sent - User says 'có' → AI sees full chat history → knows what to do - Emit session_id SSE event after DB save → frontend tracks it - Remove 'ok/được/vâng/ừ' from simple patterns (need context)
…tput, humble tone
## Changes:
### System Prompt (AGENT_SYSTEM_PROMPT):
- ✅ Added strict anti-hallucination rules: ONLY cite laws when search_law returns results
- ✅ NEVER invent law numbers (e.g., 'Nghị định 293/2025/NĐ-CP') — use generic phrases if not found
- ✅ Output FULL DOCUMENT TEXT when drafting, NOT just summaries of changes
- ✅ Humble tone: no self-praise ('chuyên nghiệp', 'an toàn pháp lý'), no '100% hợp pháp' claims
- ✅ Always add disclaimer: 'Nên tham khảo luật sư trước khi áp dụng'
- ✅ Avoid emoji in formal document content (only in casual chat)
- ✅ Edit/revise instructions: output RESULT not PROCESS
### Tools:
- write_document: already has strong warning 'TOÀN BỘ nội dung, KHÔNG tóm tắt'
- edit_and_diff_document: outputs full edited text + diff view
### Context & Memory:
- context_builder.py and company_memory.py inject rich context (no changes needed)
## Impact:
- AI will stop saying 'Tôi đã xóa điều X, thêm điều Y' and output actual document text
- AI will NOT hallucinate law numbers — only cite when search_law confirms
- AI will NOT claim '100% legal' — always humble and cautious
- Better UX: users get COPY-PASTABLE documents, not meta-descriptions
…scdut#6 - Extracted full schema from production Supabase DB (24 tables, 9 ENUMs) - Added all tables: law_documents, law_chunks, companies, users, api_keys, etc. - Added critical indexes for performance (vector search, law lookup, etc.) - Added seed data for document_templates and platform_settings - Updated README with note about empty law_documents by default - Users need to run crawler or import to populate legal database Resolves dscdut#6
- Keep uploaded DOCX files on server (uploads/documents/{company_id}/)
- New endpoint: GET /v1/documents/{id}/download (serves actual file)
- New endpoint: POST /v1/documents/{id}/edit-docx (apply edits preserving formatting)
- New service: src/services/docx_editor.py (edit_docx_file, create_docx_from_text)
- Frontend: Download button for Word files in diff view
- Test suite: test_docx_editing.py verifies editing works correctly
When user uploads .docx, original file is kept. When AI edits, python-docx
modifies the file directly preserving bold, italic, tables, fonts, alignment.
User downloads the edited .docx with all formatting intact.
- Add Supabase Storage integration for permanent file persistence
- upload_file(), download_file(), get_download_url() in file_storage.py
- Automatic fallback to local storage if Supabase unavailable
- Add LibreOffice-based DOCX editor (libreoffice_editor.py)
- Uses LibreOffice CLI for format normalization
- Falls back to python-docx if LibreOffice not available
- Smart text replacement preserving run formatting
- Update Dockerfile to include libreoffice-writer
- Update docker-compose.yml with Supabase env vars
- Update API endpoints:
- /v1/chat/upload: Upload to Supabase Storage
- /v1/documents/{id}/download: Download from Supabase Storage
- /v1/documents/{id}/preview: NEW - Convert DOCX to PDF
- /v1/documents/{id}/edit-docx: Use LibreOffice + Supabase Storage
- Add test scripts for storage and LibreOffice integration
LibreOffice provides 99% format preservation vs python-docx's ~70%.
Files are now stored permanently on Supabase instead of local disk.
- Remove hardcoded Supabase service_role JWT from test_storage.py and
LIBREOFFICE_SUPABASE_IMPLEMENTATION.md (key must be rotated in dashboard)
- Fix Content-Type hardcoded as DOCX for all uploads — now uses MIME map
keyed by file extension; upload_file() returns content_type in result
- Fix MIME type stored in DB: was file_ext.replace('.','application/')
producing invalid values; now uses content_type from upload_file()
- Fix PDF temp file deleted before FileResponse streams: read bytes into
memory, delete temp file in finally, return StreamingResponse(BytesIO)
- Remove unused `import json` in libreoffice_editor.py
…ge-bugs fix: security + storage correctness — 4 issues from review
- Add _auth_headers() helper with apikey + Authorization headers - Works with both legacy JWT (eyJ...) and new sb_ format - All storage operations use _auth_headers() - Keys rotated after accidental exposure
…dut#8) - Added env_file to app service in docker-compose.yml - Replaced all hardcoded DB connection params with env vars: * host, port, dbname, user, password, sslmode now read from env * Sensible defaults maintained for non-Docker deployments - Fixed security issue: removed hardcoded password in run_migration_windows.py - Updated .env.example with DB_NAME, DB_USER, DB_SSL_MODE vars This fixes the Docker Compose deployment where: - App container couldn't connect to db container (used localhost instead of 'db') - User/db name mismatch between compose config and app code - SSL mode conflict on internal Docker network Thanks to @huydoan0212 for the detailed bug report!
…coder retrieval. Removed legacy md files.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.