Skip to content

Commit 086d5a3

Browse files
committed
fix: Ensure StillMe mentions correct embedding model and improve tracking visibility
Model Information: - Create stillme_core/rag/model_info.py as single source of truth - Add explicit warnings in prompts to prevent mentioning old model (all-MiniLM-L6-v2) - Ensure StillMe always mentions paraphrase-multilingual-MiniLM-L12-v2 Self-Tracking Visibility: - Log task tracking at info level (not just debug) for visibility - StillMe can now see its own tracking in logs - Helps verify that self-tracking is working correctly
1 parent 6ae4e51 commit 086d5a3

3 files changed

Lines changed: 46 additions & 4 deletions

File tree

backend/identity/prompt_builder.py

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -525,7 +525,7 @@ def _build_stillme_instruction(self, detected_lang: str) -> str:
525525
2. **KHÔNG có mechanical disclaimer**: Nếu bạn có foundational knowledge trong context, KHÔNG nói "Dựa trên kiến thức tổng quát (không từ cơ sở tri thức RAG)" - bạn ĐÃ CÓ StillMe knowledge trong context!
526526
3. **Mention cụ thể về StillMe features**: Khi giải thích sự khác biệt của StillMe, bạn PHẢI mention:
527527
- **RAG (Retrieval-Augmented Generation)**: StillMe sử dụng RAG với ChromaDB vector database
528-
- **Embedding Model**: paraphrase-multilingual-MiniLM-L12-v2 (384 dimensions, tối ưu cho multilingual Q&A retrieval, hỗ trợ 50+ ngôn ngữ)
528+
- **Embedding Model**: paraphrase-multilingual-MiniLM-L12-v2 (384 dimensions, tối ưu cho multilingual Q&A retrieval, hỗ trợ 50+ ngôn ngữ) - CRITICAL: Đây là model hiện tại, KHÔNG phải all-MiniLM-L6-v2
529529
- **Continuous Learning**: StillMe học tự động mỗi 4 giờ (6 lần/ngày) từ RSS feeds, arXiv, CrossRef, và Wikipedia
530530
- **Validation Chain**: Multi-layer validation (CitationRequired, EvidenceOverlap, ConfidenceValidator, FactualHallucinationValidator, FallbackHandler) để giảm hallucinations 80%
531531
- **Transcends Knowledge Cutoff**: StillMe KHÔNG bị giới hạn bởi training data cutoff dates - nó cập nhật knowledge liên tục qua RAG
@@ -536,7 +536,7 @@ def _build_stillme_instruction(self, detected_lang: str) -> str:
536536
537537
**1. RAG Architecture:**
538538
- StillMe sử dụng RAG với ChromaDB làm vector database
539-
- Content được embed bằng paraphrase-multilingual-MiniLM-L12-v2 model
539+
- Content được embed bằng paraphrase-multilingual-MiniLM-L12-v2 model (KHÔNG phải all-MiniLM-L6-v2)
540540
- 384-dimensional embeddings tối ưu cho multilingual Q&A retrieval
541541
- Khi trả lời, StillMe tìm kiếm ChromaDB bằng semantic similarity
542542
@@ -582,7 +582,7 @@ def _build_stillme_instruction(self, detected_lang: str) -> str:
582582
2. **NO mechanical disclaimer**: If you have foundational knowledge in context, DO NOT say "Based on general knowledge (not from StillMe's RAG knowledge base)" - you HAVE StillMe knowledge in context!
583583
3. **Mention SPECIFIC StillMe features**: When explaining StillMe's differences, you MUST mention:
584584
- **RAG (Retrieval-Augmented Generation)**: StillMe uses RAG with ChromaDB vector database
585-
- **Embedding Model**: paraphrase-multilingual-MiniLM-L12-v2 (384 dimensions, optimized for multilingual Q&A retrieval, supports 50+ languages)
585+
- **Embedding Model**: paraphrase-multilingual-MiniLM-L12-v2 (384 dimensions, optimized for multilingual Q&A retrieval, supports 50+ languages) - CRITICAL: This is the CURRENT model, NOT all-MiniLM-L6-v2
586586
- **Continuous Learning**: StillMe learns automatically every 4 hours (6 cycles/day) from RSS feeds, arXiv, CrossRef, and Wikipedia
587587
- **Validation Chain**: Multi-layer validation (CitationRequired, EvidenceOverlap, ConfidenceValidator, FactualHallucinationValidator, FallbackHandler) to reduce hallucinations by 80%
588588
- **Transcends Knowledge Cutoff**: StillMe is NOT limited by training data cutoff dates - it continuously updates knowledge through RAG
@@ -593,7 +593,7 @@ def _build_stillme_instruction(self, detected_lang: str) -> str:
593593
594594
**1. RAG Architecture:**
595595
- StillMe uses RAG with ChromaDB as vector database
596-
- Content is embedded using paraphrase-multilingual-MiniLM-L12-v2 model
596+
- Content is embedded using paraphrase-multilingual-MiniLM-L12-v2 model (NOT all-MiniLM-L6-v2)
597597
- 384-dimensional embeddings optimized for multilingual Q&A retrieval
598598
- When answering, StillMe searches ChromaDB using semantic similarity
599599

stillme_core/monitoring/self_tracking.py

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -80,6 +80,9 @@ def track_task_execution(
8080
estimate_text = estimator.format_estimate(estimate)
8181
logger.info(f"📊 StillMe self-estimate: {estimate_text}")
8282
logger.info(f" I'm an AI system that tracks my own performance to improve estimates over time.")
83+
else:
84+
# Still log at debug level for internal tracking visibility
85+
logger.debug(f"📊 StillMe tracking: {task_description} (estimate: {estimate.estimated_minutes:.2f} min, confidence: {estimate.confidence:.0%})")
8386

8487
try:
8588
# Yield estimate for use in task

stillme_core/rag/model_info.py

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
"""
2+
Model Information for StillMe
3+
4+
Provides accurate model information that StillMe can use in responses.
5+
This ensures StillMe always mentions the correct model names and versions.
6+
"""
7+
8+
# CRITICAL: This is the SINGLE SOURCE OF TRUTH for embedding model information
9+
# If model changes, update this file and re-run foundational knowledge update
10+
11+
EMBEDDING_MODEL_NAME = "paraphrase-multilingual-MiniLM-L12-v2"
12+
EMBEDDING_MODEL_DIMENSIONS = 384
13+
EMBEDDING_MODEL_DESCRIPTION = "sentence-transformers model optimized for multilingual Q&A retrieval, supports 50+ languages"
14+
15+
def get_embedding_model_info() -> dict:
16+
"""
17+
Get current embedding model information.
18+
19+
Returns:
20+
Dictionary with model information:
21+
- name: Model name
22+
- dimensions: Embedding dimensions
23+
- description: Human-readable description
24+
"""
25+
return {
26+
"name": EMBEDDING_MODEL_NAME,
27+
"dimensions": EMBEDDING_MODEL_DIMENSIONS,
28+
"description": EMBEDDING_MODEL_DESCRIPTION
29+
}
30+
31+
def get_embedding_model_display_name() -> str:
32+
"""
33+
Get formatted model name for display in responses.
34+
35+
Returns:
36+
Formatted string: "paraphrase-multilingual-MiniLM-L12-v2 (384 dimensions, ...)"
37+
"""
38+
return f"{EMBEDDING_MODEL_NAME} ({EMBEDDING_MODEL_DIMENSIONS} dimensions, {EMBEDDING_MODEL_DESCRIPTION})"
39+

0 commit comments

Comments
 (0)