RAG: Add retrieval quality metrics and evaluation

[sfloess]

Part of #2: Add RAG (Retrieval Augmented Generation) with citations

Scope

Implement retrieval quality metrics to measure and monitor the effectiveness of the RAG pipeline across its stages.

Details

• Retrieval metrics: Precision@K, Recall@K, MRR (Mean Reciprocal Rank), NDCG
• Reranking metrics: Precision lift over base retrieval
• Generation metrics: Citation coverage, answer groundedness, faithfulness score
• End-to-end metrics: Query latency (per stage), total pipeline latency
• Logging and reporting infrastructure for tracking quality over time
• Optional integration with existing Prometheus exporter for dashboarding

Metrics Output

{
    "retrieval": {"precision_at_5": 0.80, "mrr": 0.75, "ndcg": 0.82},
    "reranking": {"precision_lift": 0.15},
    "generation": {"coverage": 0.88, "faithfulness": 0.91},
    "latency": {"retrieval_ms": 45, "rerank_ms": 120, "generation_ms": 1200, "total_ms": 1365}
}

Acceptance Criteria

• Retrieval metrics (Precision@K, MRR, NDCG) are computed per query
• Reranking lift is measured against base retrieval
• Generation quality metrics (coverage, faithfulness) are reported
• Per-stage latency is tracked
• Metrics can be logged to file or exported to monitoring stack
• Unit tests for metric computation functions