Real-time monitoring and AI-powered insights for rural development projects
Features • Installation • AI RAG System • Architecture • Demo
RuraLens is a cutting-edge village digital twin platform that combines real-time IoT monitoring, 3D visualization, and AI-powered Retrieval-Augmented Generation (RAG) to provide intelligent insights for rural development schemes. The platform enables administrators, field workers, and citizens to ask natural language questions and receive accurate, citation-backed answers from government documents and live data.
Rural development projects often suffer from:
- ❌ Information scattered across multiple documents
- ❌ Lack of real-time visibility into scheme progress
- ❌ Difficulty identifying discrepancies and delays
- ❌ Limited citizen engagement and feedback
- ✅ AI-Powered Q&A: Ask questions in plain language, get instant answers
- ✅ Smart Citations: Every answer backed by verifiable document sources
- ✅ Real-Time Monitoring: Live IoT sensor data and scheme tracking
- ✅ Automated Compliance: AI analyzes vendor reports vs government plans
- ✅ Interactive 3D Maps: Visualize projects and sensor locations
The crown jewel of RuraLens - an intelligent question-answering system that understands context and provides accurate, sourced answers.
User Question → RAG Backend → Pathway MCP → Document Store
↓ ↓ ↓
PII Filter Vector Search Embeddings
↓ ↓ ↓
Rate Limit LLM Processing Knowledge Base
↓ ↓ ↓
Cache Check ← Answer Generated ← Top Results
↓
Citation Enrichment (Geo + Metadata)
↓
Frontend Display with Map Integration
| Feature | Description |
|---|---|
| 🔍 Natural Language Queries | Ask questions like "Why is Swachh Bharat delayed?" |
| 📚 Multi-Source Knowledge | Searches schemes, reports, sensor data, citizen feedback |
| 🎯 Smart Citations | Every answer includes source documents with relevance scores |
| 🗺️ Map Integration | Click citations to view locations on 3D map |
| ⚡ Lightning Fast | 120-second cache, sub-second responses |
| 🔒 PII Protection | Auto-redacts sensitive information (Aadhaar, PAN, emails) |
| 📊 Geo-Aware | Citations include precise coordinates with 4-level fallback |
Advanced infrastructure failure prediction using Graph Neural Networks to simulate cascading impacts across village infrastructure.
| Feature | Description |
|---|---|
| 🌐 Real Infrastructure Network | Auto-generates graph from water tanks, pumps, power nodes, buildings |
| 💥 Failure Simulation | Click any infrastructure node to trigger realistic failures |
| 🔄 Cascading Impact | AI predicts how failures propagate through connected infrastructure |
| 📊 Impact Visualization | Color-coded nodes show damage levels (green → yellow → orange → red) |
| ➕ Dynamic Node Addition | Right-click map to add new infrastructure nodes with auto-connections |
| 🔗 Smart Edge Generation | Nodes automatically connect based on type and proximity |
| 📈 Accumulated Damage | Multiple failures compound damage realistically |
| 🗺️ Interactive 3D Map | View entire infrastructure network on MapLibre GL with smooth animations |
Infrastructure Node Selected → Trigger Failure
↓
GNN API Analysis (or Local Simulation)
↓
Calculate Impact Propagation via Edges
↓
Score Each Connected Node (0-100%)
↓
Update Map Visualization
↓
Display in InfoPanel with Details
- View Network: Navigate to "Village Analyzer" from sidebar
- Explore Map: All infrastructure appears as labeled nodes on 3D map
- Trigger Failure: Click any node → Select failure type and severity
- Watch Propagation: See impacts spread through network in real-time
- Add Nodes: Right-click map → Add new infrastructure → Auto-connects
Scenario 1: Water Pump Failure
- Main Pump Station fails →
- Connected pipes show 60% impact →
- Hospital, School lose water supply (80% impact) →
- Consumer areas show reduced service (40-70%)
Scenario 2: Power Transformer Failure
- Transformer fails →
- All pumps lose power (90% impact) →
- Entire water system compromised →
- Critical buildings affected
Scenario 3: Multiple Cascading Failures
- First failure: Pump at 50% damage
- Second failure: Same pump now 56% damage (accumulated)
- Third failure: Pump cascades to failure (>90%)
Enhanced Visual Experience:
- Opaque Popups: Beautiful gradient backgrounds (slate-900 → slate-800) with glowing borders
- Color-Coded Infrastructure: Green (healthy) → Yellow (impacted) → Orange (severe) → Red (failed)
- Smooth Animations: Pulsing failed nodes, glowing impacts
- Detailed Tooltips: Hover over nodes to see health, type, and impact details
- Map Controls: Zoom, pitch, rotation, fullscreen, reset view
- Dual-Mode Display: Normal monitoring + Failure simulation modes
- Live Dashboard: Water quality, power consumption, scheme progress
- Alert System: Automatic notifications for critical infrastructure issues
- Citizen Feedback: Anonymous report system with GPS tracking
- Government Schemes: Track budget, timeline, and completion status
Q: "What water problems are reported in Zone B?"
A: Citizens report water pressure issues in Zone B, particularly during peak hours
(7-9 AM) affecting approximately 45 households. Sensor data confirms 30% pressure
drop during these times.
📍 Citations: [Sensor-042 (89% match), Citizen Report #127 (75% match)]
Q: "Why is MGNREGA road scheme delayed?"
A: The MGNREGA Rural Road Development is delayed by 14 days due to monsoon weather
impact and labor shortage. Vendor reported 40% workforce availability in October.
📍 Citations: [Vendor Report Phase-2 (92% match), Weather Log (78% match)]
Q: "Show me all schemes with budget overruns"
A: 2 schemes show budget variance: S-123 (+12% due to material cost increase),
S-456 (+8% from scope expansion approved in review).
📍 Citations: [Financial Report Q3 (94% match), Budget Analysis (81% match)]
┌─────────────────────────────────────────────────────────────────┐
│ Frontend (React) │
│ ┌──────────────┐ ┌──────────────┐ ┌────────────────────┐ │
│ │ Dashboard │ │ RAG Modal │ │ 3D Map Viewer │ │
│ │ Components │ │ with │ │ (MapLibre GL) │ │
│ │ │ │ Citations │ │ + GNN Viz │ │
│ └──────────────┘ └──────────────┘ └────────────────────┘ │
│ ┌──────────────┐ ┌──────────────┐ ┌────────────────────┐ │
│ │ Impact │ │ Failure │ │ Node Addition │ │
│ │ Predictor │ │ Popups │ │ Interface │ │
│ └──────────────┘ └──────────────┘ └────────────────────┘ │
└─────────────────────────────────────────────────────────────────┘
↓
┌─────────────────────────────────────────────────────────────────┐
│ Backend (Node.js/Express) │
│ ┌──────────────┐ ┌──────────────┐ ┌────────────────────┐ │
│ │ RAG Router │ │ PII Filter │ │ Cache Manager │ │
│ │ + Auth │ │ Sanitizer │ │ (120s TTL) │ │
│ └──────────────┘ └──────────────┘ └────────────────────┘ │
│ ┌──────────────┐ ┌──────────────┐ ┌────────────────────┐ │
│ │ Citation │ │ Geo Fallback│ │ Audit Logger │ │
│ │ Enrichment │ │ 4-Level │ │ + Trace IDs │ │
│ └──────────────┘ └──────────────┘ └────────────────────┘ │
│ ┌──────────────┐ ┌──────────────┐ ┌────────────────────┐ │
│ │ GNN Proxy │ │ Infrastructure│ │ Schemes API │ │
│ │ Client │ │ Graph Store │ │ Routes │ │
│ └──────────────┘ └──────────────┘ └────────────────────┘ │
└─────────────────────────────────────────────────────────────────┘
↓
┌─────────────────────────────────────────────────────────────────┐
│ AI Services Layer │
│ ┌────────────────────────────────────────────────────────────┐ │
│ │ Pathway MCP Server (Python/Rust) │ │
│ │ • DocumentStore: Schemes, Reports, Sensors, Feedback │ │
│ │ • VectorSearch: Embedding-based semantic search │ │
│ │ • LLM: OpenAI/Gemini for answer generation │ │
│ │ • REST API: /v1/pw_ai_answer endpoint │ │
│ └────────────────────────────────────────────────────────────┘ │
│ ┌────────────────────────────────────────────────────────────┐ │
│ │ GNN Service (Python) │ │
│ │ • Graph Neural Network for impact prediction │ │
│ │ • Node embedding and feature extraction │ │
│ │ • Cascading failure simulation │ │
│ │ • REST API: /api/gnn/predict-structured endpoint │ │
│ └────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────┘
↓
┌─────────────────────────────────────────────────────────────────┐
│ MongoDB Database │
│ Collections: schemes, users, vendorReports, citizenReports, │
│ feedback, gnnNodes, gnnEdges, infrastructureGraph │
└─────────────────────────────────────────────────────────────────┘
Frontend:
- ⚛️ React 18 with TypeScript
- 🎨 Tailwind CSS for styling
- 🗺️ MapLibre GL for 3D maps
- 🔌 WebSocket for real-time updates
- 📱 Capacitor for mobile apps
- 🎭 Lucide React for icons
Backend:
- 🟢 Node.js with Express
- 🔐 JWT authentication
- 💾 MongoDB with Mongoose
- 🤖 Gemini AI for document analysis
- 📊 WebSocket server for live data
AI/RAG Layer:
- 🐍 Python Pathway framework
- 🦀 Rust-powered vector search
- 🧠 OpenAI GPT-4 / Google Gemini
- 📚 Document embeddings & semantic search
# Required
Node.js >= 18.x
Python >= 3.11
MongoDB >= 6.0
Git
# Optional (for production RAG)
WSL2 (Windows) or Linux
Docker (for containerized deployment)# 1. Clone repository
git clone https://github.com/Abhishekmishra2808/village-digital-twin.git
cd village-digital-twin
# 2. Install frontend dependencies
npm install
# 3. Install backend dependencies
cd backend
npm install
# 4. Set up environment variables
cp .env.example .env
# Edit .env with your MongoDB URI, API keys, etc.
# 5. Start development servers
npm run dev # Frontend (Port 3000)
npm start # Backend (Port 3001, from backend folder)For immediate testing without full Pathway setup:
# 1. Install Flask
pip install flask
# 2. Start mock Pathway server
cd llm-app/templates/question_answering_rag
python mock_pathway_server.py
# Server runs on http://localhost:8080
# 3. Backend .env should have:
PATHWAY_MCP_URL=http://localhost:8080/v1/pw_ai_answer
PATHWAY_MCP_TOKEN=mock_token_for_developmentFor production with real Pathway server:
# On Linux/WSL2:
cd llm-app/templates/question_answering_rag
bash setup-wsl.sh
# Configure with your LLM API key
export OPENAI_API_KEY=your_key_here
# or
export GEMINI_API_KEY=your_key_here
# Run Pathway server
source pathway-env/bin/activate
python app.pyThe "Ask AI" button appears in:
- 📊 Admin Dashboard - Top right corner
- 👥 Citizen Dashboard - Top right corner
- 📋 Schemes View - Header toolbar
Access from the sidebar menu → "Village Analyzer"
Features:
- 🌐 View Infrastructure Network: Automatically loaded from real village data
- 🗺️ Interactive 3D Map: All infrastructure nodes displayed with labels and icons
- 💥 Trigger Failures: Click any node → Select failure type and severity
- 📊 Watch Impacts: See how failures cascade through the network
- ➕ Add Nodes: Right-click map → Add new infrastructure with auto-connections
- 🔄 Reset Network: Clear all failures and return to original state
- 📈 Accumulated Damage: Multiple failures compound realistically
Using the Analyzer:
Step 1: Navigate to "Village Analyzer" from sidebar
Step 2: View the infrastructure network on the map
• Green nodes = Healthy (100% operational)
• Yellow nodes = Minor impact (30-60% operational)
• Orange nodes = Severe impact (10-30% operational)
• Red nodes = Failed (<10% operational)
Step 3: Click any infrastructure node to trigger a failure
┌──────────────────────────────────┐
│ ⚠️ Trigger Failure ✕ │
├──────────────────────────────────┤
│ Main Pump Station │
│ pump • Health: 100% │
│ │
│ Failure Type: [Supply Disruption]│
│ Severity: [Low][Medium][High] │
│ │
│ [💥 Trigger Failure] │
└──────────────────────────────────┘
Step 4: Watch the impact propagate
• GNN calculates impact on connected nodes
• Map updates with color-coded damage levels
• InfoPanel shows detailed impact analysis
Step 5: Add more infrastructure (Optional)
• Right-click anywhere on the map
• Enter node name and select type
• Node auto-connects based on proximity and type
Step 6: Trigger multiple failures to see accumulated damage
• Each failure adds to existing damage
• Nodes above 90% damage cascade to failure
Click "Ask AI" button → Modal opens:
┌────────────────────────────────────────────────────┐
│ 🤖 Ask AI about Schemes ✕ │
├────────────────────────────────────────────────────┤
│ │
│ What water problems are reported in Zone B? │
│ ┌────────────────────────────────────────────┐ │
│ │ │ │
│ └────────────────────────────────────────────┘ │
│ [Ask AI] │
│ │
│ 💡 Example questions: │
│ • Why is Swachh Bharat scheme delayed? │
│ • Show budget status of MGNREGA project │
│ • What are citizen complaints in Zone A? │
└────────────────────────────────────────────────────┘
┌────────────────────────────────────────────────────┐
│ Answer: │
│ ─────────────────────────────────────────────── │
│ Citizens report low water pressure in Zone B, │
│ particularly during morning hours (7-9 AM). │
│ Sensor data confirms 30% pressure drop. │
│ │
│ 📚 Citations (2): │
│ ┌────────────────────────────────────────────┐ │
│ │ 📄 Sensor Report 89% │ │
│ │ "Sensor-042 recorded pressure drops..." │ │
│ │ [Show on Map] [View Document] │ │
│ └────────────────────────────────────────────┘ │
│ ┌────────────────────────────────────────────┐ │
│ │ 💬 Citizen Report 75% │ │
│ │ "Multiple complaints from Zone B..." │ │
│ │ [Show on Map] [View Document] │ │
│ └────────────────────────────────────────────┘ │
│ │
│ 🔍 Trace ID: trace_1763664692_2300 • Cached ✓ │
└────────────────────────────────────────────────────┘
- Show on Map: Pans to citation location with highlight
- View Document: Opens full source document
- Relevance Score: Shows AI confidence (0-100%)
| Feature | Implementation |
|---|---|
| 🔒 PII Sanitization | Auto-redacts Aadhaar, PAN, emails, phones before LLM |
| 🎫 JWT Authentication | All RAG queries require valid user token |
| ⏱️ Rate Limiting | 10 queries/minute per user to prevent abuse |
| 🔑 Service-to-Service Auth | Backend ↔ Pathway uses PATHWAY_MCP_TOKEN |
| 📊 Audit Logging | Every query logged with trace_id, user, latency |
| 🚫 Fail-Open Policy | Returns graceful errors if Pathway unavailable |
// Example: PII auto-redacted before sending to LLM
Input: "Contact John at [email protected], Aadhaar: 1234-5678-9012"
Output: "Contact John at [EMAIL_REDACTED], Aadhaar: [AADHAAR_REDACTED]"POST /api/rag-query
Content-Type: application/json
Authorization: Bearer <JWT_TOKEN>{
"question": "Why is Swachh Bharat scheme delayed?",
"scheme_id": "S-123", // Optional: filter by scheme
"bbox": [77.5, 28.4, 77.6, 28.5], // Optional: geographic filter
"max_citations": 5 // Optional: limit citations (default 5)
}{
"answer": "The Swachh Bharat Mission is delayed by 14 days due to monsoon weather...",
"citations": [
{
"doc_id": "vendor-report-VR-2024-001",
"type": "vendor_report",
"snippet": "Weather delays in October pushed timeline by 3 weeks...",
"score": 0.92,
"timestamp": "2024-10-15T10:30:00Z",
"geo": {
"lat": 28.4595,
"lon": 77.0266
}
},
{
"doc_id": "scheme-S-123",
"type": "scheme",
"snippet": "Phase 2 originally scheduled for October completion...",
"score": 0.87,
"timestamp": "2024-09-01T00:00:00Z",
"geo": {
"lat": 28.4612,
"lon": 77.0312
}
}
],
"trace_id": "trace_1763664729_8574",
"cached": false,
"processing_time_ms": 612
}| Code | Description |
|---|---|
400 |
Invalid request (missing question, malformed bbox) |
401 |
Unauthorized (invalid/missing JWT token) |
429 |
Rate limit exceeded (>10 queries/min) |
502 |
RAG service temporarily unavailable |
500 |
Internal server error |
cd backend
powershell -File test-rag.ps1Expected output:
=== Testing RAG Feature ===
1. Logging in...
✅ Login successful!
2. Sending RAG query...
✅ RAG query successful!
Answer: Multiple schemes are experiencing delays...
Citations: 2 (scores: 0.9, 0.82)
Cached: False
- Open http://localhost:3000
- Login with:
[email protected]/admin123 - Click "Ask AI" button (blue gradient)
- Type:
"Why is MGNREGA delayed?" - Verify:
- ✅ Answer appears within 1 second
- ✅ Citations show with scores
- ✅ "Show on Map" buttons work
- ✅ Cached indicator on repeat query
📁 Project Root
├── 📄 README.md (this file)
├── 📁 docs/
│ └── 📄 README_RAG.md (detailed RAG setup)
├── 📁 backend/
│ ├── 📁 routes/
│ │ └── 🔧 rag.js (RAG endpoint)
│ ├── 📁 utils/
│ │ ├── 🔧 pathwayClient.js (MCP client)
│ │ ├── 🔧 piiSanitizer.js (privacy filter)
│ │ └── 🔧 ragCache.js (response cache)
│ └── 📄 test-rag.ps1 (test script)
├── 📁 src/
│ ├── 📁 components/
│ │ └── 📁 Rag/
│ │ └── 🎨 RagQueryModal.tsx (UI component)
│ ├── 📁 hooks/
│ │ └── 🔧 useRagQuery.ts (React hook)
│ └── 📁 utils/
│ └── 🔧 mapHighlighter.ts (map integration)
└── 📁 llm-app/
└── 📁 templates/question_answering_rag/
├── 🐍 app.py (Pathway server)
├── 🐍 mock_pathway_server.py (dev mock)
└── 📜 setup-wsl.sh (Linux setup)
The RAG feature is integrated across the platform:
1. Admin Dashboard
- Click "Ask AI" in top-right corner
- Query about scheme status, delays, budget
- View citations with geo-coordinates
2. Schemes View
- "Ask AI" button in toolbar
- Ask about specific schemes
- Citations link to scheme details
3. Citizen Dashboard
- Public-facing AI assistant
- Simplified query interface
- Helps citizens track projects
We welcome contributions! Please see our Contributing Guide for details.
- 🧠 AI/ML: Improve RAG accuracy, add new LLM providers
- 🎨 UI/UX: Enhance modal design, add voice input
- 📊 Analytics: Query insights, popular questions dashboard
- 🌐 i18n: Multi-language support for queries
- 📱 Mobile: Optimize RAG UI for mobile devices
This project is licensed under the MIT License - see LICENSE file.
Developed by: Village Digital Twin Team
Contact: [email protected]
GitHub: @Abhishekmishra2808
- Pathway - For the amazing RAG framework
- OpenAI/Google - For LLM APIs
- MongoDB - For flexible document storage
- React Community - For excellent UI libraries
⭐ Star this repo if you find it useful!
Made with ❤️ for rural development