FogCache is a production-grade distributed caching and content delivery platform that combines high-performance edge caching, adaptive replication, machine-learning-driven placement, and fault-tolerant design to deliver low-latency content at scale.
FogCache is designed to simulate and implement real-world Content Delivery Network (CDN) behavior. It improves application performance by caching content at edge nodes close to clients while dynamically adapting to traffic patterns using analytics and machine learning.
- Reduce request latency through intelligent edge caching
- Minimize load on the origin server
- Provide high availability and resilience under failures
- Adapt automatically to traffic using analytics and ML
- Validate correctness, performance, and reliability through rigorous testing
FogCache was developed using a production-first engineering methodology:
- Correctness & Consistency
- Performance Optimization
- Resilience & Fault Tolerance
- Traffic Intelligence & ML Integration
- Deployment & Real-World Validation
- Language: Java
- Framework: Spring Boot
- Networking: REST APIs
- Concurrency: Java Executors & Locks
- Caching: Custom LRU & LFU cache engines
- Machine Learning: Python + Flask microservice
- Deployment: Docker, Docker Compose
- Testing Tools: ApacheBench (ab), curl, chaos testing, fault injection
| Service | Responsibility |
|---|---|
| Origin Server | Source of truth for all content |
| Edge Server | Caches and serves client content |
| Load Balancer | Routes traffic across healthy edges |
| Cache Engine | Implements LRU/LFU, eviction, stampede protection |
| Hot-Key Tracker | Detects heavy-traffic content |
| Replication Manager | Replicates hot content across edges |
| Prefetch Engine | Proactively warms cache |
| Pattern Analyzer | Extracts real-time traffic features |
| ML Service | Predicts content classification (HOT/WARM/COLD) |
| Metrics Engine | Records latency, hits, misses, errors |
| Admin APIs | Exposes system observability |
Click the image to open it in full size. Use your mouse / trackpad to zoom and scroll in the viewer.
🧩 Why this diagram matters
This architecture illustrates the complete FogCache pipeline:
- Client → Load Balancer → Edge Servers
- Edge Cache with LRU/LFU eviction
- Hot-Key detection & adaptive replication
- Prefetching & ML-based placement
- Pattern analysis & feature extraction
- Metrics, Admin APIs & observability
- Origin shielding & automatic recovery
The design represents a real-world CDN + intelligent control plane.
| Component | Output |
|---|---|
| Origin Server | Raw content data |
| Edge Server | Cached content, low-latency responses |
| Load Balancer | Stable routing decisions |
| Cache Engine | HIT/MISS decisions, evictions |
| Hot-Key Tracker | Hot-key detection logs |
| Replication Manager | Cross-edge data durability |
| Prefetch Engine | Preloaded cache entries |
| Pattern Analyzer | Feature vectors |
| ML Service | Content classification & confidence |
| Metrics Engine | Latency, hit ratio, throughput |
| Admin APIs | Cluster & system visibility |
FogCache was validated through 27 structured testing phases, covering correctness, performance, resilience, intelligence, and deployment.
- HIT/MISS validation
- LRU/LFU eviction correctness
- Origin consistency
- Result: All correctness constraints satisfied
- Load balancing
- Failover validation
- Per-edge metrics isolation
- Result: Deterministic routing & observability verified
-
Tool: ApacheBench
-
Results:
- 2000+ RPS
- 99% cache hit ratio
- 4× latency reduction
- Replication correctness
- Cache stampede prevention
- Result: Single origin call under 50 concurrent requests
- Random node failures
- Zero downtime
- Automatic recovery
- Real-time pattern extraction
- Metrics correctness
- Java → Python ML pipeline
- Adaptive placement & predictive prefetching
- Survived partial failures with no outage
- DNS service discovery
- End-to-end correctness in containers
- 100% test pass
- Production-grade system stability
- Latency reduction: ~4×
- Cache hit ratio: ~99%
- Throughput: 2000+ RPS
- Origin load reduction: >95%
- Zero data loss under failures
- ML-driven adaptive system behavior
| Metric | Before | After |
|---|---|---|
| Latency | ~17 ms | ~4 ms |
| Origin Calls | 100+ | <5 |
| Failure Recovery | Manual | Automatic |
| System Stability | Moderate | Production-grade |
docker compose upcurl http://localhost:8083/content?id=demo
curl http://localhost:8083/content?id=democurl http://localhost:8083/metricscurl http://localhost:8083/admin/nodes
curl http://localhost:8083/admin/hotkeys
curl http://localhost:8083/admin/ml/decisions- Kubernetes multi-edge scaling
- Advanced ML models
- Geo-aware routing
- Persistent observability backend
- Visual monitoring dashboard
- ApacheBench
- Docker & Spring Boot ecosystems
- Distributed systems & CDN research
- Open-source community