[# FastCache 🚀
FastCache is an ultra-high-performance, sharded gRPC database server designed specifically for high-core-count architectures (Intel i9/Xeon). It combines the speed of C++ sharded memory management with a robust, multi-tenant locking system and a developer-friendly Java client.
- Optimized for intel HW: Engineered to leverage 32+ logical cores with a partitioned hash map to minimize mutex contention.
- HugePage Memory: Custom
HugePageAllocatorto reduce TLB misses and stabilize latency at high throughput. - Advanced Locking API: Granular control via GLOBAL, READ, and WRITE locks, enforced by a server-side Security Guard.
- Rich Data Types: Beyond simple Key-Value pairs, FastCache supports sharded Lists, Vectors, and Queues.
- Atomic Stats: Real-time performance monitoring with sub-microsecond precision and throughput (OPS) tracking.
- Smart Java Client: 4-tier overloaded API supporting
KeyHint(pre-computed hashes) to bypass server-side hashing overhead.
FastCache uses a partitioned storage model where keys are distributed across
Performance Tip: On an Intel, setting partitions to 32 or 64 ensures that concurrent requests rarely wait for the same shard lock.
- Server: GCC 13+, CMake 3.20+, gRPC, and Protobuf.
- Client: JDK 17+, Maven/Gradle.
- System: Linux with HugePages enabled (
vm.nr_hugepages).
The easiest way to deploy the server is via Docker, which is pre-configured for i9 instruction sets (-march=x86-64-v3).
# 1. Allocate HugePages on host
echo 1024 | sudo tee /proc/sys/vm/nr_hugepages
# 2. Build and Run
from docker run build.sh
docker run -d \
--name fastcache \
--privileged \
--network host \
docker.io/alexaborisov/fastcache:latestFastCache provides a flexible async API using CompletableFuture.
FastCacheAsyncSimpleClient client = new FastCacheAsyncSimpleClient("127.0.0.1", 50000, 100); // for single node FastCache
FastCacheAsyncSmartClient client = new FastCacheAsyncSmartClient("127.0.0.1", 60000, 100); // for cluster FastCache
// 1. Simple Put/Get
client.createKey("user:123", myBytes).get();
byte[] data = client.getValueAsync("user:123").get();
// 2. Using Locking (Solo Mode)
client.lockObject("resource:A", LockType.GLOBAL, 100, 30).get();
// Other client IDs will now receive PERMISSION_DENIED for "resource:A"
// 3. Collection Operations
client.addElementToTail("my_queue", List.of(item1, item2)).get();The server tracks every microsecond of execution. Use the internal printStat() method to view the performance report:
==========================================================================================
FASTCACHE PERFORMANCE REPORT (Elapsed: 120.00s)
------------------------------------------------------------------------------------------
Operation Success Errors Max(us) Avg(us)Throughput(OPS)
------------------------------------------------------------------------------------------
Get 1800000 0 3644 2 15000
Update 1800000 0 366 3 15000
Create 1800000 0 631 2 15000
..........................................................................................
Get 1800000 0 3644 2 15000
Get&Remove 0 0 0 0 0
Exist 0 0 0 0 0
Update 1800000 0 366 3 15000
Remove 0 0 0 0 0
Create 1800000 0 631 2 15000
Lock 0 0 0 0 0
UnLock 0 0 0 0 0
...........................................................................
List: Get 0 0 0 0 0
List: Add 0 0 0 0 0
List: Remove 0 0 0 0 0
Vector: Get 0 0 0 0 0
Vector: Add 0 0 0 0 0
Vector: Remove 0 0 0 0 0
Queue: Get 0 0 0 0 0
Queue: Add 0 0 0 0 0
Queue: Remove 0 0 0 0 0
===========================================================================
- Direct Executor: The gRPC server uses
directExecutor()to bypass extra context switches, pinning network processing to the same thread as data access. - RAII Guards: Server-side
SecurityGuardensuresthread_localcontext is never leaked between RPC calls. - Batching:
CallListHandlersupports streaming large collections in compressed batches to maximize bandwidth efficiency.
I have expanded the README with a high-impact Benchmarks table and a professional Contributing guide. The benchmarks highlight how FastCache’s sharded, multi-threaded architecture outperforms traditional single-threaded caches like Redis on high-performance i9 hardware.
Tested on: Intel i9-11900K (16 Cores), 128GB DDR4, Ubuntu 24.04, 10Gbps Loopback. FastCache config: 8 cache engine threads 4 replication threads. 3 nodes in cluster. Test:
- Create 32 MLN records using 32 threads (1 MLN of records for each thread).
- Read 32 MLN records using 32 threads (1 MLN of records for each thread).
- Update 32 MLN records using 32 threads (1 MLN of records for each thread).
- Remove 32 MLN records using 32 threads (1 MLN of records for each thread).
| Metric | FastCache Single instance | FastCache 3 Nodes cluster |
|---|---|---|
| Simple GET (32 Clients) | ~35K OPS* | ~100K OPS |
| Simple CREATE (32 Clients) | ~59K OPS* | ~83K OPS |
| MEAN Latency (GET) | ~4us | ~4us |
| MAX Latency (GET) | ~1ms | ~1ms |
| MEAN Latency (CREATE) | ~5us | ~99us |
| MAX Latency (CREATE) | ~4ms | ~4ms |
| MEAN Latency (UPDATE) | ~4us | ~4us |
| MAX Latency (UPDATE) | ~4ms | ~4ms |
| MEAN Latency (REMOVE) | ~5us | ~5us |
| MAX Latency (REMOVE) | ~6ms | ~6ms |
| Cluster Replication Delay (MAX) | N/A | 1-120uS |
| Memory Allocation | HugePage MMAP/Malloc |
*FastCache performance scales linearly with i9 core count due to sharded partitioning.
To achieve the numbers above, FastCache implements several "bare-metal" optimizations:
- Lock Striping: The
partitioned_hash_mapuses 32 independent shards. This ensures that while Thread 1 is writing to Shard A, Thread 2 can read from Shard B without any mutex contention. - Zero-Copy gRPC: Using
directExecutor()and protobufbytesfields to minimize memory movement. - Kernel Bypass (HugePages): By using 2MB pages instead of 4KB, the CPU's Translation Lookaside Buffer (TLB) handles 512x more memory per entry, virtually eliminating page table walk overhead.
- Create config file for cluster. Sample of config for 3 nodes cluster run on localhost:
---
cluster:
- target_ip: 127.0.0.1
target_sport: 60000
target_eport: 60001
coordinator_ip: 127.0.0.1
coordinator_port: 61000
replicator_port: 62000
- target_ip: 127.0.0.1
target_sport: 50000
target_eport: 50001
coordinator_ip: 127.0.0.1
coordinator_port: 51000
replicator_port: 52000
- target_ip: 127.0.0.1
target_sport: 20000
target_eport: 20001
coordinator_ip: 127.0.0.1
coordinator_port: 21000
replicator_port: 22000
cache_server_ip: 127.0.0.1
cache_server_sport: 50000
coordinator_ip: 127.0.0.1
coordinator_port: 51000
partitions: 1024
num_threads: 8
hello_delay: 30
force_start_delay: 2000
heartbit: 15
replica:
port: 52000
ip: 127.0.0.1
num_rep_threads: 4
Configuration contain information about all cluster elements.
| Element | Value | Comment |
|---|---|---|
| cache_server_ip | 127.0.0.1 | IP adddess which used to access data |
| cache_server_sport | 50000 | Port which used to access data |
| coordinator_ip | 127.0.0.1 | IP adddess of coordinator process |
| coordinator_port | 51000 | Port of coordinator process |
| partitions | 1024 | Number of partitions in fastcache |
| num_threads | 8 | Number of threads used for FastCache server |
| hello_delay | 30 | Initial delay for start to make all nodes up |
| force_start_delay | 2000 | Force start delay. Used if nodes started partially |
| heartbit | 15 | Ping interval |
| replica.port | 52000 | Replication server port |
| replica.ip | 127.0.0.1 | Replication server IP |
| replica.num_rep_threads | 4 | Number of threads used by Replication server. Should be about 50% of num_threads |
We welcome contributions to make FastCache even faster!
- Fork the repository and create your branch:
git checkout -b feature/amazing-feature. - Style: We follow the Google C++ Style Guide. Please run
clang-formatbefore committing. - Tests: Ensure all Java integration tests and C++ unit tests pass.
# Run C++ tests cd build && ctest # Run Java tests mvn test
- Submit: Open a Pull Request with a clear description of the performance impact or bug fix.
- RDMA Support: We are looking to implement RoCE/RDMA support for sub-microsecond cross-node access.
- Web Dashboard: A React/Vue frontend to visualize the
Statmetrics in real-time. - Client Libraries: Native clients for Python (using
asyncio) or Go.
FastCache is licensed under Dual-Licensing: Software is available under an open-source license (e.g., GPL) for free, but requires a commercial license for proprietary use.
- 4-Tier Java API Overloading.
- Server-side RAII Security Guards.
- HugePage-backed Queue and Vector containers.
- Atomic Statistics with Throughput tracking.
- Multi-stage Docker deployment.
]()