A distributed key-value store built from scratch in Go, implementing the Raft consensus algorithm for fault-tolerant replication across a cluster of nodes.
Most applications that need shared state across multiple servers reach for an existing tool like Redis or etcd. This project implements the consensus layer itself — the algorithm that makes distributed agreement possible.
Raft ensures that a cluster of nodes always agrees on the same state, even when nodes crash or become temporarily unreachable. As long as a majority of nodes are alive, the cluster keeps accepting reads and writes. If the leader crashes, the remaining nodes automatically elect a new one and continue without data loss.
Each node in the cluster can be in one of three roles: follower, candidate, or leader. All writes go through the leader. The leader replicates every write to its followers before acknowledging success to the client — a write is only committed once a majority of nodes have it, which means no committed write can ever be lost even if the leader crashes immediately after.
Leader failure is detected via heartbeats. The leader sends a heartbeat to all followers every 50ms. If a follower doesn't hear from the leader within a randomized timeout (150–300ms), it starts an election. Election timeouts are randomized to prevent multiple nodes from starting elections simultaneously and splitting the vote.
The key-value store supports four operations: SET, DELETE, INCREMENT, and COMPARE_AND_SET.
main.go — entry point, boots the node and HTTP server
node.go — Node struct and state
log_entry.go — LogEntry struct and operation types
config.go — config loader, quorum derivation
election.go — election timeout goroutine, leader election, voting
heartbeat.go — heartbeat goroutine, log replication, commit logic
rpc.go — HTTP handlers for inter-node and client communication
apply.go — applies committed log entries to the key-value store
config.yaml — cluster peer list
Dockerfile — container build
docker-compose.yml — local three-node cluster setup
Prerequisites: Docker Desktop, Go 1.21+
git clone https://github.com/tia-s/dkvs
cd dkvs
go mod tidy
docker-compose up --buildThree nodes boot and elect a leader automatically. You'll see heartbeat and election activity in the logs.
Write a value:
curl -X POST http://localhost:8001/write \
-H "Content-Type: application/json" \
-d '{"op":"SET","key":"user_123","value":"47"}'{"success":true,"message":""}Read from any node — all replicas return the same value:
curl http://localhost:8001/read?key=user_123
curl http://localhost:8002/read?key=user_123
curl http://localhost:8003/read?key=user_123{"key":"user_123","value":"\"47\""}
{"key":"user_123","value":"\"47\""}
{"key":"user_123","value":"\"47\""}Kill the leader and keep writing:
docker-compose stop node1
curl -X POST http://localhost:8002/write \
-H "Content-Type: application/json" \
-d '{"op":"SET","key":"user_123","value":"99"}'
curl http://localhost:8002/read?key=user_123{"success":true,"message":""}
{"key":"user_123","value":"\"99\""}The remaining two nodes elect a new leader within 300ms and continue serving writes.
Bring the failed node back — it catches up automatically:
docker-compose start node1
curl http://localhost:8001/read?key=user_123{"key":"user_123","value":"\"99\""}The rejoining node receives missing log entries from the leader's next heartbeat and immediately reflects the current state.
| Operation | Description |
|---|---|
SET |
Write an absolute value |
DELETE |
Remove a key |
INCREMENT |
Add a delta to an existing numeric value |
COMPARE_AND_SET |
Write only if current value matches expected — useful for atomic updates |
- Rate limiting layer on top of the DKVS using
COMPARE_AND_SETfor atomic counter updates - Log compaction and snapshots so the log doesn't grow indefinitely
- Dynamic cluster membership — adding and removing nodes without downtime
- TLS between nodes