VeloxDB

A high-performance, Redis-compatible key-value datastore built from scratch in Go.

Designed with a sharded actor model to achieve parallelism across CPU cores with lock-free execution within shards. Operates via the Redis RESP protocol — fully compatible with redis-cli and standard Redis clients.

Zero external dependencies — built entirely on Go's standard library

---

Benchmark

Tested against Redis 8.0.3 on the same machine with 50 concurrent clients and 100,000 requests:

Without Pipelining

Command	VeloxDB (req/sec)	Redis 8.0.3 (req/sec)	VeloxDB vs Redis
SET	161,030	177,619	90.7%
GET	151,285	175,131	86.4%

With Pipelining (16 commands)

Command	VeloxDB (req/sec)	Redis 8.0.3 (req/sec)	VeloxDB vs Redis
SET	952,381	1,538,461	61.9%
GET	1,136,363	1,923,076	59.1%

Pipelining delivers a 7x throughput boost on VeloxDB. Without pipelining, VeloxDB runs at ~87% of Redis throughput — remarkable for a pure Go implementation with no C extensions.

---

Features

• Redis-compatible TCP server (RESP protocol)
• Supported commands: GET, SET, DEL, EXPIRE, TTL, INFO
• Sharded architecture — keys distributed across N shards via FNV hash
• Lock-free execution — each shard runs a single-threaded event loop
• Goroutine-per-connection for concurrent client handling
• Pipeline-aware buffered writer — flushes once per batch, not per command
• AOF persistence — crash recovery via write-ahead log replay
• Three AOF sync modes — always, everysec, no (configurable)
• RDB snapshotting — periodic JSON dump for fast restarts
• Active TTL expiry — background scanner deletes expired keys every second
• Lazy expiry — keys also checked and deleted on access
• Atomic metrics — tracks commands/sec, per-command counts, uptime via INFO
• Environment-based configuration with sensible defaults

---

Architecture

Client (redis-cli)
        ↓
TCP Server (accept loop)
        ↓
Connection Handler (goroutine per client)
        ↓
RESP Parser
        ↓
Shard Router → fnv32a(key) % N
        ↓
Shard Channel (buffered)
        ↓
Shard Event Loop (single goroutine)
        ↓
In-Memory Store (map[string]Item + TTL)  +  Atomic Metrics
        ↓
AOF Persistence (appendonly.aof)  +  RDB Snapshot (dump.json)
        ↓
Response → Client

RESP — Redis Serialization Protocol

RESP is a plain-text formatting protocol — an agreed-upon way to structure data so both sides know where one value ends and the next begins.

When you type SET foo bar, the client sends:

*3\r\n$3\r\nSET\r\n$3\r\nfoo\r\n$3\r\nbar\r\n

• *3 → array of 3 elements
• $3 → next string is 3 bytes long
• SET, foo, bar → the actual values
• \r\n → delimiter between parts

Sharding Model

Keys are distributed across shards using FNV hashing:

shard = fnv32a(key) % numShards

Each shard owns its data exclusively, runs a single-threaded event loop, and receives requests via a buffered channel. Multiple goroutines send to the same shard simultaneously — the channel queues them. Only one goroutine ever touches the store.

Actor Model (Per Shard)

Goroutine → Request{Command, ResponseChan} → Channel → Event Loop → Store
                                                                        ↓
                                                               ResponseChan ← result

Pipelining

The connection handler uses a pipeline-aware buffered writer. Responses are buffered and flushed only when no more commands are waiting in the read buffer:

writer.Write(resp)
if reader.Buffered() == 0 {
    writer.Flush()  // flush entire batch at once
}

This reduces syscall overhead under high concurrency — delivering a 7x throughput improvement with pipelined clients.

Persistence

AOF (Append Only File) — every write command appended to data/appendonly.aof in RESP format after execution. Channel-based writer — one goroutine owns the buffer exclusively, eliminating race conditions without locks. Three sync modes:

Mode	Behavior	Data Loss Risk
always	fsync after every write	Zero
everysec	fsync every second (default)	Up to 1 second
no	OS decides when to flush	Up to 30 seconds

RDB (Snapshot) — periodic full JSON dump of the store to data/dump.json. Written atomically via temp file + rename — no corrupt files on crash. On startup: load RDB first, then replay only recent AOF entries. Configurable interval via RDB_INTERVAL.

TTL Expiry

Two-layer expiry for memory efficiency:

• Lazy expiry — keys checked on every GET/TTL access, deleted if expired
• Active expiry — background scanner per shard runs every second, deletes expired keys proactively via the shard channel

Concurrency Model

Layer	Strategy
Networking	Goroutine per connection
Execution	Single goroutine per shard
Coordination	Buffered channels
AOF writes	Channel-based actor — one goroutine owns buffer
Persistence flush	Configurable — always/everysec/no
Metrics	Lock-free atomic counters

---

Getting Started

Prerequisites

• Go 1.22+

Run Locally

git clone https://github.com/AritraC1/veloxdb.git
cd veloxdb
go run cmd/server/main.go

Connect

redis-cli -p 6379

127.0.0.1:6379> SET foo bar
OK
127.0.0.1:6379> GET foo
"bar"
127.0.0.1:6379> EXPIRE foo 10
(integer) 1
127.0.0.1:6379> TTL foo
(integer) 8
127.0.0.1:6379> DEL foo
(integer) 1
127.0.0.1:6379> INFO
uptime_in_seconds:42
total_commands:5
commands_per_second:0.12
get_commands:1
set_commands:1
del_commands:1
expire_commands:1
ttl_commands:1

Run Benchmark

# Start real Redis on port 6380
redis-server --port 6380

# Run benchmark (standard)
./scripts/benchmark.sh

# Run benchmark (pipelined)
redis-benchmark -p 6379 -n 100000 -c 50 -q -t set,get -P 16

---

Configuration

All config is read from environment variables with sensible defaults:

Variable	Default	Description
PORT	6379	TCP port
NUM_SHARDS	8	Number of shards
BUFFER_SIZE	64	Shard channel buffer
AOF_PATH	data/appendonly.aof	AOF file path
AOF_SYNC_MODE	everysec	AOF sync strategy — always, everysec, no
RDB_PATH	data/dump.json	RDB snapshot path
RDB_INTERVAL	60	Snapshot interval in seconds

PORT=6380 NUM_SHARDS=16 AOF_SYNC_MODE=always go run cmd/server/main.go

---

Project Structure

veloxdb/
├── cmd/server/main.go         # Entry point
├── internal/
│   ├── server/                # TCP server, connection handler, buffered writer
│   ├── protocol/              # RESP parser + encoder
│   ├── shard/                 # Shard, event loop, manager, TTL scanner
│   ├── store/                 # In-memory KV store, Item, TTL expiry
│   ├── command/               # Stateless command handlers + registry
│   ├── persistence/           # AOF writer, RDB snapshot, replay
│   ├── metrics/               # Atomic metrics + INFO command
│   └── config/                # Environment-based config
├── scripts/
│   └── benchmark.sh           # Benchmark vs Redis
└── data/                      # AOF + RDB files (gitignored)

---

Design Decisions

Why sharding over a global mutex? A single sync.RWMutex becomes a bottleneck under concurrent writes. Sharding distributes keys across independent stores — each with its own event loop — so writes to different keys execute in parallel with no coordination overhead.

Why channels over locks within shards? Each shard's event loop is the only goroutine that touches its store. Channels provide the serialization — no sync.Mutex needed inside shards. This is the actor model applied to data storage.

Why a channel-based AOF writer? AOF is shared across all shards. Instead of a mutex, the AOF has its own goroutine and channel — same actor model as shards. One goroutine owns the buffer exclusively, eliminating race conditions without locking. Natural batching falls out of this design.

Why three AOF sync modes? Different use cases have different durability requirements. always for zero data loss, everysec for the best balance (Redis's default), no for maximum throughput when durability is handled by replication.

Why RDB + AOF together? AOF alone means replaying potentially millions of entries on restart. RDB gives a fast baseline — load snapshot, replay only recent AOF. Same approach Redis uses.

Why active + lazy TTL expiry? Lazy expiry alone means expired keys stay in memory until accessed — wasteful for short-lived keys nobody reads. Active expiry proactively cleans up every second via the shard channel — no locks, consistent with the actor model.

Why FNV hash? Fast, deterministic, good key distribution. No cryptographic overhead needed for routing.

Why atomic counters for metrics? Metrics are updated on every command across all goroutines simultaneously. Atomic operations (~1-2ns) cost 10-15x less than a mutex (~20-30ns) and never block — consistent with the lock-free design philosophy.

Why pipeline-aware flushing? Calling conn.Write per command makes one syscall per response. Buffering responses and flushing when the read buffer is empty batches multiple responses into one syscall — reducing kernel overhead under pipelined workloads.

---

Trade-offs and Limitations

• Not fully Redis-compatible — only GET, SET, DEL, EXPIRE, TTL, INFO
• No transactions (MULTI/EXEC)
• No pub/sub
• Hot-key bottleneck — if all clients hit the same key, one shard becomes a bottleneck
• Fixed shard count — resharding requires a restart
• Cross-key atomicity not supported across shards
• RDB uses JSON — binary format would be 3x smaller and 4x faster to load

---

Roadmap

Phase 1 — Core ✅

• TCP server + RESP protocol
• GET, SET, DEL, EXPIRE, TTL
• Sharded actor model
• AOF persistence + crash recovery

Phase 2 — Performance ✅

• RESP encoder — centralized response formatting
• Environment-based config
• Pipeline-aware buffered writer — 7x throughput under pipelining
• Atomic metrics + INFO command
• Benchmarking vs Redis

Phase 3 — Reliability ✅ (in progress)

• Active TTL expiry — background scanner per shard
• Channel-based AOF writer — lock-free, three sync modes
• RDB snapshotting — atomic JSON dump, configurable interval
• Consistent hashing — reshard without restart
• Replication — primary-replica model
• Cluster mode

Phase 4 — Features

• More commands — INCR, DECR, EXISTS, MGET, MSET
• LRU/LFU eviction
• Transactions (MULTI/EXEC)
• Pub/Sub

---

What This Project Demonstrates

• Custom TCP server and binary protocol implementation (RESP)
• Concurrent system design in Go — goroutines, channels, no mutexes
• Sharded actor model for parallel, lock-free execution
• Pipeline-aware I/O for high-throughput batch processing
• Dual persistence — AOF for durability, RDB for fast restarts
• Two-layer TTL expiry — lazy + active
• Channel-based AOF writer — lock-free shared resource without mutex
• Lock-free metrics with atomic counters
• Tradeoffs in distributed systems design — sharding, consistency, durability

---

License

Apache License 2.0. See LICENSE for details.

---

Acknowledgements

Inspired by the design of Redis and modern in-memory data systems.