English | 简体中文
Kiwi is a Rust database targeting the observable behavior of Redis 8.8.1. RocksDB is the complete authoritative storage layer, and OpenRaft provides strong consistency and high availability.
The compatibility and interface-design baseline is pinned to Redis tag 8.8.1, commit 77b6c308396c9700672390a210143a8496fb4b10. Current required work runs Cache OFF and focuses on compatibility, authoritative recovery, Raft correctness, and system stability. The Embedded Redis Hot Tier is a future design boundary and is not authorized for implementation until the stability gate passes and a separate approval is given.
- Dual Runtime Architecture: Network and storage runtimes are separated for performance isolation
- RocksDB Authority: RocksDB stores the complete, durable, recoverable data set
- Redis 8.8.1 Compatibility Target: Protocol, commands, errors, TTL, transactions, and client behavior must be verified against an exact upstream baseline
- Stability-First Delivery: Compatibility, RocksDB close/reopen recovery, and single-group OpenRaft correctness must pass the system stability gate before deferred acceleration work is reconsidered
- Raft Consensus: Integrates OpenRaft for strong consistency and high availability
- Adaptor Pattern: Custom adapter layer bridging storage with OpenRaft
- High Performance: Optimized request processing with dedicated thread pools
- Asynchronous Communication: Message channel-based asynchronous communication
- Fault Isolation: Network and storage operations run in isolated runtimes
src/server/ → Entry point (main.rs)
src/net/ → TCP server, connection handling, cluster routing
src/cmd/ → Command definitions: Cmd trait, CmdMeta, command table
src/executor/ → Command executor: tokio async task pool
src/storage/ → Multi-instance RocksDB ownership, column families, TTL
src/resp/ → RESP protocol: parser, encoder, RespData types
src/raft/ → Raft consensus: OpenRaft integration, RocksDB log store, state machine, router
src/conf/ → Configuration: loading, validation, RaftClusterConfig
src/client/ → Client context: connection state, argv, reply buffer
src/common/runtime/ → Runtime management: async channel between net & storage
src/common/macro/ → Proc macros: #[stack_trace_debug]
src/kstd/ → Utilities: LockMgr (sharded key-level locking)
Client → TCP accept [network runtime] → RESP parse → Command lookup
→ connection-local execution or executor_ext admission/dispatch
→ StorageClient → bounded MessageChannel
→ StorageServer [storage runtime] → Cmd.execute() → Storage/RocksDB
← oneshot response ←
→ RESP encode [network runtime] → write back to client
CmdExecutor is not the active production request queue on this path.
The sole authoritative project entry is:
It links the approved requirements, decisions, work packages, current state, GitHub Issues, implementation plans, and verification evidence. README checklists are intentionally not used as a second roadmap.
Kiwi's normal development, CI, and release baseline is Rust 1.97.1 stable. After
you clone the repository, rust-toolchain.toml makes rustup select that exact
toolchain automatically. All Kiwi workspace crates use Rust 2024 Edition.
protoc and the native C/C++ tools needed to build RocksDB are also required.
On Windows, use the Rust MSVC target and install the Visual Studio C++ build
tools. On Linux and macOS, install the platform C/C++ build dependencies used by
the project in addition to protoc. See the platform-specific commands in the
development guide.
# Install rustup; rust-toolchain.toml selects Rust 1.97.1 stable in this repo
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
# macOS: Xcode CLT + protobuf compiler + CMake (required to build RocksDB from source)
xcode-select --install
brew install protobuf cmake
# Debian/Ubuntu Linux: native C/C++ toolchain + protobuf compiler
sudo apt install clang cmake libclang-dev llvm-dev pkg-config protobuf-compilerOn Windows, use the same official Protobuf 27.1 release as CI. Download and
extract protoc-27.1-win64.zip, add its bin directory to PATH, and run
protoc --version. See the development guide
for the PowerShell commands.
Dated nightly toolchains are reserved for specialized checks such as Sanitizers. They do not define the normal development or release baseline.
git clone https://github.com/arana-db/kiwi.git
cd kiwi
# Verify the compiler selected by this checkout
rustup show active-toolchain
rustc --version --verbosemake standalone # builds (release) and starts kiwi on 127.0.0.1:7379
# In another terminal:
redis-cli -p 7379 set foo bar # OK
redis-cli -p 7379 get foo # "bar"make cluster # start a 3-node Raft cluster (default)
make cluster NODES=5 # start a 5-node clusterSee docs/cluster.md for the manual step-by-step procedure and Raft architecture details.
| Document | Description |
|---|---|
| docs/development.md | Dev environment, build optimization, sccache |
| docs/cluster.md | Raft cluster quickstart and write-path verification |
| docs/key-encoding.md | Key encoding internals (Chinese) |
| Redis 8.8.1 compatibility contract | Exact Oracle, raw RESP, TCL, and client-test boundaries |
| Redis 8.8.1 system boundaries | Cache OFF request, storage, and consensus boundaries |
| System stability gate | Required evidence before deferred hot-tier work can be reconsidered |
| Deferred native ABI contract | Future interface design; not an implementation authorization |
| Combined distribution licensing | Future Redis-derived library and source-distribution obligations |
| Product requirements (PRD) | Goals, scope, and Redis 8.8.1 compatibility target |
| Engineering quality gates | Code, test, and release quality gates |
| Personas and user stories | Target users and usage scenarios |
| Documentation index | Map of the whole docs/ tree and suggested reading order |
| Design plans & specs | Dated design records (plans + specs) |
kiwi --sample-config > kiwi.conf |
Write the default configuration to kiwi.conf |
kiwi --full-sample-config > kiwi.conf |
Write all available configuration keys to kiwi.conf |
Kiwi uses the Arana-maintained rust-rocksdb fork to provide the TableProperties Collector/Factory FFI required by Storage LogIndex. The dependency uses the maintenance release tag v0.51.0-arana.2. Published release tags must not be moved, and Cargo.lock records the exact resolved commit for auditable and reproducible builds.
Contributions are welcome. See CONTRIBUTING.md for guidelines.
Kiwi-authored source is licensed under the Apache License 2.0. See LICENSE. Future official distributions that include the Redis-derived native library must satisfy the applicable AGPL-3.0-only obligations described in the architecture and third-party notices.