A distributed, multi-tenant key-value & object store written in Rust
minikv provides strong consistency (Raft + 2PC), durability (WAL), and production-grade observability, security, and multi-tenancy — all in a modern Rust codebase.
Built in public as a learning-by-doing project — now evolved into a complete, reference implementation of distributed systems in Rust.
minikv v0.7.0 brings advanced data management and query capabilities :
- Secondary indexes : Search keys by value content with
GET /search?value=<substring> - Multi-key transactions : Execute multiple operations atomically with
POST /transaction - Streaming/batch import/export : Bulk data operations with
POST /admin/import&GET /admin/export - Durable S3-backed object store : Persistent storage for S3-compatible API via pluggable backends
Previous highlights (v0.6.0) : enterprise security, multi-tenancy, encryption at rest, quotas, audit logging, persistent backends, watch/subscribe system.
minikv is a distributed key-value store written in Rust, designed for simplicity, speed, and reliability.
Who is this for ?
minikv is for engineers learning distributed systems, teams experimenting with Rust-based infrastructure, and anyone curious about consensus, durability, and system trade-offs.
- Clustered : Raft consensus and 2PC for transactional writes
- Virtual Sharding : 256 vshards for elastic scaling & balancing
- WAL : Write-ahead log for durability
- gRPC for node communication, HTTP REST & S3 API for clients
- Bloom filters, snapshots, watch/subscribe for performance & reactivity
- Rust – core logic
- Shell – orchestration/automation
- JavaScript – benchmarks, tools
- Makefile – build flows
git clone https://github.com/whispem/minikv.git
cd minikv
cargo build --release
# Start a node
cargo run -- --config config.example.toml
# API examples
curl localhost:8080/health/ready # readiness
curl localhost:8080/metrics # Prometheus metrics
curl localhost:8080/admin/status # admin dashboard
# Create API key (admin)
curl -X POST http://localhost:8080/admin/keys -d '{"role":"ReadWrite","tenant_id":"acme"}'
# S3 (demo)
curl -X PUT localhost:8080/s3/mybucket/mykey -d 'hello minikv!'
curl localhost:8080/s3/mybucket/mykeyFor cluster setup and advanced options, see the documentation.
- Raft: consensus and leader election
- 2PC: atomic distributed/batch writes
- Virtual Shards: scale and rebalance across 256 partitions
- Pluggable Storage: in-memory, RocksDB, Sled
- Admin API: HTTP endpoints for status, metrics and config
- Config: via environment, file or CLI flags
- Write throughput : over 50,000 operations/sec (single node, in-memory)
- Sub-millisecond read latency
- Cluster tested (3–5 nodes, commodity VMs)
- Built-in Prometheus metrics
- Raft consensus (multi-node, strong consistency)
- Two-phase commit (2PC) for atomic multi-key transactions
- 256 virtual shards for cluster scaling and rebalancing
- Write-ahead log (WAL) for durability
- Auto-rebalancing, graceful leader failover, hot-join and node removal
- Time-To-Live keys (TTL)
- LZ4 compression (configurable)
- Bloom filters and index snapshots
- Pluggable and persistent storage: in-memory, RocksDB, Sled
- Batch & range operations, prefix queries
- HTTP REST (CRUD, batch, range, admin)
- S3-compatible API (with TTL extensions)
- gRPC (internal)
- WebSocket and SSE endpoints for real-time watch/subscribe events
- API keys (Argon2) and JWT authentication
- Role-based access control (RBAC) and audit logging
- Multi-tenant isolation
- AES-256-GCM encryption at rest
- Per-tenant quotas (storage, requests, rate limits)
- TLS (HTTP & gRPC)
- Admin dashboard
- Prometheus metrics (counters, histograms)
- Request and endpoint statistics
- Structured logging and tracing spans
- Kubernetes health probes
- Memory-safe Rust
- Test suite, automated CI
- Documentation and sample config
- Single static binary
minikv started as a 24-hour challenge by a Rust learner (82 days into the language!). It now serves as both a playground and a reference for distributed systems, demonstrating curiosity, learning-by-doing, and robust engineering.
cargo test # Run all tests
cargo clippy --fix # Lint and fix
cargo fmt # Format codeContinuous Integration runs on push & PR via .github/workflows/ci.yml.
Issues and PRs welcome! See CONTRIBUTING.md.