Redistill - High Performance Redis Compatible Key Value Database

Redis has always been one of my go-to KV databases. I strongly believe in — and advocate for — a memory-first, disk-later approach when designing high-throughput, low-latency systems. Redis is fast and durable, but I kept wondering: can we build something Redis-like that’s even faster?

That curiosity pushed me to experiment deeply with Redis and the RESP protocol. Eventually, I decided to “distil” Redis — strip it down to the essentials and build a lighter version that supports only the commands I use most often (Strings and Hashes) — and see if we could push throughput and latency even further.

Turns out, we did.

I’m happy to announce that I’m working on an open-source project called Redistill — a distilled Redis. It’s written in Rust and based on benchmarks run on AWS c7i.16xlarge (Intel, 64 cores, 128GB RAM) using memtier_benchmark, I can confidently say this is the fastest single-instance, Redis-compatible database available today.

Project link: https://github.com/redistill-io/redistill

This is just the beginning. The project is barely 8 weeks old. I still need to figure out clustering and other production-ready features, but so far, as a single instance, this is ready to use ina production environment.

Motivation

The entire motivation behind the project was to build my own KV database that works faster than Redis and supports the commands I use frequently. It doesn't need to be versatile like Redis, but it does one thing best - the fastest KV database possible.

I also wanted to learn Rust, and this was a good opportunity to revisit the fundamentals of system design and build a database from scratch (not entirely scratch, I use RESP protocol).

Key Characteristics

Redistill supports the RESP protocol, so there is no need for code changes on the client side. Your existing code, which works with Redis, will work with Redistill, assuming you use commands Redistill supports.

Below are some key features:

• High-performance in-memory key-value database


• Redis protocol compatible (RESP) - works with all Redis clients


• 9.07M operations/second - 4.5x faster than Redis, 1.7x faster than Dragonfly


• 5x lower latency (p50: 0.48ms vs Redis 2.38ms)


• Multi-threaded architecture with lock-free reads


• Optional persistence - Snapshot support for warm restarts (disabled by default for max performance)


• Production-ready security and monitoring features

Why not all Redis commands?

This was intentional.

I wanted a KV database for my personal and work uses. I used String and Hash commands almost 90% of the time. So keeping it lightweight was the design and not a restriction.

Will I add more commands in the future? Of course, this is a FOSS project, so I'll let contributors add the commands as they seem fit.

Why Redistill?

• High-Performance KV Database - Fastest in-memory key-value database available


• Maximum Single-Instance Performance - 4.5x faster than Redis, 1.7x faster than Dragonfly per instance


• Lower Latency - Sub-millisecond p50 latency (0.48ms)


• Cost Efficient - 50-83% infrastructure savings (fewer instances needed)


• Drop-in Compatible - Works with existing Redis clients


• Production Ready - TLS, authentication, monitoring, health checks


• Multi-threaded - Utilises all CPU cores efficiently


• Optional Persistence - Snapshot support for warm restarts when needed (disabled by default)

Quick Start

We recommend using Docker to quickly spin up Redistill in your machine or server.

# Run with default settings

docker run -d --name redistill -p 6379:6379 shahidontech/redistill:latest



# Test it works

redis-cli ping

# PONG



redis-cli set hello world

# OK



redis-cli get hello

# "world"

We also have other methods, such as Homebrew and building from source.

Performance

Competitive Benchmark (c7i.16xlarge)

Independent comparison on AWS c7i.16xlarge (Intel, 64 cores, 128GB RAM) using memtier_benchmark with production-like configuration:

Test Configuration:

Duration: 60 seconds

Threads: 8, Connections: 160 (20 per thread)

Pipeline: 30, Data size: 256 bytes

Workload: 1:1 SET: GET ratio

Key Observations:

• Redistill processed 544M total operations (2.7x more than Dragonfly, 4.5x more than Redis)


• Consistent low latency across all percentiles


• No errors or connection issues across all systems

Use cases

Redistill is a high-performance key-value database perfect for applications requiring maximum speed and minimal latency.

Perfect For

Session Storage

• 1M+ operations/second


• Sub-millisecond latency


• Automatic TTL expiration


• 60% cost reduction vs alternatives

API Response Caching

• 95%+ cache hit rates


• 50-150x faster than database queries


• Automatic memory management


• LRU eviction is built in

Rate Limiting

• Millions of counters


• TTL-based cleanup


• High write throughput


• Perfect for API gateways

Real-time Leaderboards

• Fast reads for rankings


• Periodic score updates


• Can rebuild from the database


• Sub-millisecond queries

Not Recommended For

• Critical persistent data (optional snapshots available, but not real-time durability)


• Financial or transactional data (use ACID-compliant database)


• Data that cannot be regenerated (use a database)

Examples

The Redistill uses the RESP protocol to communicate, so any Redis client library should be able to connect to Redistill and communicate as they are communicating with Redis with the same commands.

For example, in Python:

import redis

r = redis.Redis(host='localhost', port=6379, password='your-password')

r.set('key', 'value')

value = r.get('key')

In Node.js:

const Redis = require('ioredis');

const redis = new Redis({host: 'localhost', port: 6379, password: 'your-password'});

await redis.set('key', 'value');

Contributing

Contributions are welcome! Please:

• Open an issue to discuss proposed changes


• Follow Rust coding conventions


• Include tests for new features


• Update relevant documentation

Project link: https://github.com/redistill-io/redistill