I Built a Redis Clone from Scratch in Node.js — RESP, Persistence, Replication, Pub/Sub and Transactions

Most of us use Redis like this:

SET name Alice

GET name

It feels simple.

You send a command.

Redis stores something.

You get the value back.

But I wanted to understand what actually happens behind that simplicity.

So I built a Redis-compatible server from scratch in Node.js.

Not a wrapper around Redis.

Not a client library.

Not a toy object in memory.

A real TCP server that can understand Redis-style commands, parse the RESP protocol, store data in memory, persist writes, support transactions, handle Pub/Sub, and replicate data from master to replica.

The repo includes a Node.js Redis server, a CLI client, a RESP parser, storage modules, persistence modules, replication logic, Pub/Sub, tests, and a React sandbox for visualizing internals.

Repo

Live Sandbox

Why I Built This

I have used Redis in multiple projects for caching, queues, sessions, rate limiting, and real-time features.

But I realized something:

I knew how to use Redis, but I did not deeply understand how Redis works.

SET session:123 abc EX 60

This one command looks tiny.

But internally, a Redis-like server has to do a lot:

• Accept a TCP connection


• Read raw bytes from the socket


• Parse the command format


• Validate the command


• Store the key-value pair


• Track expiry metadata


• Append the write to persistence


• Return a protocol-compatible response


• Potentially propagate the write to replicas

That is a lot of systems engineering hidden behind one line.

So the goal of this project was simple:

Rebuild enough of Redis from scratch to understand the core ideas behind it.

*What I Built

*

The project is a Redis-compatible server built in Node.js. The README describes it as implementing the RESP protocol, dual-layer persistence with AOF and RDB, master/replica replication, Pub/Sub messaging, transactions, and an interactive React sandbox.

The server supports:

• Strings


• Lists


• Hashes


• Key expiry


• PING


• DEL


• FLUSHALL


• Transactions


• Pub/Sub


• Replication


• Persistence

Some example commands:

SET name Alice EX 60

GET name



LPUSH queue task1 task2 task3

LRANGE queue 0 -1



HSET user:1 name Bob age 30

HGET user:1 name



MULTI

SET a 1

SET b 2

EXEC



SUBSCRIBE news

PUBLISH news "Hello World"

The README lists support for strings, lists, hashes, key expiry, PING, DEL, and FLUSHALL as part of the core command set.

The Architecture

At a high level, the server looks like this:

`TCP Client

   ↓

Node.js TCP Server

   ↓

RESP Parser

   ↓

Command Router

   ↓

Command Handler

   ↓

In-Memory Store

   ↓

Persistence / Replication / Response`

When a client sends:

SET name Alice

the server does roughly this:

1. Receive bytes over TCP


2. Parse the bytes into a command


3. Identify SET as the command


4. Validate the arguments


5. Store name = Alice in memory


6. Append the write to AOF


7. Update the RDB-style snapshot


8. Propagate the command to replicas


9. Send +OK back to the client

That flow made me realize that a database is not just “storage”.

• Networking


• Protocol parsing


• Command execution


• Memory management


• Persistence


• Replication


• Client state


• Failure handling

Layer 1: TCP Server

The first layer was the TCP server.

Redis does not work like a normal REST API.

There is no:

POST /set

GET /get

Instead, clients connect to a TCP port and send protocol-formatted commands.

In this project, the server runs on port 6379 by default, similar to Redis.

Example:

node server.js

or:

node server.js --port 6380

The server listens for socket connections, receives chunks of data, parses them, executes commands, and writes responses back to the client.

This was the first important lesson:

A database server is also a network server.

Layer 2: RESP Protocol

Redis clients communicate using RESP, the Redis Serialization Protocol.

That means the server cannot just read plain strings and split them by spaces forever.

It needs to understand structured protocol messages.

For example, a Redis command can be represented like this:

*2

$3

GET

$4

name

That means:

["GET", "name"]

The parser has to understand arrays, bulk strings, simple strings, integers, and errors.

The interesting part is TCP chunking.

A command may arrive like this:

*2\r\n$3\r\nGET\r\n

and then later:

$4\r\nname\r\n

Or multiple commands may arrive together in a single chunk.

So the parser cannot assume that every socket event equals one full command.

It needs to buffer incomplete data and continue parsing once more bytes arrive.

That was one of the first places where this stopped feeling like a normal backend project and started feeling like a systems project.

Layer 3: In-Memory Store

Once the command is parsed, the server needs somewhere to store data.

At the core, Redis is an in-memory data store.

So I built a storage layer around an in-memory Map.

But the store cannot treat every value the same way.

A string is different from a list.

A list is different from a hash.

A key with expiry is different from a normal key.

So the storage layer had to track:

key

value

type

expiry metadata

Supported data types include:

String

List

Hash

The README describes the server as supporting three data types: strings, lists, and hashes.

Example string command:

SET name Alice

GET name

Example list command:

LPUSH queue task1 task2

RPOP queue

LRANGE queue 0 -1

Example hash command:

HSET user:1 name Bob age 30

HGET user:1 name

HGETALL user:1

This layer taught me why Redis has strict type behavior.

For example, if a key is storing a string, you should not be able to run a list command on it.

That kind of validation is what turns a basic key-value map into a real database-like system.

Layer 4: Expiry

Redis is heavily used for temporary data:

sessions

OTP codes

cache entries

rate limits

temporary locks

So expiry support was important.

The clone supports commands like:

SET name Alice EX 60

That means the key should automatically expire after 60 seconds.

There are two ways to think about expiry:

Active expiry: background cleanup

Lazy expiry: delete only when accessed

In this project, expiry metadata is tracked separately, and keys can be checked when accessed.

So when a user runs:

GET name

the server checks:

Does this key exist?

Does it have expiry metadata?

Has the expiry time passed?

If yes, delete and return null.

If no, return the value.

This sounds small, but expiry affects many parts of the system:

GET should respect expiry

DEL should remove expiry metadata

Persistence should preserve expiry information

Replication should handle expiring keys correctly

The sandbox should show TTL countdowns

Layer 5: Persistence

An in-memory store is fast, but if the process dies, memory disappears.

So I added persistence.

The project implements two persistence styles:

AOF

RDB

AOF means Append Only File.

Every write command gets appended to a log file.

Example:

SET name Alice

LPUSH queue task1

HSET user:1 name Bob

On restart, the server can replay the commands and rebuild the database.

RDB is snapshot-style persistence.

Instead of replaying every command, the server stores a full snapshot of the current database.

The README describes this as dual-layer persistence: AOF writes every command to database.aof, while RDB writes a JSON snapshot to dump.rdb.

This helped me understand the tradeoff:

AOF gives a detailed write history.

RDB gives a compact snapshot.

AOF can be more durable.

RDB can be faster to load.

In real Redis, these persistence modes have many more details.

But even implementing a simplified version made the core design much clearer.

Layer 6: Transactions

Redis supports transactions using:

MULTI

EXEC

DISCARD

So I implemented that too.

The idea is:

• MULTI starts a transaction block.


• Commands are queued instead of executed immediately.


• EXEC runs the queued commands.


• DISCARD cancels them.

Example:

MULTI

SET name Alice

GET name

EXEC

The important thing here is that transaction state is per client.

One client may be inside a transaction.

Another client may not be.

That means the server needs to maintain client-specific state, not just global database state.

The README lists MULTI, EXEC, and DISCARD as supported transaction commands.

This changed how I thought about the server.

Earlier, I saw it as:

command comes in → execute command

After transactions, it became:

command comes in

    ↓

is this client inside MULTI?

    ↓

if yes, queue command

if no, execute command

That is a much more realistic server model.

Layer 7: Pub/Sub

The next feature was Pub/Sub.

Redis Pub/Sub lets clients subscribe to channels and receive messages when someone publishes to that channel.

Example:

SUBSCRIBE news



Then another client can run:



PUBLISH news "Hello World"

Internally, this means the server needs a mapping like:

news -> [socket1, socket2, socket3]

alerts -> [socket4, socket5]

When a message is published, the server finds all subscribers and pushes the message to their sockets.

The README lists SUBSCRIBE, UNSUBSCRIBE, and PUBLISH, with channel-to-subscriber socket mapping and cleanup on disconnect.

This was one of the most fun parts of the project because it moved the server beyond request-response.

Normal commands are like:

client asks → server replies

Pub/Sub is different:

client subscribes

server remembers the socket

later, server pushes messages to it

That makes the server feel alive.

Layer 8: Replication

Replication was the hardest and most interesting part.

The clone supports master-replica replication.

You can run one server as master:

node server.js --port 6379

And another as replica:

node server.js --port 6380 --replicaof localhost 6379

Writes on the master are propagated to the replica. The README also describes full sync with an RDB snapshot and partial resync on reconnect.

At a high level:

1. Replica connects to master


2. Replica performs handshake


3. Master sends snapshot


4. Master streams future write commands


5. Replica applies writes


6. Replica stays read-only for normal clients

The project includes:

1. master replication ID


2. 1 MB circular backlog


3. full resync


4. partial resync


5. replica handshake state machine


6. auto-reconnect


7. REPLCONF ACK heartbeats


8. read-only replica mode

Those replication details are listed in the README’s replication section.

This was where I realized that replication is not just “copy data to another server”.

It involves:

• connection state


• offset tracking


• backlogs


• snapshots


• handshakes


• heartbeats


• reconnect behavior


• read-only enforcement


• command propagation

Replication turned this from a key-value store into a distributed systems project.

Layer 9: React Sandbox

After building the server, I wanted a way to make the internals easier to understand.

So I built a React + Vite sandbox that simulates the server in the browser.

The sandbox lets you type Redis commands and watch internal state update live.

It shows:

• Database


• Expiry store


• AOF log


• RDB snapshot


• Pub/Sub channels

The README says the sandbox simulates the server entirely in the browser and includes tabs for database, expiry, AOF log, RDB snapshot, and Pub/Sub.

This is useful because most backend internals are invisible.

You run:

SET name Alice EX 60

and normally you only see:

OK

But in the sandbox, you can see:

The key appears in the database

The TTL countdown starts

The AOF log updates

The RDB snapshot changes

That made the project not just functional, but explainable.