Making Redis Optional: Why I’m pivoting to a 'Postgres-First' Architecture and why chose Valkey as alternative

I've spent the last few weeks rethinking the dependency graph of Logtide.

Currently, if you want to run Logtide, you must spin up a Redis instance. Don't get me wrong - Redis is incredible. But for many self-hosted users running on a $20/month VPS, adding a 200MB RAM sidecar just to handle a few thousand background jobs a day feels... unnecessary.

So I'm making Redis optional. Here's the "why" and the "how" behind the move to a pluggable architecture.

The "Dependency Tax"

Logtide is designed for simplicity. For a typical deployment handling around 50k logs/day, our current stack (Postgres + Redis + SvelteKit + Fastify) works perfectly, but Redis is only doing three things:

1. Managing background jobs (via BullMQ)


2. Session caching


3. Rate limiting

At scale, Redis is the right tool. But for a single-node VPS? It's an extra service to monitor, secure, and feed with RAM. I realized that for many of our users, PostgreSQL is more than enough.

Three Paths, One Interface

Instead of forcing a "one size fits all" approach, I've refactored the backend to support three different scenarios:

1. The "Zero-Dependency" Path (Postgres Only)

This is now the default for small-to-medium deployments. We're leveraging PostgreSQL SKIP LOCKED for the job queue and UNLOGGED tables for caching.

The benefit: One database. One backup strategy. Zero extra RAM overhead.

The trade-off: Slightly higher latency, but negligible for most background tasks.

2. The Valkey Path (The Open Source Alternative)

With Redis changing its license, many are looking at Valkey. It's a Linux Foundation project, fully open-source (BSD-3), and backed by AWS.

It's a drop-in replacement. We didn't even have to change our BullMQ logic just pointed the connection URL to the Valkey container.

Why Valkey instead of Redis? Redis switched to RSALv2/SSPL licensing in March 2024. Not OSI-approved. Valkey forked from Redis 7.2.4 and kept the BSD-3 license. Same performance, actual open source.

3. The Legacy Path (Redis)

If you already have a managed Redis instance or a complex cluster, nothing changes. You can keep using it exactly as before.

Under the Hood: The Postgres Queue

The hardest part was detaching from BullMQ, which is tightly coupled to Redis. I abstracted the logic into a provider-based system.

For the Postgres implementation, the "magic" happens with FOR UPDATE SKIP LOCKED. This allows multiple workers to grab jobs without race conditions:

WITH next_job AS (

  SELECT id FROM jobs

  WHERE queue = $1 

    AND attempts < max_attempts 

    AND scheduled_at <= NOW()

  ORDER BY scheduled_at

  LIMIT 1

  FOR UPDATE SKIP LOCKED

)

UPDATE jobs SET attempts = attempts + 1

FROM next_job WHERE jobs.id = next_job.id

RETURNING *;

It's an elegant way to turn a relational database into a reliable queue without the complexity of a dedicated message broker.

The Pluggable Interface

The abstraction is simple. Two interfaces:

interface CacheProvider {

  get(key: string): Promise<string | null>;

  set(key: string, value: string, ttl?: number): Promise<void>;

  del(key: string): Promise<void>;

}



interface QueueProvider {

  enqueue(job: Job): Promise<void>;

  dequeue(): Promise<Job | null>;

  complete(jobId: string): Promise<void>;

}

Three implementations: PostgresCacheProvider, ValkeyCacheProvider, RedisCacheProvider.

Configure via environment variables:

services:

  backend:

    environment:

      CACHE_PROVIDER: valkey

      QUEUE_PROVIDER: postgres

You can mix them. Cache on PostgreSQL, queue on Valkey. Whatever fits your deployment.

Switching to Valkey (If You Want)

For existing Redis users curious about Valkey, the migration is trivial.

Add Valkey to your docker-compose.yml:

services:

  valkey:

    image: valkey/valkey:7.2-alpine

    ports:

      - "6379:6379"

    volumes:

      - valkey-data:/data

Update the connection URL:

# Old

REDIS_URL=redis://redis:6379



# New

VALKEY_URL=redis://valkey:6379  # same protocol

Restart:

docker compose down

docker compose up -d

That's it. Valkey uses the Redis protocol. Same commands, same client libraries. ioredis, BullMQ, redis-py, they all work without changes.

Performance: Does It Matter?

I benchmarked all three approaches. Test: 10,000 background jobs.

Valkey is 2.7x faster than PostgreSQL.

But here's the thing: does a background log-rotation job care if it takes 2 seconds instead of 0.7? Probably not.

Cache performance was similar. Valkey wins, but the difference is milliseconds.

For most self-hosted deployments, the simplicity of PostgreSQL-only outweighs the performance gain.

Postgres Cache: UNLOGGED Tables

For those curious about the Postgres cache implementation:

CREATE UNLOGGED TABLE cache (

  key TEXT PRIMARY KEY,

  value JSONB NOT NULL,

  expires_at TIMESTAMPTZ NOT NULL

);

UNLOGGED means:

• Skip Write-Ahead Log (faster writes)


• Data doesn't survive crashes (fine for cache)


• 2-3x faster than regular tables

A periodic job cleans up expired entries:

DELETE FROM cache WHERE expires_at < NOW();

Simple. Effective. No extra service.

The Licensing Question

This whole thing started when Redis changed licenses. March 2024, they switched to dual RSALv2/SSPL licensing.

Not OSI-approved. Restricts cloud providers from offering "Redis as a Service" without releasing source code.

Valkey forked from Redis 7.2.4 and kept BSD-3. Linux Foundation governance. AWS, Google, Oracle backing it.

For self-hosted users, the licensing doesn't directly affect you. But it signals where the projects are headed.

Real-World Usage

Current Logtide (v0.4.0): Redis required.

Next release: Redis optional. PostgreSQL default. Valkey supported.

If you're already using Logtide with Redis:

1. Keep Redis, it still works


2. Switch to Valkey, change the image, restart

Your choice.

Technical Challenges

The main challenge was BullMQ. It's built for Redis/Valkey. Abstracting it required wrapping the entire queue interface.

For atomic operations (like rate limiting), PostgreSQL needs transactions where Redis has INCR:

BEGIN;

  UPDATE rate_limits SET count = count + 1 WHERE key = $1;

  SELECT count FROM rate_limits WHERE key = $1;

COMMIT;

Slightly slower. Still fast enough.

For pub/sub, we already use PostgreSQL LISTEN/NOTIFY for live log streaming. It works fine.

When to Use What

PostgreSQL-only:

• Small deployment (< 100k logs/day)


• Single VPS


• Want minimal dependencies

Valkey:

• Higher throughput (> 100k logs/day)


• Need sub-millisecond cache


• Already familiar with Redis

Redis:

• Existing Redis infrastructure


• Don't care about licensing

My Takeaway

Yes, Valkey/Redis is 3x faster for job queues. But does it matter for background tasks? For most self-hosted users, no.

The simplicity of a single database, one backup, one service to monitor, one point of failure, usually coff coff wins.

But having the choice matters. Some users need the performance. Some want the licensing clarity of Valkey. Some already have Redis.

So we support all three.

PostgreSQL-first. Valkey as an optional boost. Redis if you want it.

Repo: https://github.com/logtide-dev/logtide