PostgreSQL Streaming WAL Archiver in Go (pgrwl)

A production-grade, cloud-native PostgreSQL WAL archiving system designed for:

• streaming WAL to S3 with compression, encryption, and retention


• Kubernetes-native PostgreSQL backup workflows


• zero data loss and reliable Point-in-Time Recovery (PITR)

Project

https://github.com/hashmap-kz/pgrwl

Features

• WAL receiver (replication protocol)


• Continuous WAL streaming


• Backup to S3 (MinIO, AWS, etc.)


• Backup to SFTP (backup server)


• WAL compression (gzip, zstd-ready)


• WAL encryption (AES-GCM)


• WAL retention management


• WAL monitoring and observability


• Kubernetes & container ready


• Helm chart support


• YAML / JSON / ENV config


• Lightweight single binary


• Structured logging


• Integration tests (containerized)


• Unit tests


• Backup automation (streaming basebackup)


• Continuous backup for PostgreSQL

Key Capabilities

Streaming WAL

• Uses PostgreSQL replication protocol


• Supports synchronous WAL streaming


• Enables zero data loss setups

Storage Backends

• S3-compatible storage


• SFTP backup servers

Compression + Encryption

Pipeline based on filename:

000000010000000000000001.gz.aes

Flow:

• compress -> encrypt -> upload


• download -> decrypt -> decompress

Architecture

PostgreSQL

   | (replication protocol)

WAL Receiver

   |

Local FS (fsync)

   |

Uploader (S3 / SFTP)

   |

Retention manager

   |

HTTP server (restore_command)

Continuous Backup

• real-time WAL streaming


• safe off-site storage


• full PITR support


• near-zero RPO

Kubernetes Ready

• run as StatefulSet


• works with StatefulSets / CNPG / Virtual Machines


• deploy via Helm


• GitOps-friendly

Configuration Example

main:

  listen_port: 7070

  directory: wals

receiver:

  slot: pgrwl_v5

log:

  level: trace

  format: text

  add_source: true

Testing

• integration tests with real PostgreSQL containers


• end-to-end WAL validation


• unit-tested components

Why pgrwl?

• simple deployment (single binary)


• production-grade reliability


• cloud-native design


• built for Kubernetes and containers


• secure and efficient WAL handling

Contribute

• Star the repo (https://github.com/hashmap-kz/pgrwl)


• Open issues (https://github.com/hashmap-kz/pgrwl/issues)


• Suggest improvements


• Submit PRs (https://github.com/hashmap-kz/pgrwl/blob/master/CONTRIBUTING.md)

Summary

pgrwl is a lightweight, powerful, production-ready WAL archiving solution that brings:

• streaming


• security


• automation


• observability

to PostgreSQL backups.