How to Install Apache Kafka on Ubuntu 26.04 LTS (KRaft Mode, No ZooKeeper)

Table of Contents

1. What is Apache Kafka and When Should You Use It


2. KRaft Mode: Why ZooKeeper is No Longer Needed


3. Prerequisites


4. Step 1: Install Java 21


5. Step 2: Create a Dedicated Kafka User


6. Step 3: Download and Install Apache Kafka 4.2


7. Step 4: Configure Kafka in KRaft Mode


8. Step 5: Format the Storage Directory


9. Step 6: Create a systemd Service


10. Step 7: Configure UFW Firewall


11. Step 8: Test with Producer and Consumer


12. Step 9: Basic Topic Management


13. Production Configuration Tips


14. Kafka vs RabbitMQ: When to Use Which


15. Common Issues and Quick Fixes


16. Next Steps

Real-time data is everywhere — microservices firing events, IoT sensors streaming telemetry, user actions triggering notifications, logs flowing from dozens of servers simultaneously. Managing all of that reliably, at scale, without building a custom integration between every producer and every consumer, is exactly the problem Apache Kafka was built to solve.

This guide walks through installing Apache Kafka 4.2 on Ubuntu 26.04 LTS (Resolute Raccoon) using KRaft mode — the modern deployment approach that removes the ZooKeeper dependency entirely. If you've seen older Kafka tutorials that tell you to start ZooKeeper first, those are outdated: starting with Kafka 4.0, ZooKeeper support was completely removed.

What is Apache Kafka and When Should You Use It

Apache Kafka is a distributed event streaming platform — a durable, ordered log that producers write to and consumers read from. Unlike a traditional message queue where a message disappears after being consumed, Kafka retains events for a configurable period, allowing multiple independent consumers to read the same stream at their own pace.

Real-world use cases where Kafka fits well:

• 
  Microservice decoupling — Service A publishes an event; Services B, C, and D each consume it independently, without A knowing or caring who's listening.


• 
  Log and metrics aggregation — Centralize logs from dozens of services into one stream, then fan out to Elasticsearch, S3, and a monitoring stack simultaneously.


• 
  Real-time analytics — Process a stream of user events, transactions, or sensor readings as they happen rather than batching overnight.


• 
  Event sourcing — Store every state change as an immutable event, with the ability to replay history to rebuild application state.


• 
  Change Data Capture (CDC) — Stream database changes (inserts, updates, deletes) to downstream systems in real time.

Kafka is not the right tool for every messaging need. If you're sending a task to exactly one worker and want it acknowledged once, a simpler queue (Redis Streams, RabbitMQ, or even PostgreSQL LISTEN/NOTIFY) is probably sufficient and far easier to operate. Kafka earns its complexity at scale.

KRaft Mode: Why ZooKeeper is No Longer Needed

Before Kafka 3.x, every Kafka deployment required a separate Apache ZooKeeper cluster to manage broker metadata, leader elections, and cluster state. This meant running and maintaining two distributed systems for every Kafka deployment — doubling the operational complexity.

KRaft (Kafka Raft Metadata mode) replaces ZooKeeper by moving cluster metadata management directly into Kafka itself, using the Raft consensus algorithm. The result:

• 
  Simpler deployment — one system to install, configure, monitor, and upgrade instead of two.


• 
  Faster startup and failover — Kafka no longer needs to synchronize with an external ZooKeeper cluster.


• 
  Better scalability — ZooKeeper had practical limits on the number of partitions it could track; KRaft removes those limits.


• 
  Kafka 4.0+: ZooKeeper support is completely removed. There is no option to use ZooKeeper with Kafka 4.x — KRaft is the only deployment mode.

Prerequisites

• Ubuntu 26.04 LTS (Resolute Raccoon) — fresh install or existing server


• Minimum 4GB RAM (8GB+ recommended for production workloads)


• At least 2 CPU cores


• 
  sudo access


• Internet access to download packages

Step 1: Install Java 21

Kafka 4.x requires Java 17 or higher. Java 21 LTS is the recommended choice for Ubuntu 26.04 — it's the current Long-Term Support release and ships cleanly from Ubuntu's default repository:

sudo apt update

sudo apt install -y openjdk-21-jdk-headless

Verify the installation:

java -version

Expected output:

openjdk version "21.x.x" ...

OpenJDK Runtime Environment (build 21.x.x+...)

Set JAVA_HOME so Kafka can find it:

echo 'export JAVA_HOME=/usr/lib/jvm/java-21-openjdk-amd64' | \

  sudo tee /etc/profile.d/java.sh

source /etc/profile.d/java.sh

Confirm:

echo $JAVA_HOME

# /usr/lib/jvm/java-21-openjdk-amd64

Step 2: Create a Dedicated Kafka User

Running Kafka as root is a security risk. Create a dedicated system user with no login access:

sudo useradd -r -m -U -d /opt/kafka -s /bin/false kafka

This follows the same principle of least privilege covered in our via kafka-exporter or JMX exporter gives you real-time visibility into consumer health.

Production Configuration Tips

A single-node Kafka installation on Ubuntu is sufficient for development and low-traffic production workloads. For anything that needs to scale or survive a broker failure, here are the key settings to revisit:

Memory tuning:

Kafka's JVM heap is set to 1GB by default. For production, adjust in /opt/kafka/bin/kafka-server-start.sh:

export KAFKA_HEAP_OPTS="-Xmx4G -Xms4G"

Set both -Xmx and -Xms to the same value to prevent JVM heap resizing pauses.

Log retention by size, not just time:

# Retain logs for 7 days OR until they exceed 10GB per partition, whichever comes first

log.retention.hours=168

log.retention.bytes=10737418240

Auto topic creation in production:

# Disable auto-creation — require topics to be created explicitly

auto.create.topics.enable=false

Auto-created topics use default partition/replication settings, which are rarely correct for specific use cases. Disable this and create topics explicitly with the right parameters.

Replication factor for multi-broker clusters:

# With 3 brokers, use replication factor 3 and min ISR 2

# (data survives loss of any 1 broker, writes require 2 brokers to acknowledge)

default.replication.factor=3

min.insync.replicas=2

Regular backups:

The /var/lib/kafka/data directory contains all event data. Back it up the same way you'd back up any other data volume — a scheduled snapshot with retention policy, tested restore process, stored separately from the host. The scripted approach from our to track consumer group lag, message throughput, and broker health in real time.