Spring Batch DB Cluster Partitioning 3.0.0 — distributed partitioning with just a database, now on Spring Boot 4 & Java 21

<dependency>

    <groupId>io.github.jchejarla</groupId>

    <artifactId>spring-batch-db-cluster-core</artifactId>

    <version>3.0.0</version>

</dependency>

The 30-second mental model

Three small tables augment Spring Batch's own schema:

• 
  BATCH_NODES — node registry + heartbeats


• 
  BATCH_PARTITIONS — partition lifecycle: PENDING → CLAIMED → COMPLETED / FAILED
  


• 
  BATCH_JOB_COORDINATION — binds a job execution to its master node

Whichever node launches a job becomes the master for that execution (decentralized — there's no elected leader singleton). The master asks the database which nodes are alive, partitions the work across them, and workers poll for tasks they can claim transactionally. If a node dies, its heartbeat lapses, and its transferable partitions are recovered onto healthy nodes. That's the whole idea — and everything that makes it safe lives in the database's transactions.

What's new in 3.0.0

Platform upgrade

3.0.0 moves the whole line to Spring Boot 4.1 / Spring Batch 6 / Spring Framework 7, with a Java 21 baseline. The 2.x line stays on Spring Boot 3 for maintenance. Worker steps now run on virtual threads, and the master's completion/orphan monitors are off the shared ForkJoinPool.commonPool, so a busy master coordinating many jobs can't starve the common pool.

Load-aware assignment

A new LEAST_LOADED strategy assigns each partition to the node with the lowest live load, steering work away from nodes already busy with other jobs — instead of blindly round-robining.

Heartbeat-loss fencing

A node that loses its heartbeat now cancels its own in-progress tasks rather than plowing ahead, so it stops doing work the master is about to recover elsewhere. Fewer surprises during a partial outage.

Job-centric observability

Alongside the existing node-centric actuator endpoints, there's now a read-only BatchClusterQueryService and job-centric endpoints:

GET /actuator/batch-cluster-jobs

GET /actuator/batch-cluster-jobs/{jobExecutionId}

Given a job execution, you get the master node, partition count, a status histogram, and where each partition is running — handy for dashboards and debugging a stuck job.

Phase-timing capture (opt-in)

Turn on spring.batch.cluster.capture-phase-timings=true and the master records its coordination phases (received → partitioned → distributed → completion-detected) to an append-only table using the database clock — the raw basis for coordination-overhead reporting.

Seven databases

3.0.0 adds SQL Server, MariaDB, and Db2, joining PostgreSQL, MySQL, Oracle, and H2. Each dialect is exercised by an opt-in Testcontainers cross-database validation suite, so the per-database SQL isn't validated against H2 alone anymore.

Correctness fixes worth calling out

Distributed coordination lives or dies on the edge cases, so a few of these matter:

• 
  A failed partition now fails the job — regardless of whether you wired a custom aggregator. Previously a FAILED partition could slip through as job success.


• 
  Compare-and-set partition transitions — a partition that already reached a terminal state can never be resurrected and re-run, closing a race between a briefly-stalled node completing and the master reassigning its partition.


• 
  Skew-proof heartbeats — registration and heartbeat timestamps are written with the database clock, so liveness is judged by a single clock and is immune to skew between a node and the database.

Breaking changes (read before you upgrade)

This is a major release; a few things changed on purpose:

1. 
   A JDBC JobRepository is now required. Spring Batch 6 defaults to an in-memory ResourcelessJobRepository, which can't coordinate a cluster. Opt into JDBC with @EnableBatchProcessing + @EnableJdbcJobRepository. If clustering is on with the resourceless repo, startup now fails fast with an actionable message instead of silently doing nothing.


2. 
   ClusterAwareAggregator takes a JobRepository constructor argument now (Batch 6 removed the old JobExplorer-based constructor).


3. 
   Spring Batch schema auto-init was removed by Spring Boot 4. Create the Batch schema with Flyway/Liquibase or spring.sql.init; the cluster auto-DDL runs after it.


4. 
   spring.batch.cluster.node-id is gone — node ids are generated as <prefix>-<uuid>, unique per JVM and per restart. Use the optional node-id-prefix to control the readable part.


5. 
   SCALE_UP mode was removed (it duplicated round-robin); assignment strategies now receive List<ClusterNode> (carrying live load) instead of List<String>.