Liveness vs Readiness: Health Check Endpoints That Wont Restart-Loop Your Service

Liveness vs. Readiness

A single /health endpoint that returns 200 OK feels like enough — until your orchestrator restarts a perfectly healthy pod because its database was slow for two seconds. The fix is to stop treating "health" as one boolean and split it into two questions.

Liveness asks: is this process broken beyond repair? If the answer is yes, the only cure is a restart. A deadlocked event loop or a corrupted in-memory state qualifies. A missing database connection does not — restarting won't bring the database back.

Readiness asks: can this instance serve traffic right now? A service that just booted and hasn't warmed its caches, or one whose database is temporarily unreachable, is alive but not ready. The load balancer should route around it without killing it.

Getting this distinction wrong is the single most common cause of restart loops in production.

A liveness probe should be trivial

Liveness must not depend on anything external. If it checks the database, a database blip triggers a restart storm across every instance at once. Keep it dumb:

// Express

app.get('/livez', (req, res) => {

  res.status(200).json({ status: 'ok' });

});

If this endpoint responds at all, the process can accept connections and run JavaScript. That is exactly what liveness is meant to prove — nothing more.

A readiness probe checks dependencies

Readiness is where you actually verify the things a request needs: the database, a cache, a downstream API. Run the checks in parallel, give each a short timeout, and return 503 if any critical one fails.

const { Pool } = require('pg');

const pool = new Pool();



async function checkDb() {

  const c = await pool.connect();

  try { await c.query('SELECT 1'); return true; }

  finally { c.release(); }

}



app.get('/readyz', async (req, res) => {

  const checks = {};

  try {

    await Promise.race([

      checkDb(),

      new Promise((_, r) => setTimeout(() => r(new Error('timeout')), 2000)),

    ]);

    checks.database = 'ok';

  } catch (e) {

    checks.database = 'failing';

  }



  const healthy = Object.values(checks).every(v => v === 'ok');

  res.status(healthy ? 200 : 503).json({

    status: healthy ? 'ok' : 'degraded',

    checks,

  });

});

The Promise.race timeout matters: a health check that hangs is worse than one that fails, because it holds the probe open until the orchestrator's own timeout fires.

Use a standard response body

There's an IETF draft, application/health+json, that standardizes the shape. You don't have to adopt it wholesale, but its structure — an overall status plus a per-dependency breakdown — is worth copying so monitoring tools can parse every service the same way:

{

  "status": "pass",

  "checks": {

    "database:responseTime": [{ "status": "pass", "observedValue": 12, "observedUnit": "ms" }],

    "cache:connection":      [{ "status": "pass" }]

  }

}

Statuses are pass, warn, or fail. A warn lets you signal "the cache is down but I can still serve from the origin" without pulling the instance out of rotation.

The same pattern in FastAPI:

from fastapi import FastAPI, Response

import asyncpg



app = FastAPI()



@app.get("/livez")

async def livez():

    return {"status": "ok"}



@app.get("/readyz")

async def readyz(response: Response):

    checks = {}

    try:

        conn = await asyncpg.connect(timeout=2)

        await conn.execute("SELECT 1")

        await conn.close()

        checks["database"] = "ok"

    except Exception:

        checks["database"] = "failing"

    ok = all(v == "ok" for v in checks.values())

    response.status_code = 200 if ok else 503

    return {"status": "ok" if ok else "degraded", "checks": checks}

Wire it into Kubernetes

Point the two probe types at the two endpoints, and give readiness a longer failure threshold so brief blips don't yank you out of rotation:

livenessProbe:

  httpGet: { path: /livez, port: 8080 }

  periodSeconds: 10

  failureThreshold: 3

readinessProbe:

  httpGet: { path: /readyz, port: 8080 }

  periodSeconds: 5

  failureThreshold: 2

A few rules that save you

Never require auth on probe endpoints — the orchestrator can't log in. Never do heavy work (no full table scans, no writes). And never let readiness cascade: if service A's readiness pings service B's readiness, one outage takes down the whole graph. Check your dependencies, not your dependencies' dependencies.

Once your health endpoints return a consistent, structured body, it's worth exercising them the same way you test the rest of your API — hitting /readyz, asserting the 503-on-failure behavior, and saving those requests alongside your other collections. APIKumo is handy here: you can save your liveness and readiness calls in a shared workspace, assert on status codes and the checks object, and share a live docs page so everyone on the team knows exactly what a healthy response looks like.