30 July 2026 · 7 min read · recloud engineering

Zero-downtime deployments on Kubernetes, done properly

"Kubernetes gives you zero-downtime deployments" is true the way "a piano gives you music" is true. The machinery is there; most clusters we audit still drop requests on every rollout. The failures are always the same four, and each has a small, specific fix.

1. Readiness is a contract, not a formality

A pod is sent traffic the moment its readiness probe passes. If your probe returns 200 before caches are warm and connection pools are open, users are your probe. Make readiness check the things the request path actually needs:

readinessProbe:
  httpGet: { path: /ready, port: 8080 }   # checks DB pool + downstream deps
  initialDelaySeconds: 5
  periodSeconds: 5
livenessProbe:
  httpGet: { path: /healthz, port: 8080 } # process-level only — never check deps here
  periodSeconds: 10

And keep dependencies out of liveness: a database blip that fails liveness probes turns one incident into a cluster-wide restart storm.

2. The termination race

On pod deletion, two things happen in parallel: the kubelet sends SIGTERM, and the endpoints controller removes the pod from load balancing. Nothing orders them. For a short window, traffic still arrives at a process that has been told to die. The fix is a preStop hook that keeps the pod serving while the endpoint removal propagates:

lifecycle:
  preStop:
    exec: { command: ["sleep", "8"] }
terminationGracePeriodSeconds: 40

Then have the application drain on SIGTERM: stop accepting, finish in-flight requests, close idle keep-alives. If you run an ingress controller, check its own drain behaviour too — the same race exists one hop earlier.

3. Rolling update budgets

Defaults (maxUnavailable: 25%) are wrong for anything latency-sensitive. For services that must not lose capacity during rollout:

strategy:
  rollingUpdate: { maxUnavailable: 0, maxSurge: 1 }

New capacity comes up and passes readiness before old capacity leaves. Slower, yes. That's the point.

4. PodDisruptionBudgets — because deploys aren't the only disruption

Node drains for kernel patches will happily evict every replica of your service at once unless you say otherwise:

apiVersion: policy/v1
kind: PodDisruptionBudget
metadata: { name: api-pdb }
spec:
  minAvailable: 2
  selector: { matchLabels: { app: api } }

Combine with topologySpreadConstraints so those replicas aren't all on the node being drained.

Verifying it, not believing it

Run a load generator through a full rollout and count non-200s — vegeta or k6 for ten minutes while you kubectl rollout restart. The first run almost always finds a dropped-connection window measured in seconds. After the four fixes above, the honest number is zero, and you have the report to prove it. Make that test part of CI for your critical services: zero-downtime is a property you regression-test, not a milestone you reach once.