Kafka RetriableCommitFailedException: Causes and Fix

You call commitAsync(), wire up a callback so you can log commit failures, and this shows up in your logs on the next coordinator hiccup:

org.apache.kafka.clients.consumer.RetriableCommitFailedException: Offset commit failed with a retriable exception. You should retry committing the latest consumed offsets.
	at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator$OffsetCommitResponseHandler.handle(ConsumerCoordinator.java:1198)
	at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator$OffsetCommitResponseHandler.handle(ConsumerCoordinator.java:1123)
	at org.apache.kafka.clients.consumer.internals.AbstractCoordinator$CoordinatorResponseHandler.onSuccess(AbstractCoordinator.java:1290)
	at org.apache.kafka.clients.consumer.internals.AbstractCoordinator$CoordinatorResponseHandler.onSuccess(AbstractCoordinator.java:1265)
Caused by: org.apache.kafka.common.errors.DisconnectException

The message tells you exactly what to do — "You should retry committing the latest consumed offsets." — which is a strong hint that the client is handing this back to you deliberately rather than dealing with it internally. That is the whole story of this exception, and it is the part almost every codebase gets wrong.

This is not CommitFailedException (fatal, your generation is gone), and it is not RebalanceInProgressException (code 27, a rebalance is mid-flight). This is the retriable commit failure: the commit didn't land, but nothing is broken, and the correct response is to try again.

Versions here are kafka-clients 3.x / 4.0 (KRaft-era), Spring Kafka 3.x. The class and its semantics have been stable since KIP-266 (Kafka 2.0) reworked commit timeouts.

1. The Error

The exact text, verbatim:

org.apache.kafka.clients.consumer.RetriableCommitFailedException: Offset commit failed with a retriable exception. You should retry committing the latest consumed offsets.

Sometimes with a Caused by, sometimes without, depending on which retriable error the broker returned:

Caused by: org.apache.kafka.common.errors.CoordinatorNotAvailableException
Caused by: org.apache.kafka.common.errors.NotCoordinatorException
Caused by: org.apache.kafka.common.errors.CoordinatorLoadInProgressException
Caused by: org.apache.kafka.common.errors.DisconnectException
Caused by: org.apache.kafka.common.errors.TimeoutException: The request timed out.

Where you see it depends on how you commit:

• commitAsync() — delivered to your OffsetCommitCallback as the exception argument. This is by far the most common way people meet it, because the callback is the only place the async path can report anything.
• commitSync() — you generally do not see this class. commitSync() retries retriable errors internally until default.api.timeout.ms (60s) elapses, then throws TimeoutException. More on why below.
• Spring Kafka — surfaces through the container's commit path; with AckMode.BATCH/RECORD the container owns the commit and logs it, but a bare consumer.commitAsync() in your listener behaves like the plain-client case.

The class hierarchy is the whole point:

KafkaException
 └─ ApiException
     └─ RetriableException
         └─ RetriableCommitFailedException   // "just retry"

RetriableCommitFailedException is a RetriableException. Its sibling CommitFailedException is not — it extends KafkaException directly and means "you can't retry this, your commit is permanently rejected because the group rebalanced you out." Two different classes, two different outcomes. Catching them the same way is the root of most incidents around this error.

2. How to Reproduce It

A single-broker local cluster plus a consumer that commits async and kills the broker mid-run reproduces it deterministically.

# docker-compose.yml — single-broker KRaft, Kafka 4.0
services:
  kafka:
    image: apache/kafka:4.0.0
    container_name: kafka
    ports:
      - "9092:9092"
    environment:
      KAFKA_NODE_ID: 1
      KAFKA_PROCESS_ROLES: broker,controller
      KAFKA_LISTENERS: PLAINTEXT://:9092,CONTROLLER://:9093
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9092
      KAFKA_CONTROLLER_LISTENER_NAMES: CONTROLLER
      KAFKA_CONTROLLER_QUORUM_VOTERS: 1@kafka:9093
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,CONTROLLER:PLAINTEXT
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
      KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1
      KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1

docker compose up -d
docker exec -it kafka /opt/kafka/bin/kafka-topics.sh \
  --create --topic orders --partitions 3 --replication-factor 1 \
  --bootstrap-server localhost:9092

A consumer that commits asynchronously and reports failures through the callback:

// kafka-clients 4.0.0
Properties props = new Properties();
props.put("bootstrap.servers", "localhost:9092");
props.put("group.id", "orders-consumer");
props.put("key.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
props.put("value.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
props.put("enable.auto.commit", "false");

try (KafkaConsumer<String, String> consumer = new KafkaConsumer<>(props)) {
    consumer.subscribe(List.of("orders"));
    while (true) {
        ConsumerRecords<String, String> records = consumer.poll(Duration.ofMillis(500));
        for (ConsumerRecord<String, String> r : records) {
            process(r);
        }
        // The naive pattern: fire-and-forget async commit, log on failure.
        consumer.commitAsync((offsets, exception) -> {
            if (exception != null) {
                // You WILL land here as RetriableCommitFailedException.
                log.error("commit failed for {}", offsets, exception);
            }
        });
    }
}

Now trigger it. While the consumer is running, take the coordinator away:

# Force a coordinator disconnect / unavailability window
docker pause kafka      # or: docker restart kafka
sleep 8
docker unpause kafka

During that window, commitAsync() sends an OffsetCommit request, the connection to the coordinator drops (DisconnectException) or the broker replies COORDINATOR_NOT_AVAILABLE / NOT_COORDINATOR while __consumer_offsets finds a new leader. The client wraps those retriable broker errors into RetriableCommitFailedException and hands them to your callback. On a healthy cluster you can also provoke it during any rolling restart, preferred-leader election, or __consumer_offsets partition leadership move.

3. Why It Happens — Surface Level

An offset commit is a network round-trip to the group coordinator, and that round-trip can fail transiently for reasons that have nothing to do with your consumer's health: the coordinator broker is briefly unreachable, the __consumer_offsets partition is between leaders, a request timed out, or the TCP connection dropped. None of these mean your consumer lost its partitions — it just means this particular commit didn't land right now. Kafka signals that by wrapping the broker's retriable error code in RetriableCommitFailedException instead of throwing the fatal CommitFailedException.

The offsets themselves are safe. Nothing was corrupted. The last successfully committed position is still whatever it was before this attempt. You simply have an uncommitted position that you can commit again.

4. Why It Happens — Under the Hood

Inside ConsumerCoordinator.OffsetCommitResponseHandler, every partition's error code in the OffsetCommit response is classified. Three buckets matter:

1. Fatal group errors — UNKNOWN_MEMBER_ID, ILLEGAL_GENERATION, FENCED_INSTANCE_ID. These mean the coordinator no longer recognizes your membership/generation. The handler resets member state and the commit surfaces as CommitFailedException — non-retriable, because retrying with a dead generation is pointless.
2. Rebalance-in-progress — REBALANCE_IN_PROGRESS (code 27). The generation is still alive but a cooperative rebalance is underway. This becomes RebalanceInProgressException, which — importantly — is not a subclass of RetriableCommitFailedException. You "retry" it by calling poll() to let the rebalance complete, not by immediately re-committing.
3. Retriable transport/coordinator errors — COORDINATOR_NOT_AVAILABLE (15), NOT_COORDINATOR (16), COORDINATOR_LOAD_IN_PROGRESS (14), REQUEST_TIMED_OUT, plus client-side DisconnectException. These become RetriableCommitFailedException. The commit failed for reasons that will very likely clear on the next attempt once coordinator discovery re-resolves and the connection re-establishes.

Now the critical asymmetry between the two commit APIs:

commitSync() retries internally. Since KIP-266 (Kafka 2.0), commitSync() loops on retriable errors — re-discovering the coordinator, backing off retry.backoff.ms, and re-sending — until default.api.timeout.ms (60s) is exhausted. Only then does it give up, and it throws TimeoutException, not RetriableCommitFailedException. So if you exclusively use commitSync(), you'll rarely see this class at all; you'll see timeouts instead.

commitAsync() does not retry — by design. Auto-retrying async commits would reorder them: imagine commit A (offset 100) fails and is queued for retry, commit B (offset 200) succeeds, then the retry of A lands and overwrites the committed offset back to 100. On the next rebalance you reprocess 100 records. To avoid this hazard, commitAsync() makes exactly one attempt and delivers any retriable failure straight to your callback. You are the retry policy. That's what "You should retry committing the latest consumed offsets" is telling you — and why blindly re-committing the stale offsets from a failed async callback is a bug, not a fix.

5. The Fix

The fix is how you handle the callback, not a config knob. There is no enable.commit.retries=true.

Wrong — treats a transient hiccup as a fatal error and often re-commits stale offsets:

// BEFORE — misclassifies and/or re-commits the wrong offsets
consumer.commitAsync((offsets, exception) -> {
    if (exception != null) {
        log.error("commit failed", exception);
        consumer.commitAsync(offsets, null);   // re-commits STALE offsets → reprocessing risk
    }
});

Right — let periodic async commits self-heal, and guarantee the final position with a synchronous commit:

// AFTER — async in the hot loop, sync on shutdown/rebalance
try (KafkaConsumer<String, String> consumer = new KafkaConsumer<>(props)) {
    consumer.subscribe(List.of("orders"), new ConsumerRebalanceListener() {
        @Override public void onPartitionsRevoked(Collection<TopicPartition> partitions) {
            consumer.commitSync();          // block until the boundary commit lands
        }
        @Override public void onPartitionsAssigned(Collection<TopicPartition> partitions) { }
    });

    while (running) {
        ConsumerRecords<String, String> records = consumer.poll(Duration.ofMillis(500));
        for (ConsumerRecord<String, String> r : records) process(r);

        // Async in the steady state. A single failure is harmless: the NEXT
        // commitAsync will commit a HIGHER offset and supersede it. Do NOT re-commit here.
        consumer.commitAsync((offsets, exception) -> {
            if (exception instanceof RetriableCommitFailedException) {
                log.warn("retriable commit failure, next cycle will supersede: {}", offsets);
            } else if (exception != null) {
                log.error("non-retriable commit failure", exception);
            }
        });
    }
} finally {
    // ...
}

// On shutdown, commit synchronously so nothing is left uncommitted:
try {
    consumer.commitSync();
} catch (Exception e) {
    log.error("final commit failed", e);
} finally {
    consumer.close();
}

The insight: a lone failed async commit doesn't need an explicit retry, because your next commitAsync() one poll cycle later commits a higher offset and makes the failed one irrelevant. You only need a hard, retrying commit at the boundaries where there is no "next cycle" — on partition revocation and on shutdown — and commitSync() already retries those internally.

If you genuinely need to retry a specific async commit (e.g. you commit per-record, not periodically), guard the retry with a generation/offset check so you never overwrite a newer committed position with an older one. The simplest correct choice is: don't do per-commit retries, use commitSync() at your commit points and let KIP-266 handle the retrying.

For low-throughput / POC consumers where simplicity beats latency, just use commitSync() everywhere and catch TimeoutException. For high-throughput consumers where you can't block the poll loop on every batch, use the async-steady-state + sync-at-boundaries pattern above.

6. Best Practices & The Better Design

The better design is to stop hand-rolling commit retry logic and let a container manage commit timing.

Spring Kafka — container-managed commits (AckMode.BATCH):

@Bean
public ConcurrentKafkaListenerContainerFactory<String, String> kafkaListenerContainerFactory(
        ConsumerFactory<String, String> cf) {
    var factory = new ConcurrentKafkaListenerContainerFactory<String, String>();
    factory.setConsumerFactory(cf);
    factory.getContainerProperties().setAckMode(ContainerProperties.AckMode.BATCH);
    return factory;
}

@KafkaListener(topics = "orders", groupId = "orders-consumer")
public void onMessage(ConsumerRecord<String, String> record) {
    process(record);   // container commits after the batch; retriable failures are handled for you
}

With container-managed AckMode.BATCH (the default) or RECORD, the container commits after processing and treats a RetriableCommitFailedException as what it is — it logs and moves on, because the next batch commit supersedes it. You never write commit-retry code, and you never risk the stale-offset overwrite bug. This is the single biggest reliability win for most teams.

Two config-level companions that reduce how often you hit the error at all:

• CooperativeStickyAssignor (default-eligible via [RangeAssignor, CooperativeStickyAssignor] since 3.0) keeps partitions across rebalances, so fewer commits race a rebalance boundary.
• Static membership (group.instance.id) suppresses needless rebalances on restart, shrinking the windows where the coordinator is transiently unavailable to your consumer.

And the design rule that makes retriable commit failures safe by construction: make your processing idempotent. If reprocessing a handful of records after an uncommitted position is a no-op (upsert by key, dedup on an idempotency key, transactional outbox on the write side), then a lost commit costs you nothing but a few redundant operations. Chasing exactly-once through commit gymnastics is a losing game; idempotent consumers turn "at-least-once + occasional retriable commit failure" into effectively-once.

7. How to Prevent It Long-Term

Retriable commit failures never fully disappear — coordinators move, brokers restart, networks blip. The goal is to make them a non-event.

• Alert on the right signal. A trickle of RetriableCommitFailedException during deploys/rebalances is normal and self-healing. What's not normal is rising consumer lag combined with commits that never succeed, or commit-latency-avg/commit-rate (JMX: kafka.consumer:type=consumer-coordinator-metrics) degrading persistently. Alert on lag and commit-rate, not on the exception count.
• Watch the coordinator layer. OfflinePartitionsCount > 0 on __consumer_offsets, frequent NOT_COORDINATOR, or COORDINATOR_LOAD_IN_PROGRESS bursts point at broker/ISR problems, not consumer bugs. Keep offsets.topic.replication.factor=3 and min.insync.replicas=2 in production so the offsets topic survives a broker loss.
• Ban the anti-pattern in review. A CI/lint rule that flags commitAsync callbacks which re-invoke commit* on the same offsets catches the stale-offset overwrite bug before it ships. Equally, flag catch blocks that treat RetriableCommitFailedException and CommitFailedException identically.
• Test it. A Testcontainers integration test that pauses/kills the broker for a few seconds mid-consumption asserts the consumer keeps its assignment, lag drains after recovery, and no records are lost. Chaos-test the commit path the same way you'd test failover.
• Prefer container-managed commits so the retry policy is one well-tested code path, not scattered per-listener callbacks.

8. Key Takeaways

• RetriableCommitFailedException = retry, CommitFailedException = fatal. One extends RetriableException; the other extends KafkaException. Never catch them the same way.
• commitAsync() never retries (to avoid reordering commits) — it hands the failure to your callback. commitSync() retries internally until default.api.timeout.ms, then throws TimeoutException.
• Don't re-commit stale offsets in the failed callback. The next periodic commitAsync() commits a higher offset and supersedes the failure. Only commit synchronously at boundaries: onPartitionsRevoked and shutdown.
• Let a container own commits (Spring Kafka AckMode.BATCH) so you never hand-roll retry logic or hit the stale-offset overwrite bug.
• Make processing idempotent. Then an occasional lost commit is a few redundant operations, not a correctness incident.