> ## Content Index > Fetch the complete content index at: https://www.ggorantala.dev/llms.txt > Use this file to discover other available public pages before exploring further. # Kafka LEADER_NOT_AVAILABLE: Why It Happens and How to Fix It - URL: https://www.ggorantala.dev/kafka-leader-not-available-fix/ - Published: 2026-06-19T09:54:00.000Z - Updated: 2026-08-29T08:27:55.000Z - Description: Kafka LEADER_NOT_AVAILABLE floods your producer logs the moment you write to a fresh topic. Here's the exact cause, the reproduction, and the real fix. - Author: Gopi Gorantala - Tags: kafka-errors, LEADER_NOT_AVAILABLE, kafka-producer, kafka-docker, spring-kafka, Java, event-streaming You start a producer, aim it at a brand-new topic, and the console lights up: ```text WARN [Producer clientId=producer-1] Error while fetching metadata with correlation id 3 : {orders=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient) WARN [Producer clientId=producer-1] Error while fetching metadata with correlation id 4 : {orders=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient) WARN [Producer clientId=producer-1] Error while fetching metadata with correlation id 5 : {orders=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient) ``` Two very different things hide behind that single line. Sometimes it clears in a second and your record lands — a transient election blip you should ignore. Sometimes it loops forever until `max.block.ms` fires and you get: ```text org.apache.kafka.common.errors.TimeoutException: Topic orders not present in metadata after 60000 ms. ``` This article separates the benign case from the broken one, and gives you the fix for each. ## 1\. The Error `LEADER_NOT_AVAILABLE` is Kafka error code **5** (`LeaderNotAvailableException`). The broker is telling the client: *this topic-partition exists (or is being created), but right now there is no leader replica you can produce to or fetch from.* It is a **retriable** error — the client is supposed to refresh metadata and try again, which is exactly why it surfaces as a `WARN` in `NetworkClient` and not a thrown exception, at least at first. Do not confuse it with its two neighbors: | Error | Code | Meaning | | ----------------------------- | ---- | -------------------------------------------------------------------------------------- | | UNKNOWN\_TOPIC\_OR\_PARTITION | 3 | The topic/partition doesn't exist on this broker at all. | | LEADER\_NOT\_AVAILABLE | 5 | The partition exists but has no elected leader *right now*. | | NOT\_LEADER\_OR\_FOLLOWER | 6 | This broker is neither leader nor follower — your metadata points at the wrong broker. | Versions where this shows up unchanged: `kafka-clients` 3.x–3.9 and 4.0, Spring Kafka 3.x, against KRaft or ZooKeeper brokers. The message wording (`Error while fetching metadata with correlation id ...`) has been stable since 2.x, so search hits from older posts still apply. Typical setups where it bites: - **Docker/local POC** — first produce to an auto-created or just-created topic. - **Managed Kafka / K8s** — during a rolling broker restart or a partition reassignment. - **CI** — a test produces immediately after `AdminClient.createTopics()` returns. ## 2\. How to Reproduce It ### Minimal KRaft cluster ```yaml # docker-compose.yml services: kafka: image: apache/kafka:3.9.0 container_name: kafka ports: - "9092:9092" environment: KAFKA_NODE_ID: 1 KAFKA_PROCESS_ROLES: broker,controller KAFKA_CONTROLLER_QUORUM_VOTERS: 1@kafka:9093 KAFKA_LISTENERS: PLAINTEXT://:19092,CONTROLLER://:9093,EXTERNAL://:9092 KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:19092,EXTERNAL://localhost:9092 KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,CONTROLLER:PLAINTEXT,EXTERNAL:PLAINTEXT KAFKA_CONTROLLER_LISTENER_NAMES: CONTROLLER KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1 KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1 KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1 KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: 0 KAFKA_AUTO_CREATE_TOPICS_ENABLE: "true" ``` ```bash docker compose up -d ``` ### Trigger A — transient (auto-creation race) Produce to a topic that doesn't exist yet, with auto-create on: ```bash docker exec -it kafka /opt/kafka/bin/kafka-console-producer.sh \ --bootstrap-server localhost:19092 \ --topic orders ``` The first send races topic creation. The controller creates `orders`, but the leader for partition 0 isn't elected and propagated to the broker's metadata cache the instant the producer asks. You'll see one or two `LEADER_NOT_AVAILABLE` WARNs, then it clears and your message goes through. This is normal. Kafka's own docs and the `kafka-topics.sh` tooling emit this on first touch of a new topic. ### Trigger B — persistent (broker restart mid-produce) Start a steady producer, then bounce the broker while it runs: ```java // LeaderNotAvailableDemo.java (kafka-clients 3.9.0) Properties p = new Properties(); p.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092"); p.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class); p.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, StringSerializer.class); p.put(ProducerConfig.MAX_BLOCK_MS_CONFIG, 60000); try (var producer = new KafkaProducer(p)) { for (int i = 0; ; i++) { producer.send(new ProducerRecord<>("orders", "k" + i, "v" + i), (md, ex) -> { if (ex != null) ex.printStackTrace(); }); Thread.sleep(500); } } ``` ```bash # while it runs, in another terminal: docker restart kafka ``` During the restart window the partition has no online leader. The producer parks the batch and logs `LEADER_NOT_AVAILABLE` repeatedly until the broker comes back and elects a leader. On a single-broker cluster (RF=1) there is no other replica to take over, so the gap lasts the full restart. On RF≥3 with a healthy ISR, a follower is promoted in well under a second and you'd barely notice. ### Trigger C — the one that never clears Misconfigure `advertised.listeners` so the client can reach the bootstrap endpoint but the returned leader host is unroutable, or create a topic whose partitions can't get a leader (e.g. all replicas offline). The WARN loops until `max.block.ms` and you get the `TimeoutException: Topic orders not present in metadata after 60000 ms`. This is the case people actually need to fix. ## 3\. Why It Happens — Surface Level A Kafka producer can't send to a partition until it knows which broker leads it. It gets that from a `MetadataResponse`. If the partition currently has no leader — because it's mid-creation, mid-election, or its replicas are offline — the broker returns `LEADER_NOT_AVAILABLE` for that topic in the metadata response. The client caches nothing usable, waits `retry.backoff.ms`, and asks again. So the WARN is not itself a failure. It's the client doing the right thing: retrying a retriable metadata condition. It only becomes a real problem when the condition never resolves, because then the retries exhaust `max.block.ms` (in `send()`/metadata wait) or `delivery.timeout.ms` (for a batch already accepted). ## 4\. Why It Happens — Under the Hood Every partition has a leader replica; all reads and writes go through it. The controller (the KRaft quorum controller, or the ZooKeeper-era controller broker) decides leadership and publishes it. When leadership is unknown, the metadata `leader` field is set to `-1` and the partition carries the `LEADER_NOT_AVAILABLE` error. There are three mechanically distinct ways to land there: **Topic creation lag.** `createTopics` (or auto-creation) is asynchronous with respect to the produce path. The controller writes the topic to the metadata log, then leaders get assigned and the assignment propagates to each broker's `MetadataCache`. A producer that fetches metadata inside that window sees the topic with no leader yet. Under KRaft this window is small but non-zero; the broker applies metadata records from the controller as a log it replays, so there's inherent propagation delay. **Leader election in flight.** During a controlled shutdown, rolling restart, or preferred-leader rebalance, the current leader steps down and a new one is elected from the ISR. Between step-down and the new leader's election being applied on the broker your client is talking to, that partition reports no leader. Cleanly-run clusters make this sub-second; the exposure scales with how fast the controller elects and how fast each broker applies the update. **No electable replica.** If every replica for the partition is offline, or `min.insync.replicas` can't be met and unclean election is disabled (`unclean.leader.election.enable=false`, the safe default), there is nothing to elect. `LEADER_NOT_AVAILABLE` then persists until you restore a replica. This is the genuinely broken state, and it usually rides alongside under-replicated-partition alarms. The client side is governed by two clocks. `max.block.ms` (default 60s) bounds how long `send()` blocks waiting for metadata before throwing `TimeoutException: Topic ... not present in metadata`. Once a batch is accepted into the accumulator, `delivery.timeout.ms` (default 120s) is the total budget for delivery including these metadata retries. `retry.backoff.ms` (default 100ms) paces the retries. So a "flood" of WARNs is just `(elapsed / retry.backoff.ms)` lines — noisy, but bounded. ## 5\. The Fix First decide which case you're in. If the WARN clears within a second or two and records land, **there is nothing to fix** — see best practices for silencing the noise. If it persists to a `TimeoutException`, fix the cluster or the config. ### Fix A — don't produce before the topic has a leader (POC / CI) Create the topic and wait for it, instead of racing auto-creation: ```diff - // producer sends immediately, racing auto-creation - producer.send(new ProducerRecord<>("orders", key, value)); + try (Admin admin = Admin.create(Map.of( + AdminClientConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092"))) { + admin.createTopics(List.of(new NewTopic("orders", 3, (short) 1))) + .all().get(); // block until controller acks + } + // now the leader is elected before the first send + producer.send(new ProducerRecord<>("orders", key, value)); ``` `createTopics(...).all().get()` returning does **not** guarantee the leader has propagated to every broker's cache, so in tight CI loops also give the producer a real budget rather than fighting the race: ```diff Properties p = new Properties(); - p.put(ProducerConfig.MAX_BLOCK_MS_CONFIG, 5000); + p.put(ProducerConfig.MAX_BLOCK_MS_CONFIG, 60000); // absorb propagation lag ``` ### Fix B — turn off auto-creation so the race can't happen Auto-created topics are the single biggest source of these WARNs in dev because the produce path *is* the creation trigger. Turn it off and provision topics explicitly: ```properties # broker auto.create.topics.enable=false ``` Now a missing topic fails fast and loudly (`UNKNOWN_TOPIC_OR_PARTITION` → `TimeoutException`) instead of half-creating under load. You create topics deliberately via `AdminClient`, `kafka-topics.sh`, or infra-as-code. ### Fix C — persistent case: restore a leader If it never clears, the partition has no electable leader. Check it: ```bash docker exec -it kafka /opt/kafka/bin/kafka-topics.sh \ --bootstrap-server localhost:19092 \ --describe --topic orders ``` ```text Topic: orders Partition: 0 Leader: none Replicas: 2 Isr: ``` `Leader: none` with an empty ISR means every replica is down. Bring the replica brokers back, or if a broker is permanently gone, reassign the partition to a live broker with `kafka-reassign-partitions.sh`. Do **not** reach for `unclean.leader.election.enable=true` as a reflex — it trades the outage for silent data loss. ### Fix D — persistent case: routing is wrong If `--describe` shows a healthy `Leader: 1` but your producer still loops, the leader host it's being handed is unreachable from the client. That's an `advertised.listeners` problem, not a leadership problem: ```diff - KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:9092 + KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:19092,EXTERNAL://localhost:9092 ``` The client resolves the advertised leader host directly; if it can't route there, every produce after metadata refresh fails even though the cluster is healthy. ## 6\. Best Practices & The Better Design **Provision topics as code, never by producing to them.** The correct-by-construction setup declares topics before any client writes. In Spring, register them as beans and let `KafkaAdmin` create them at startup: ```java @Configuration public class TopicConfig { @Bean public KafkaAdmin.NewTopics topics() { return new KafkaAdmin.NewTopics( TopicBuilder.name("orders").partitions(3).replicas(3) .config(TopicConfig.MIN_IN_SYNC_REPLICAS_CONFIG, "2") .build() ); } } ``` `KafkaAdmin` blocks the context until topics exist, so by the time a `@KafkaListener` or `KafkaTemplate` runs, leaders are elected. In production, prefer Strimzi `KafkaTopic` CRDs or Terraform so topic config lives in review, not in application startup timing. **Give restarts a real replication factor.** RF=1 makes every broker restart a guaranteed leaderless gap for its partitions. RF=3 with `min.insync.replicas=2` lets a follower take over in milliseconds, so a rolling restart produces at most a flicker of WARNs instead of a stall. Restart brokers one at a time and wait for ISR to fully recover between steps. **Set client timeouts that match reality.** Keep `max.block.ms` generous enough to swallow a normal election (the 60s default is fine; don't drop it to 1–2s in code that runs at startup), rely on 3.0+ default idempotent-producer retry behavior, and let `delivery.timeout.ms` be the single number that defines "give up." Don't paper over a broken cluster with a huge timeout — fix the ISR. ## 7\. How to Prevent It Long-Term Alert on the cluster conditions that make `LEADER_NOT_AVAILABLE` persistent rather than transient: - `kafka.controller:type=KafkaController,name=OfflinePartitionsCount` \> 0 — partitions with no leader. This is the direct signal. - `kafka.server:type=ReplicaManager,name=UnderReplicatedPartitions` \> 0 — ISR shrinking toward a leaderless state. - `kafka.controller:type=ControllerStats,name=LeaderElectionRateAndTimeMs` — spikes during restarts; sustained non-zero means flapping. - Producer client metric `record-error-rate` \> 0 — the WARNs turned into real failures. Team conventions that prevent it: - `auto.create.topics.enable=false` on every non-dev cluster; topics via IaC only. - Rolling-restart automation that gates each broker on `UnderReplicatedPartitions == 0` before proceeding. - A Testcontainers integration test that creates a topic, restarts the broker mid-produce, and asserts zero lost records on RF≥3 — this catches routing and timeout regressions before prod. - Treat a `LEADER_NOT_AVAILABLE` WARN in CI logs that resolves as noise to filter, not a failure to chase; wire the `TimeoutException` as the thing that fails the build. ## 8\. Key Takeaways - `LEADER_NOT_AVAILABLE` (code 5) is **retriable**: partition exists, no leader *right now*. It's not `UNKNOWN_TOPIC_OR_PARTITION` (doesn't exist) or `NOT_LEADER_OR_FOLLOWER` (wrong broker). - A brief burst on first-produce or during a restart is normal and self-heals; only a burst that ends in `TimeoutException: Topic ... not present in metadata after 60000 ms` is a real problem. - The dev fix is to stop racing auto-creation: `auto.create.topics.enable=false` plus explicit topic provisioning and a sane `max.block.ms`. - The persistent fix is a cluster fix: restore an electable replica (`Leader: none`) or correct `advertised.listeners` (leader reachable but unroutable). - Alert on `OfflinePartitionsCount` and `UnderReplicatedPartitions`; RF=3 + `min.insync.replicas=2` turns restart stalls into a flicker.