> ## Content Index
> Fetch the complete content index at: https://www.ggorantala.dev/llms.txt
> Use this file to discover other available public pages before exploring further.

# Kafka "Connection to node -1 could not be established" — Fix
- URL: https://www.ggorantala.dev/kafka-connection-to-node-1-broker-may-not-be-available/
- Published: 2026-02-03T21:06:00.000Z
- Updated: 2026-08-29T08:18:40.000Z
- Description: Fix Kafka's "Connection to node -1 (localhost/127.0.0.1:9092) could not be established. Broker may not be available" — the Docker advertised.listeners fix.
- Author: Gopi Gorantala
- Tags: kafka-errors, kafka-docker, advertised-listeners, spring-kafka, kafka-connection-refused, Java, docker-compose

## 1\. The Error

You start a producer, consumer, or Spring Boot app against a Kafka broker running in Docker, and the log fills with this on repeat:

```text
[kafka-admin-client-thread | adminclient-1] WARN org.apache.kafka.clients.NetworkClient -
[AdminClient clientId=adminclient-1] Connection to node -1 (localhost/127.0.0.1:9092)
could not be established. Broker may not be available.
```

Or the producer variant:

```text
WARN org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1]
Connection to node -1 (localhost/127.0.0.1:9092) could not be established.
Broker may not be available.
```

Left alone, it escalates into a hard failure:

```text
org.apache.kafka.common.errors.TimeoutException:
Topic demo-topic not present in metadata after 60000 ms.
```

There's a second, closely related signature that trips people up even more — the node ID is **positive** and the host is a Docker-internal name:

```text
WARN org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-demo-1, groupId=demo]
Connection to node 1 (kafka/172.18.0.5:9092) could not be established.
Broker may not be available.
```

Those two are different bugs, and the node ID tells you which one you have. This article covers both.

**Typical setup:** Kafka 3.x/4.x in Docker (`apache/kafka`, `confluentinc/cp-kafka`, `bitnami/kafka`), `kafka-clients` 3.7+, Spring Boot 3.x with `spring-kafka` 3.2+, client running on the host machine or in a sibling container.

## 2\. How to Reproduce It (step-by-step)

Here is a KRaft single-broker compose file with the classic mistake baked in:

```yaml
# docker-compose.yml — BROKEN on purpose
services:
  kafka:
    image: apache/kafka:3.7.0
    container_name: kafka
    ports:
      - "9092:9092"
    environment:
      KAFKA_NODE_ID: 1
      KAFKA_PROCESS_ROLES: broker,controller
      KAFKA_CONTROLLER_QUORUM_VOTERS: 1@kafka:9093
      KAFKA_LISTENERS: PLAINTEXT://0.0.0.0:9092,CONTROLLER://0.0.0.0:9093
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:9092        # <-- the bug
      KAFKA_CONTROLLER_LISTENER_NAMES: CONTROLLER
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,CONTROLLER:PLAINTEXT
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
```

```bash
docker compose up -d
```

Now hit it from the **host** with a plain console producer (or any Java client):

```bash
# from the host machine
kafka-console-producer.sh --bootstrap-server localhost:9092 --topic demo-topic
```

The initial TCP connection to `localhost:9092` **succeeds** — the port is published. The client fetches metadata, the broker says "reach me at `kafka:9092`", and the client starts dialing a hostname that doesn't resolve on your host:

```text
WARN [Producer clientId=console-producer] Connection to node 1 (kafka/172.18.0.5:9092)
could not be established. Broker may not be available.
```

To reproduce the **node -1** variant instead, do any of these:

```bash
# 1. Broker simply not running / still starting
docker compose stop kafka

# 2. Port not published — delete the ports: mapping and restart

# 3. Wrong port in the client
kafka-console-producer.sh --bootstrap-server localhost:9093 --topic demo-topic
```

Same reproduction in Spring Boot, `spring-kafka` 3.2.x:

```properties
# application.properties
spring.kafka.bootstrap-servers=localhost:9092
```

```java
// kafka-clients 3.7.0 — plain producer repro
Properties props = new Properties();
props.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092");
props.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class);
props.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, StringSerializer.class);

try (var producer = new KafkaProducer<String, String>(props)) {
    producer.send(new ProducerRecord<>("demo-topic", "k", "v")).get(); // hangs, then TimeoutException
}
```

**Environment-specific trigger matrix** — this is why "works on my machine" is rampant with this error:

| Client location                          | Bootstrap that works | What breaks                                                      |
| ---------------------------------------- | -------------------- | ---------------------------------------------------------------- |
| Host machine                             | localhost:9092       | Advertised listener says kafka:9092 → node 1 unreachable         |
| Sibling container (same compose network) | kafka:9092           | Advertised listener says localhost:9092 → container dials itself |
| Another machine / k8s pod                | broker-dns:9092      | Advertised listener says internal IP → unreachable               |

## 3\. Why It Happens — Surface Level

**`node -1`**: the client can't complete a TCP connection to the bootstrap address you configured. Nothing Kafka-specific yet — the broker is down, still starting (KRaft format + startup takes a few seconds), the port isn't published from the container, a firewall is in the way, or you typed the wrong port.

**`node 1` (or any non-negative ID)**: bootstrap worked. The client successfully fetched cluster metadata, and the broker told it to connect to the address in `advertised.listeners`. That address is only resolvable/routable inside the Docker network, so every subsequent connection fails. Your bootstrap config is fine; the **broker's** config is lying to the client.

## 4\. Why It Happens — Under the Hood

Kafka clients never talk only to the bootstrap servers. Bootstrap is a phonebook lookup, not a proxy:

1. The client opens a connection to a bootstrap address. Internally, `NetworkClient` assigns bootstrap endpoints **synthetic negative node IDs**: `-1` for the first, `-2` for the second, and so on. That's what `node -1` means — "the first bootstrap server, before I know anything about the real cluster."
2. Over that connection it sends a `MetadataRequest`. The response contains the real broker list — node IDs and, crucially, the **host:port each broker advertises** (`advertised.listeners`), not the address you dialed.
3. The bootstrap connection is discarded for data traffic. Produce requests go to the advertised address of each partition leader; fetch requests likewise; consumer group coordination goes to the advertised address of the group coordinator (chosen by hashing `group.id` against `__consumer_offsets` partitions).

So there are two independent hops, and each has its own failure signature:

- **Hop 1 fails → `node -1`**: pure reachability. The client retries with exponential backoff (`reconnect.backoff.ms` → `reconnect.backoff.max.ms`) until `max.block.ms` (producer, default 60 s) or the operation timeout expires, then throws `TimeoutException`.
- **Hop 2 fails → `node <brokerId>`**: metadata poisoning. The broker resolves `advertised.listeners` at startup and hands that string to every client verbatim. Docker's default hostname inside the network (`kafka`, or the container ID) means nothing on your host; `localhost` inside a container means the container itself. There is no fallback — the client will not "just use the address that worked for bootstrap." That's by design: in a multi-broker cluster, the bootstrap address and the leader of your partition are usually different machines.

One more subtlety: `listeners` controls what interface/port the broker **binds**; `advertised.listeners` controls what it **tells clients**. Binding `0.0.0.0:9092` and advertising `kafka:9092` is legal and common — and exactly how this bug is born. In KRaft mode the `CONTROLLER` listener adds noise, but it's only used for quorum traffic between nodes; clients never touch it.

## 5\. The Fix

### Fix A — node -1: make the bootstrap address reachable

No Kafka config change; it's plumbing. Verify in order:

```bash
docker ps                                  # is the container up?
docker logs kafka | grep -i "started"      # "Kafka Server started" present?
docker port kafka                          # is 9092 published to the host?
nc -vz localhost 9092                      # can you complete a TCP handshake?
```

If `nc` fails, fix the `ports:` mapping, the port number in the client, or wait for the broker to finish starting (add a healthcheck — see §7).

### Fix B — node 1: advertise one listener per network path

Give the broker **two listeners**: one advertised for containers on the Docker network, one advertised for the host. Diff against the broken compose file:

```diff
     ports:
       - "9092:9092"
     environment:
       KAFKA_NODE_ID: 1
       KAFKA_PROCESS_ROLES: broker,controller
       KAFKA_CONTROLLER_QUORUM_VOTERS: 1@kafka:9093
-      KAFKA_LISTENERS: PLAINTEXT://0.0.0.0:9092,CONTROLLER://0.0.0.0:9093
-      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:9092
+      KAFKA_LISTENERS: PLAINTEXT://0.0.0.0:29092,PLAINTEXT_HOST://0.0.0.0:9092,CONTROLLER://0.0.0.0:9093
+      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:29092,PLAINTEXT_HOST://localhost:9092
+      KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT
       KAFKA_CONTROLLER_LISTENER_NAMES: CONTROLLER
-      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,CONTROLLER:PLAINTEXT
+      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT,CONTROLLER:PLAINTEXT
       KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
```

Now both paths work, each getting back an address valid for **its** network:

```bash
# from the host
kafka-console-producer.sh --bootstrap-server localhost:9092 --topic demo-topic

# from a sibling container
docker run --rm -it --network <compose_network> apache/kafka:3.7.0 \
  /opt/kafka/bin/kafka-console-producer.sh --bootstrap-server kafka:29092 --topic demo-topic
```

**Which fix when:**

- **POC / laptop:** Fix B as shown. One broker, two listeners, done.
- **Client and broker both in compose (no host access needed):** skip the host listener entirely; advertise `kafka:9092` and point your app container at `kafka:9092`. Fewer moving parts.
- **Remote/production brokers:** advertise the DNS name clients actually resolve (LB or broker FQDN). Never advertise `0.0.0.0` — the broker will refuse to start with `advertised.listeners cannot use the nonroutable meta-address 0.0.0.0`.
- **Kubernetes:** same principle, harder plumbing — per-broker `advertised.listeners` pointing at per-pod Services or NodePorts. Strimzi/operator-managed listeners exist precisely because of this error.

## 6\. Best Practices & The Better Design

The robust pattern is **one named listener per network path a client can arrive from**, always. Internal (inter-broker + in-network apps), host/external, and controller are separate concerns; name them accordingly:

```yaml
# docker-compose.yml — the right way, KRaft, apache/kafka:3.7.0
services:
  kafka:
    image: apache/kafka:3.7.0
    container_name: kafka
    ports:
      - "9092:9092"
    environment:
      KAFKA_NODE_ID: 1
      KAFKA_PROCESS_ROLES: broker,controller
      KAFKA_CONTROLLER_QUORUM_VOTERS: 1@kafka:9093
      KAFKA_LISTENERS: INTERNAL://0.0.0.0:29092,EXTERNAL://0.0.0.0:9092,CONTROLLER://0.0.0.0:9093
      KAFKA_ADVERTISED_LISTENERS: INTERNAL://kafka:29092,EXTERNAL://localhost:9092
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: INTERNAL:PLAINTEXT,EXTERNAL:PLAINTEXT,CONTROLLER:PLAINTEXT
      KAFKA_INTER_BROKER_LISTENER_NAME: INTERNAL
      KAFKA_CONTROLLER_LISTENER_NAMES: CONTROLLER
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
      KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1
      KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1
    healthcheck:
      test: ["CMD-SHELL", "/opt/kafka/bin/kafka-broker-api-versions.sh --bootstrap-server localhost:9092 >/dev/null 2>&1"]
      interval: 5s
      timeout: 10s
      retries: 12

  app:
    build: .
    depends_on:
      kafka:
        condition: service_healthy
    environment:
      SPRING_KAFKA_BOOTSTRAP_SERVERS: kafka:29092   # in-network clients use INTERNAL
```

Notes on why this shape:

- **Explicit `INTERNAL`/`EXTERNAL` names** make the intent readable and survive adding TLS later (flip the protocol map entry, not the topology).
- **`kafka-broker-api-versions.sh` as the healthcheck** validates the full Kafka protocol handshake, not just an open TCP port — it catches "port open, broker still initializing."
- **`depends_on: condition: service_healthy`** kills the entire class of node -1 warnings caused by app containers racing the broker at startup.
- Client-side, fail fast in dev instead of hanging 60 seconds:

```java
props.put(ProducerConfig.MAX_BLOCK_MS_CONFIG, "10000");           // default 60000
props.put(CommonClientConfigs.REQUEST_TIMEOUT_MS_CONFIG, "5000"); // surface problems quickly
```

## 7\. How to Prevent It Long-Term

**Diagnose by node ID, institutionally.** Put it in your team runbook: negative node ID → check reachability of the bootstrap address from *where the client runs*; positive node ID → check what the broker advertises:

```bash
docker exec kafka /opt/kafka/bin/kafka-broker-api-versions.sh \
  --bootstrap-server localhost:9092 | head -1
# prints:  kafka:29092 (id: 1 rack: null) ...  <- the advertised address clients will dial
```

**One shared compose file.** Advertised-listener bugs breed when every dev hand-rolls their own Kafka container. Keep a blessed `docker-compose.yml` (the §6 one) in a platform repo and treat changes to it like code.

**CI smoke test.** A 20-line integration test with Testcontainers (`org.testcontainers:kafka`) catches broken listener setups before they hit a shared environment — Testcontainers configures dual listeners for you and exposes `kafka.getBootstrapServers()`, which is itself a hint that this problem is universal enough to be library-handled.

**Monitor connection health, not just lag.** `kafka.consumer:type=consumer-metrics` → `connection-close-rate` and the producer equivalent spike when clients flap against unreachable advertised addresses. In production, alert on sustained reconnect storms; they precede the `TimeoutException` flood.

**Environment parity checks.** On boot, log the resolved bootstrap servers and fail hard if DNS resolution fails, instead of letting the app spin on WARN logs. Spring Boot: set `spring.kafka.admin.fail-fast=true` so a misconfigured broker fails deployment at startup rather than at first produce.

## 8\. Key Takeaways

- `node -1` \= can't reach the **bootstrap** address you configured. `node 1` (any positive ID) = bootstrap worked, but the **advertised listener** the broker returned is unreachable from where your client runs. The node ID is the diagnosis.
- Bootstrap is a phonebook, not a proxy: clients always reconnect to `advertised.listeners`, so the address that works for bootstrap is irrelevant for data traffic.
- In Docker, declare one listener per network path — `INTERNAL://kafka:29092` for containers, `EXTERNAL://localhost:9092` for the host — and map them in `KAFKA_LISTENER_SECURITY_PROTOCOL_MAP`.
- `kafka-broker-api-versions.sh` shows exactly what address the broker is advertising; use it as both a debugging tool and a compose healthcheck.
- Gate app startup on broker health (`depends_on: service_healthy`) and set `spring.kafka.admin.fail-fast=true` so misconfiguration fails loudly at deploy time, not silently at runtime.