> ## Content Index
> Fetch the complete content index at: https://www.ggorantala.dev/llms.txt
> Use this file to discover other available public pages before exploring further.

# Kafka FencedInstanceIdException: Fix Static Membership Collisions
- URL: https://www.ggorantala.dev/kafka-fencedinstanceidexception-static-membership-fix/
- Published: 2026-01-06T15:18:00.000Z
- Updated: 2026-08-29T08:33:13.000Z
- Description: FencedInstanceIdException means two live consumers share one group.instance.id. Here's why the broker fences the old one, and how to fix it for good.
- Author: Gopi Gorantala
- Tags: #kafka-consumer-groups, kafka-errors, static-membership, kafka-consumer, spring-kafka, Java, event-streaming, fencedInstanceIdException

You turned on static membership to stop rolling restarts from triggering a rebalance every time. It worked — until one consumer started dying on startup with a fatal error, and your lag on a few partitions climbed while nothing rebalanced. This is the static-membership footgun: the broker is telling you two live consumers are claiming the same identity.

## 1\. The Error

The exact exception, thrown from `poll()`, `commitSync()`, or the background heartbeat thread:

```text
org.apache.kafka.common.errors.FencedInstanceIdException: The broker rejected this static consumer since
another consumer with the same group.instance.id has registered with a different member.id.

```

In Spring Kafka you'll see it wrapped, and the container stops:

```text
org.springframework.kafka.KafkaException: Seek to current after exception; nested exception is
org.apache.kafka.common.errors.FencedInstanceIdException: The broker rejected this static consumer...
...
o.s.k.l.KafkaMessageListenerContainer : Fatal consumer exception; stopping container

```

`FENCED_INSTANCE_ID` is broker error code **82**, introduced with static membership in **KIP-345** (Kafka **2.3**, requires both broker and client ≥ 2.3). It is a **fatal, non-retriable** error — the fenced consumer does not recover by re-polling. The offending setup is almost always Kafka on Docker/K8s with `group.instance.id` configured, kafka-clients 2.3+ or Spring Kafka 2.3+.

## 2\. How to Reproduce It

Stand up a single-broker KRaft cluster:

```yaml
# docker-compose.yml
services:
  kafka:
    image: apache/kafka:3.9.0
    ports:
      - "9092:9092"
    environment:
      KAFKA_NODE_ID: 1
      KAFKA_PROCESS_ROLES: broker,controller
      KAFKA_CONTROLLER_QUORUM_VOTERS: 1@kafka:9093
      KAFKA_LISTENERS: PLAINTEXT://0.0.0.0:9092,CONTROLLER://0.0.0.0:9093
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9092
      KAFKA_CONTROLLER_LISTENER_NAMES: CONTROLLER
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,CONTROLLER:PLAINTEXT
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
      KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: 0

```

```bash
docker compose up -d
docker exec -it $(docker ps -qf name=kafka) \
  /opt/kafka/bin/kafka-topics.sh --bootstrap-server localhost:9092 \
  --create --topic orders --partitions 6

```

Now start **two** consumers in the same group with the **same** `group.instance.id`:

```java
// StaticConsumer.java — run this exact class TWICE, in two terminals
import org.apache.kafka.clients.consumer.*;
import org.apache.kafka.common.serialization.StringDeserializer;
import java.time.Duration;
import java.util.*;

public class StaticConsumer {
  public static void main(String[] args) {
    Properties p = new Properties();
    p.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092");
    p.put(ConsumerConfig.GROUP_ID_CONFIG, "order-service");
    p.put(ConsumerConfig.GROUP_INSTANCE_ID_CONFIG, "order-consumer-1"); // identical on both
    p.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
    p.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);

    try (KafkaConsumer<String, String> c = new KafkaConsumer<>(p)) {
      c.subscribe(List.of("orders"));
      while (true) {
        c.poll(Duration.ofMillis(1000)).forEach(r ->
            System.out.printf("p%d @ %d%n", r.partition(), r.offset()));
      }
    }
  }
}

```

The first process joins and gets all six partitions. The moment the second process calls `poll()` and sends its `JoinGroup`, the coordinator hands the identity to the newcomer and fences the incumbent. The **first** process dies:

```text
Exception in thread "main" org.apache.kafka.common.errors.FencedInstanceIdException:
The broker rejected this static consumer since another consumer with the same group.instance.id...

```

The trigger is specifically **two simultaneously-alive consumers sharing one `group.instance.id`**. If the first process had crashed before the second started, the second would simply inherit the same assignment with no rebalance — that's static membership working as designed. Fencing only bites when both are alive.

## 3\. Why It Happens — Surface Level

`group.instance.id` is a *unique* identity within a consumer group. The group coordinator enforces uniqueness: exactly one live `member.id` may own a given `group.instance.id` at a time. When a new consumer joins with an instance ID that's already mapped to a different `member.id`, the newcomer wins and the incumbent is fenced. Every subsequent request from the old member (heartbeat, offset commit, join) is rejected with `FENCED_INSTANCE_ID`.

So the error is never a broker fault. It means you have two processes — or two threads — configured with the same static ID, both running at the same time.

## 4\. Why It Happens — Under the Hood

Dynamic membership (the default) gives each consumer an ephemeral `member.id` assigned by the coordinator on join. When a consumer leaves — session timeout, or an explicit `LeaveGroup` on close — the coordinator triggers a rebalance. That's exactly what you get pounded by during rolling restarts: every pod that bounces evicts its member and reshuffles assignments across the group.

Static membership (KIP-345) breaks that link. You supply a stable `group.instance.id`, and the coordinator persists the mapping `group.instance.id → member.id` in the group metadata. Two things change:

- On a **temporary** absence (a static member misses heartbeats but its session hasn't expired), the coordinator does **not** rebalance. It parks the assignment and waits. When the member returns with the same `group.instance.id`, it gets its old `member.id` and the same partitions back. That's why you raise `session.timeout.ms` with static membership — you want the window to cover a pod restart so no rebalance fires.
- On a **rejoin**, if the returning consumer presents a *different* `member.id` than the one the coordinator has stored for that instance ID, the coordinator treats it as a fresh registration, overwrites the stored `member.id`, and fences the previous one.

That second rule is the whole story behind the exception. During a normal restart the old process is dead, so the fence targets a member that no longer exists — harmless. But if the old process is *still running* (duplicate config, or a deploy where the old pod outlives the new one's join), the coordinator fences a live consumer. On its next heartbeat or commit, it receives `FENCED_INSTANCE_ID`, and because the identity now belongs to someone else, there is nothing to retry. KIP-345 deliberately made this fatal: silently continuing would mean two consumers fighting over the same partitions and double-committing offsets.

Note the broker-side caps that interact with this: `group.max.session.timeout.ms` (default 30 min) bounds how large a `session.timeout.ms` you can set, and `group.min.session.timeout.ms` (default 6 s) the floor. Static membership is useless if your session timeout is shorter than your restart gap — the member expires, the coordinator rebalances, and you've gained nothing.

## 5\. The Fix

The fix is always the same shape: make `group.instance.id` unique per live consumer. **Never hardcode it.**

**Before** — every replica ships the same baked-in ID:

```yaml
# application.yml — WRONG: all pods are "order-consumer-1"
spring:
  kafka:
    consumer:
      group-id: order-service
      properties:
        group.instance.id: order-consumer-1

```

**After** — derive it from the pod's stable ordinal. In a Kubernetes StatefulSet, `HOSTNAME` is `order-service-0`, `order-service-1`, … which is exactly the stable, unique identity static membership wants:

```yaml
# StatefulSet: HOSTNAME is the stable pod name (order-service-0, -1, ...)
spring:
  kafka:
    consumer:
      group-id: order-service
      properties:
        group.instance.id: ${HOSTNAME}          # unique + stable per pod
        session.timeout.ms: 45000               # cover your restart gap
        heartbeat.interval.ms: 3000

```

For raw kafka-clients, pull it from an env var you set per instance:

```java
// After: unique per process, stable across restarts of the same instance
p.put(ConsumerConfig.GROUP_INSTANCE_ID_CONFIG, System.getenv("POD_NAME")); // e.g. order-service-0

```

Two important variants:

**Spring `concurrency > 1`.** A `ConcurrentMessageListenerContainer` creates N consumer threads *in one JVM*, all sharing the same consumer config. If you set a static `group.instance.id` and `concurrency: 3`, all three threads would claim the same ID and fence each other instantly. Spring Kafka handles this for you since 2.3 by appending the child index (`order-service-0-1`, `-2`, …) to the configured `group.instance.id`. Rely on that — do **not** manually construct per-thread IDs, and be aware that if you drop below Spring 2.3 or bypass the container this suffixing disappears.

**You don't actually need static membership.** If your churn problem was really eager-assignor thrash, the lighter fix is cooperative rebalancing with no static IDs at all — remove `group.instance.id` entirely:

```properties
# Alternative fix: cooperative rebalancing instead of static membership
partition.assignment.strategy=org.apache.kafka.clients.consumer.CooperativeStickyAssignor

```

Cooperative rebalancing (KIP-429) lets unaffected consumers keep their partitions through a rebalance, so a single pod bounce no longer stops the world. Use static membership when restarts are frequent and you want *zero* rebalance on a known-transient absence; use cooperative-sticky when you want cheap rebalances without managing stable identities. They compose, but you only need static IDs for the former.

## 6\. Best Practices & The Better Design

Treat `group.instance.id` as infrastructure identity, not application config. It should be injected by the orchestrator from the workload's stable name, never a literal in a properties file that gets copied across replicas.

The right-way pattern for a Kubernetes deployment:

```yaml
# StatefulSet gives stable network identity AND stable ordinals.
# Deployment pods get random suffixes — fine as long as you inject the pod name.
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: order-service
spec:
  serviceName: order-service
  replicas: 3
  template:
    spec:
      containers:
        - name: app
          env:
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name   # order-service-0/1/2

```

```java
// Consumer config built from the injected identity
props.put(ConsumerConfig.GROUP_INSTANCE_ID_CONFIG, System.getenv("POD_NAME"));
props.put(ConsumerConfig.SESSION_TIMEOUT_MS_CONFIG, "45000");
props.put(ConsumerConfig.PARTITION_ASSIGNMENT_STRATEGY_CONFIG,
          "org.apache.kafka.clients.consumer.CooperativeStickyAssignor");

```

Deploy ordering matters as much as the ID. During a rolling update, ensure the old pod is fully terminated (its consumer closed) before or as the replacement joins. With unique per-ordinal IDs a StatefulSet's one-at-a-time, terminate-then-recreate rollout never produces two live consumers on the same ID — the fence can't happen. Blue/green or surge deployments (`maxSurge > 0`) are where you get overlap: the green pod joins while the blue pod on the same ordinal is still heartbeating, and one gets fenced. If you run static membership, keep `maxSurge: 0` for that workload, or accept that fencing is the expected handoff and let the losing consumer exit cleanly.

Finally, decide whether you need static membership at all. It exists to eliminate rebalances during *routine* restarts of long-lived, stateful consumers (heavy local state, expensive partition warm-up). For a stateless service that just processes and commits, cooperative-sticky rebalancing alone is simpler and has one fewer identity to manage.

## 7\. How to Prevent It Long-Term

Catch the collision before it reaches production:

- **Config lint / CI check.** Fail the build if `group.instance.id` is a static literal rather than a `${...}` placeholder. A duplicated hardcoded ID across replicas is the single most common cause.
- **Alert on `FENCED_INSTANCE_ID`.** Watch the client metric for fatal consumer errors and grep broker request logs for the code. A steady trickle of fences means a deploy that reliably overlaps; a burst means duplicated config just shipped.
- **Watch partition ownership and lag, not just rebalance rate.** Static membership's whole value is a flat rebalance rate — so a stuck partition won't show up there. Alert on per-partition consumer lag and on `assigned-partitions` dropping to zero on an instance that should own some.
- **Test the deploy, not just the code.** In staging, run a rolling restart and a surge deploy against the real consumer group and assert no `FencedInstanceIdException` and no lag spike. A two-instance Testcontainers test that starts both consumers with the same instance ID and asserts one is fenced pins the behavior so nobody "fixes" it by hardcoding an ID later.
- **Right-size `session.timeout.ms`.** Set it above your worst-case restart gap but under `group.max.session.timeout.ms`. Too low and static membership silently degrades to dynamic (rebalance on every restart); too high and a genuinely dead consumer's partitions stay unprocessed until the session finally expires.

## 8\. Key Takeaways

- `FencedInstanceIdException` (code 82) means **two live consumers share one `group.instance.id`** — the coordinator keeps the newcomer and fatally fences the incumbent.
- It is **not retriable**. Fix the identity; don't catch and re-poll.
- **Never hardcode `group.instance.id`.** Derive it from a stable per-instance value like the StatefulSet pod name (`${HOSTNAME}` / `POD_NAME`).
- With Spring `concurrency > 1`, let Spring Kafka 2.3+ suffix the ID per thread; don't build per-thread IDs yourself.
- If you only needed cheaper rebalances, `CooperativeStickyAssignor` without static IDs may be all you want — reserve static membership for stateful consumers where restart rebalances are genuinely expensive.