> ## Content Index > Fetch the complete content index at: https://www.ggorantala.dev/llms.txt > Use this file to discover other available public pages before exploring further. # Kafka ProducerFencedException: Why It Happens and How to Fix It - URL: https://www.ggorantala.dev/kafka-producerfencedexception-fix/ - Published: 2026-01-06T13:54:00.000Z - Updated: 2026-08-29T08:26:15.000Z - Description: ProducerFencedException: there is a newer producer with the same transactionalId. What fences a Kafka transactional producer and the exact fix, explained. - Author: Gopi Gorantala - Tags: kafka-errors, producerfencedexception, kafka-transactions, exactly-one, spring-kafka, kafka-producer, Java Your transactional producer dies mid-flight with: ```text org.apache.kafka.common.errors.ProducerFencedException: There is a newer producer with the same transactionalId which fences the current one. ``` In Spring Kafka it usually surfaces wrapped: ```text org.springframework.kafka.core.KafkaProducerException: Failed to send; nested exception is org.apache.kafka.common.errors.ProducerFencedException: There is a newer producer with the same transactionalId which fences the current one. ``` And in Kafka Streams with `processing.guarantee=exactly_once_v2` you'll see it as the cause of a task migration: ```text org.apache.kafka.streams.errors.TaskMigratedException: Producer got fenced trying to commit a transaction; it means all tasks belonging to this thread should be migrated. Caused by: org.apache.kafka.common.errors.ProducerFencedException ``` Typical setup: `kafka-clients` 3.x, Spring Kafka 3.x / Spring Boot 3.x, transactional producers (`transactional.id` set), often during a rolling deploy on Kubernetes or right after a consumer group rebalance. The exception is **fatal** for the producer instance — no retry, no `abortTransaction()`, only `close()`. The message is also famously misleading: you can get it when there is **no** newer producer anywhere. More on that below. ## 2\. How to Reproduce It (step-by-step) Single KRaft broker: ```yaml # docker-compose.yml services: kafka: image: apache/kafka:3.7.0 ports: - "9092:9092" environment: KAFKA_NODE_ID: 1 KAFKA_PROCESS_ROLES: broker,controller KAFKA_CONTROLLER_QUORUM_VOTERS: 1@kafka:9093 KAFKA_CONTROLLER_LISTENER_NAMES: CONTROLLER KAFKA_LISTENERS: PLAINTEXT://:19092,CONTROLLER://:9093,EXTERNAL://:9092 KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:19092,EXTERNAL://localhost:9092 KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,CONTROLLER:PLAINTEXT,EXTERNAL:PLAINTEXT KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1 KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1 KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1 ``` ```bash docker compose up -d docker compose exec kafka /opt/kafka/bin/kafka-topics.sh \ --bootstrap-server localhost:19092 --create --topic payments --partitions 3 ``` Minimal transactional producer (`kafka-clients:3.7.0`): ```java import org.apache.kafka.clients.producer.*; import java.util.Properties; public class TxProducer { public static void main(String[] args) throws Exception { Properties p = new Properties(); p.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092"); p.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, "org.apache.kafka.common.serialization.StringSerializer"); p.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, "org.apache.kafka.common.serialization.StringSerializer"); p.put(ProducerConfig.TRANSACTIONAL_ID_CONFIG, "payments-tx"); // the trap KafkaProducer producer = new KafkaProducer<>(p); producer.initTransactions(); while (true) { producer.beginTransaction(); producer.send(new ProducerRecord<>("payments", "k", "v")); producer.commitTransaction(); Thread.sleep(1000); } } } ``` **Trigger #1 — two instances, same `transactional.id`:** ```bash java TxProducer & # instance A, commits happily sleep 5 java TxProducer # instance B, same transactional.id ``` The moment instance B finishes `initTransactions()`, instance A's next `send()`/`commitTransaction()` throws `ProducerFencedException`. Every time, deterministically. This is exactly what happens during a rolling deploy where the new pod comes up before the old one exits, or when you scale a deployment to 2+ replicas with a hardcoded `transactional.id`. **Trigger #2 — transaction runs longer than `transaction.timeout.ms`:** ```java p.put(ProducerConfig.TRANSACTION_TIMEOUT_CONFIG, "1000"); // 1s, default is 60000 producer.beginTransaction(); producer.send(new ProducerRecord<>("payments", "k", "v")); Thread.sleep(5000); // "processing" outlives the transaction producer.commitTransaction(); // boom ``` No second producer exists, yet the producer is dead. On brokers/clients ≥ 2.7 this path typically surfaces as `InvalidProducerEpochException` (KIP-588 separated the two cases); on older combinations you get the misleading `ProducerFencedException` message verbatim. Either way, the epoch was bumped underneath you. ## 3\. Why It Happens — Surface Level A `transactional.id` identifies **one logical producer**. The transaction coordinator enforces this with an epoch number. Whenever a producer calls `initTransactions()` with a given `transactional.id`, the coordinator bumps the epoch and invalidates every producer holding an older epoch for that id. The old producer's next transactional request is rejected — it has been *fenced*. You hit it for one of three reasons: two live producers share a `transactional.id` (deploy overlap, replicas > 1, copy-pasted config); your transaction stayed open longer than `transaction.timeout.ms` and the coordinator aborted it and bumped the epoch proactively; or, in Streams/consume-process-produce apps, a rebalance handed your partitions (and their transactional identity) to another instance while you were still processing. ## 4\. Why It Happens — Under the Hood Fencing is not an error condition in the protocol's eyes — it is the mechanism that makes exactly-once possible. The whole point of `transactional.id` is **zombie fencing**: guaranteeing that a stalled, GC-paused, or network-partitioned old instance can't wake up and commit a duplicate transaction after its replacement took over. The mechanics: on `initTransactions()`, the producer sends `InitProducerId` to the transaction coordinator (the broker leading the `__transaction_state` partition that `hash(transactional.id)` maps to). The coordinator looks up the existing producer ID (PID) and epoch for that `transactional.id`, aborts any transaction still open under the old epoch, bumps the epoch, persists the new `(PID, epoch)` to `__transaction_state`, and returns it. From then on, every `Produce`, `AddPartitionsToTxn`, `TxnOffsetCommit`, and `EndTxn` request carries the epoch. Any request arriving with a stale epoch gets `PRODUCER_FENCED` back, which the client rethrows as `ProducerFencedException` and transitions the producer to a fatal state. The timeout path is the same machinery driven by the coordinator's own timer: if an open transaction exceeds `transaction.timeout.ms` (client-set, default 60 s, capped by the broker's `transaction.max.timeout.ms`, default 15 min), the coordinator writes an abort marker and bumps the epoch so a possibly-hung producer can't commit late. Pre-2.7 the client couldn't distinguish this from real fencing — hence the "newer producer" message with no newer producer. KIP-588 (Kafka 2.7) split the error codes: genuine fencing → `ProducerFencedException` (fatal), timeout-driven epoch bump → `InvalidProducerEpochException` (abortable on produce; the failed transaction is lost, the producer isn't). The rebalance case is where the history matters. Under EOS v1 (pre-KIP-447), a consume-process-produce app needed **one transactional producer per input partition** (`transactional.id` \= something like `app---`), because fencing was the only defense against a zombie committing offsets for a partition it no longer owned. KIP-447 (Kafka 2.5) moved that defense to the group coordinator: `sendOffsetsToTransaction(offsets, groupMetadata)` carries the consumer group generation, and a commit from a previous generation is rejected. That's EOS v2 — `processing.guarantee=exactly_once_v2` in Streams, `EOSMode.V2` in Spring Kafka (the default since Spring Kafka 3.0) — and it allows pooled producers with far fewer `transactional.id`s. But it does not repeal the rule that a given `transactional.id` belongs to one live producer at a time. Duplicate ids still fence, v1 or v2. ## 5\. The Fix Match the fix to the trigger. **Fix 1 — duplicate `transactional.id` across instances.** Make the id unique per producer instance: ```diff - p.put(ProducerConfig.TRANSACTIONAL_ID_CONFIG, "payments-tx"); + // unique per instance: pod name, HOSTNAME, or an instance ordinal + p.put(ProducerConfig.TRANSACTIONAL_ID_CONFIG, + "payments-tx-" + System.getenv("HOSTNAME")); ``` For producer-only apps (no consume-process-produce loop) a random suffix is fine — fencing across restarts buys you nothing there, because a new instance isn't continuing the old one's read position. In Spring Kafka: ```diff spring: kafka: producer: - transaction-id-prefix: payments-tx- + transaction-id-prefix: payments-tx-${HOSTNAME}- ``` Note: for **listener-container-initiated** transactions with `EOSMode.V2` (Spring Kafka 3.x + brokers ≥ 2.5), the prefix no longer has to be unique per instance — fencing rides on the consumer group metadata. For **producer-only** transactions, keep the prefix unique per instance. **Fix 2 — transaction outliving `transaction.timeout.ms`.** Either shrink the work inside the transaction or raise the timeout: ```diff - // default transaction.timeout.ms = 60000, processing takes 3-4 min + p.put(ProducerConfig.TRANSACTION_TIMEOUT_CONFIG, "300000"); // 5 min ``` Check the broker cap first — the client value must be ≤ `transaction.max.timeout.ms` (default 900000) or producer initialization fails with `InvalidTxnTimeoutException`. Prefer smaller transactions over bigger timeouts: a long-open transaction holds back the Last Stable Offset, so `read_committed` consumers downstream stall for the duration. If you're batch-processing 10k records per transaction and hitting the timeout, chunk it. **Fix 3 — handle it correctly when it does happen.** `ProducerFencedException` means this producer instance is done. Don't catch-and-retry, don't call `abortTransaction()` (it throws the same exception): ```java try { producer.beginTransaction(); // send, sendOffsetsToTransaction ... producer.commitTransaction(); } catch (ProducerFencedException | FencedInstanceIdException e) { producer.close(); // only valid call left // recreate the producer, or let the instance exit and restart } catch (KafkaException e) { producer.abortTransaction(); // abortable errors: safe to abort and retry } ``` That two-tier catch — fatal errors close, abortable errors abort — is the canonical transactional loop from the `KafkaProducer` javadoc, and most homegrown transaction code gets it wrong by treating everything as retryable. ## 6\. Best Practices & The Better Design If you're doing consume-process-produce, stop managing raw producers and epochs yourself. Use EOS v2 through Spring Kafka or Streams and let the framework own the lifecycle: ```java @Bean KafkaTemplate kafkaTemplate(ProducerFactory pf) { return new KafkaTemplate<>(pf); } @Bean ConcurrentKafkaListenerContainerFactory factory( ConsumerFactory cf, KafkaTransactionManager tm) { var factory = new ConcurrentKafkaListenerContainerFactory(); factory.setConsumerFactory(cf); factory.getContainerProperties().setEosMode(ContainerProperties.EOSMode.V2); factory.getContainerProperties().setKafkaAwareTransactionManager(tm); return factory; } @KafkaListener(topics = "payments", groupId = "payments-processor") void process(ConsumerRecord rec) { // runs inside a transaction; offsets committed via sendOffsetsToTransaction kafkaTemplate.send("payments-out", transform(rec)); } ``` ```properties spring.kafka.producer.transaction-id-prefix=payments-processor- spring.kafka.consumer.isolation-level=read_committed spring.kafka.consumer.properties.max.poll.interval.ms=300000 ``` With V2, the group coordinator fences zombies by generation, producers are pooled, and a rolling deploy doesn't manufacture fencing storms the way per-partition transactional ids under V1 did. Keep `transaction.timeout.ms` comfortably above your worst-case per-poll processing time, and keep `max.poll.interval.ms` above it too — if the consumer gets kicked from the group mid-transaction, the ensuing rebalance plus commit attempt is exactly how you meet `InvalidProducerEpochException` in production. Two design rules that eliminate most fencing incidents: **one `transactional.id` namespace per app, suffixed by stable instance identity** (StatefulSet ordinal or pod name — not a random UUID if you rely on fencing for correctness across restarts), and **transactions no larger than a few hundred milliseconds of work**. If a "transaction" wraps a REST call to a third party with a 30 s timeout, that's an outbox pattern problem, not a Kafka tuning problem. ## 7\. How to Prevent It Long-Term Alert on the transaction coordinator and client metrics that precede fencing: the producer's `txn-commit-time-ns-avg` creeping toward `transaction.timeout.ms`, consumer lag growth on `read_committed` consumers (a symptom of a stuck open transaction holding the LSO), and application-log rates of `ProducerFencedException` / `InvalidProducerEpochException` — a nonzero steady rate almost always means overlapping deploys or duplicated ids, not transient noise. Make identity a deploy-time invariant: derive `transactional.id` / `transaction-id-prefix` from the pod name in the manifest, and add a CI check that greps for hardcoded transactional ids in config. For rolling updates of V1-style apps, use `maxSurge: 0` so the old pod is gone before the new one calls `initTransactions()` — or better, migrate to V2 and make the overlap harmless. Load-test the timeout budget: run your consume-process-produce loop at p99 payload sizes and verify worst-case transaction duration stays under 50% of `transaction.timeout.ms`. And chaos-test rebalances — kill a pod mid-transaction and confirm the survivors recover without duplicate output; that test catches wrong exception handling (retrying a fenced producer) before production does. ## 8\. Key Takeaways - `ProducerFencedException` is the zombie-fencing mechanism working as designed — the question is who bumped your epoch: a duplicate `transactional.id`, the transaction timeout, or a rebalance. - The "newer producer" message lies on older versions: a transaction exceeding `transaction.timeout.ms` (default 60 s) produces the same fatal error with no second producer in sight. Kafka ≥ 2.7 reports that case as `InvalidProducerEpochException`. - One `transactional.id` \= one live producer. Suffix it with stable instance identity; never share it across replicas or overlapping deploys. - The exception is fatal: `close()` and recreate. Only abortable `KafkaException`s get `abortTransaction()` \+ retry. - Use EOS v2 (`EOSMode.V2` / `exactly_once_v2`, brokers ≥ 2.5) so zombie fencing rides on consumer group generations instead of an explosion of per-partition transactional ids.