RecordTooLargeException: Fixing Kafka's Message Size Limits

What shows up in the logs

You send a record that's a bit too big and get this back, either straight out of producer.send() or wrapped in an ExecutionException from the future:

org.apache.kafka.common.errors.RecordTooLargeException: The message is 1500028 bytes when serialized which is larger than 1048576, which is the value of the max.request.size configuration.

Or, if you already "fixed" it once, this one instead:

org.apache.kafka.common.errors.RecordTooLargeException: The record batch size in the append to size-repro-0 is 3000038 bytes which exceeds the maximum configured value of 1048588).

Same exception class, same-sounding message, and I'd bet most people who hit the second one assume they're looking at the first one again with a bigger number. They're not. One of these never leaves your JVM. The other one is your broker, after a real network round trip, telling you no. Which one you're looking at changes what you fix, and fixing the wrong one is how you end up raising max.request.size twice and still failing.

I went and read the throw sites for both of these in the 4.3.1 source rather than trust what six-year-old Stack Overflow answers say about them, because the two messages above come from genuinely different code paths with a details gap between them that nobody seems to write down.

Local check or broker rejection?

The fast way to tell them apart: how did the exception arrive?

If it came out of producer.send() itself, synchronously, before the call even returns a Future, that's the client. KafkaProducer checks record size in doSend(), before the record ever touches the accumulator or a socket. No broker was involved. No bytes went anywhere.

If it came out of future.get() as an ExecutionException, or through your Callback, after send() returned normally, that's the broker. The request made it onto the wire, got parsed, and was rejected during log append.

The broker side has a second, quieter tell: it doesn't log anything. ReplicaManager.appendToLocalLog catches RecordTooLargeException in the same branch as UnknownTopicOrPartitionException and CorruptRecordException, with a comment saying flat out that these are expected, known outcomes and shouldn't count as broker failures. No WARN, no ERROR, nothing in server.log. Go looking there for a smoking gun and you'll find silence. That silence is itself the confirmation you want. What does move is the per-topic BytesRejectedPerSec JMX metric (kafka.server:type=BrokerTopicMetrics,name=BytesRejectedPerSec,topic=<t>), which ticks up on every rejection whether or not anyone's watching the logs.

Reproduce it on your laptop

a. What you need. apache/kafka:4.3.1 (Docker), JDK 17, kafka-clients:4.3.1 on the classpath. One broker is enough. This never touches replication.

b. The broker. The stock single-node KRaft compose from the Kafka repo, unmodified. That's the point: you don't need to misconfigure anything to hit this.

# docker/examples/docker-compose-files/single-node/plaintext, unmodified
services:
  broker:
    image: apache/kafka:4.3.1
    container_name: broker
    ports:
      - "9092:9092"
    environment:
      KAFKA_NODE_ID: 1
      KAFKA_PROCESS_ROLES: broker,controller
      KAFKA_LISTENERS: PLAINTEXT://broker:19092,CONTROLLER://broker:19093,PLAINTEXT_HOST://0.0.0.0:9092
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://broker:19092,PLAINTEXT_HOST://localhost:9092
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
      KAFKA_CONTROLLER_QUORUM_VOTERS: 1@broker:19093
      KAFKA_CONTROLLER_LISTENER_NAMES: CONTROLLER
      KAFKA_LOG_DIRS: /tmp/kraft-combined-logs
      CLUSTER_ID: 4L6g3nShT-eMCtK--X86sw

c. The topic. Left at every default. message.max.bytes resolves to 1024*1024 + 12 = 1048588 bytes (Records.LOG_OVERHEAD is 12), and that's the ceiling neither of our sends will clear:

docker compose up -d
docker exec broker /opt/kafka/bin/kafka-topics.sh --create \
  --topic size-repro --bootstrap-server localhost:9092 --partitions 1

d. The reproducer.

<!-- pom.xml -->
<dependency>
    <groupId>org.apache.kafka</groupId>
    <artifactId>kafka-clients</artifactId>
    <version>4.3.1</version>
</dependency>
<dependency>
    <groupId>org.slf4j</groupId>
    <artifactId>slf4j-simple</artifactId>
    <version>1.7.36</version>
</dependency>

package com.example.repro;

import org.apache.kafka.clients.producer.*;
import org.apache.kafka.common.errors.RecordTooLargeException;
import org.apache.kafka.common.serialization.ByteArraySerializer;
import org.apache.kafka.common.serialization.StringSerializer;

import java.util.Properties;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.Future;

public class RecordSizeRepro {

    public static void main(String[] args) throws Exception {
        String scenario = args.length > 0 ? args[0] : "client";
        if ("client".equals(scenario)) runClientSideRejection();
        else runBrokerSideRejection();
    }

    // Step 1: defaults everywhere, 1.5 MB value. Rejected before any network call.
    private static void runClientSideRejection() {
        Properties props = baseProps();
        try (KafkaProducer<String, byte[]> producer = new KafkaProducer<>(props)) {
            byte[] payload = new byte[1_500_000];
            System.out.println("Calling send(), default max.request.size is 1048576...");
            producer.send(new ProducerRecord<>("size-repro", "k1", payload));
            System.out.println("unreachable");
        } catch (RecordTooLargeException e) {
            System.out.println("Caught on the calling thread, no request was sent:");
            System.out.println(e.getMessage());
        }
    }

    // Step 2: raise the producer's own ceiling, leave the topic at its default.
    private static void runBrokerSideRejection() throws Exception {
        Properties props = baseProps();
        props.put(ProducerConfig.MAX_REQUEST_SIZE_CONFIG, 5_242_880);
        props.put(ProducerConfig.BUFFER_MEMORY_CONFIG, 67_108_864L);
        try (KafkaProducer<String, byte[]> producer = new KafkaProducer<>(props)) {
            byte[] payload = new byte[3_000_000];
            System.out.println("Calling send(), max.request.size raised to 5 MB, topic still at ~1 MB...");
            Future<RecordMetadata> f = producer.send(new ProducerRecord<>("size-repro", "k2", payload));
            System.out.println("send() returned normally. Client-side check passed. Waiting on the broker...");
            f.get();
            System.out.println("unreachable");
        } catch (ExecutionException e) {
            System.out.println("Caught from future.get(), after a real produce request:");
            System.out.println(e.getCause());
        }
    }

    private static Properties baseProps() {
        Properties p = new Properties();
        p.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092");
        p.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class.getName());
        p.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, ByteArraySerializer.class.getName());
        p.put(ProducerConfig.MAX_BLOCK_MS_CONFIG, 10_000);
        return p;
    }
}

Step 1: run scenario 1
$ java -cp target/repro-1.0.jar com.example.repro.RecordSizeRepro client
Expected: prints the RecordTooLargeException referencing max.request.size, in under a second, no broker traffic.

Step 2: run scenario 2
$ java -cp target/repro-1.0.jar com.example.repro.RecordSizeRepro broker
Expected: "send() returned normally" prints first, then a couple hundred ms later the RecordTooLargeException referencing 1048588 comes back from future.get().

e. Confirmation signal. Scenario 1's exception text names max.request.size. Scenario 2's names a byte count ending ) with no matching (. That stray parenthesis is a genuine typo that's been sitting in this message since at least Kafka 2.8.2. If you see RecordBatchTooLargeException instead, you've hit a different, unrelated ceiling (segment.bytes, not this one); that only happens if someone shrank the segment size far below the message size limit for testing and forgot to put it back.

f. Teardown. docker compose down -v. The named volume holds the KRaft metadata log, and starting fresh against stale meta.properties produces an unrelated and much more confusing failure on next boot.

The short answer

Scenario 1: your record is bigger than max.request.size, and the Java producer catches that itself before wasting a network round trip. Scenario 2: your record cleared the producer's own limit but not the topic's, so the broker parsed the whole request and rejected it during log append. Both throw the same exception class because, structurally, they're both "you tried to write something bigger than the configured ceiling." The ceiling lives in a different place, that's all.

Two throw sites, one exception name

Here's the part that isn't in the client-side story at all: the broker doesn't have one check, it has two, and they're not equally likely to fire.

UnifiedLog.append() calls analyzeAndValidateRecords() first, and that method checks every batch's size against config().maxMessageSize() before anything else happens: no compression, no offset assignment yet. This is the check that produced scenario 2's message above, and it's the one you'll hit nearly every time, because it runs on the request exactly as the producer sent it.

There's a second check, later in the same append() call, guarded by messageSizeMaybeChanged(). That flag only comes back true if the broker had to rebuild the batch: down-converting it for an ancient client, or recompressing it because the topic's compression.type differs from what the producer sent. If the batch grows past the limit only after that rebuild, you get a second, differently-worded exception: "Message batch size is N bytes in append topartition ...". No, that missing space isn't a typo I made in this article; the source doesn't have one either. UnifiedLog.java really does concatenate "in append to" and "partition " with nothing between them. It's a cosmetic bug that's been there since the Scala Log.scala days, and I'd guess nobody's filed a Jira for it, because it's never the thing that actually costs anyone the debugging time. I only found it by diffing the exact string across five release tags, looking for wording drift. There wasn't any. That's worth knowing on its own: every Stack Overflow answer quoting either message is still accurate today, whichever year it was written in.

The reason this matters: if you're staring at the first message and reach for "the broker recompressed my batch," you're chasing a red herring. Compression-triggered rejection is real, but it's rare. Normal produce traffic almost never touches that second path. I'd have bet money the two were the same check sharing one string constant, before I actually opened the file.

Neither exception, by the way, is retriable. MESSAGE_TOO_LARGE (error code 10) is a permanent rejection on the wire, so retries and delivery.timeout.ms do nothing for either scenario. There's no transient condition to wait out.

The fix

Raising max.request.size alone doesn't fix scenario 2, it causes it. You're moving the rejection point from your own JVM to the broker, which means you now pay for a full network round trip before finding out the record still doesn't fit anywhere. All three levels have to agree, and they're three different config keys at three different levels:

# Producer (client-side ceiling, checked before send)
- max.request.size=1048576          # default
+ max.request.size=5242880          # 5 MB

# Producer (must also cover the bigger request)
- buffer.memory=33554432            # default, 32 MB
+ buffer.memory=67108864            # 64 MB, headroom above max.request.size

# Topic (server-side ceiling, checked on append; set this, not the broker default)
$ kafka-configs.sh --bootstrap-server localhost:9092 \
    --entity-type topics --entity-name size-repro --alter \
    --add-config max.message.bytes=5242880

Prefer the topic-level max.message.bytes over the broker-level message.max.bytes cluster default, unless you genuinely want every topic on the cluster to accept bigger messages. The cluster-wide knob affects replication fetch sizes and page cache pressure for topics that never needed the headroom.

On non-JVM clients built on librdkafka, the story's different in a way that matters. Its own message.max.bytes producer config is documented as advisory: "the producer is unable to reliably enforce a strict max message limit at produce time." A librdkafka producer can't guarantee the fast, synchronous, no-network rejection Java gives you in scenario 1. The real enforcement is still the broker's, and it surfaces asynchronously through the delivery report as a local MSG_SIZE_TOO_LARGE error rather than an exception thrown from the send call.

Named non-fixes, because all three show up in answers to this exact error. Raising retries or request.timeout.ms does nothing, since neither config is even consulted for a non-retriable error. Catching the exception and silently dropping the record buys you permanent, invisible data loss, tracked only by a metric (BytesRejectedPerSec) that nobody's alerting on. And raising max.partition.fetch.bytes on the consumer side is a leftover from before KIP-74 landed in 0.10.1. Since then, the broker always returns at least one full record batch to guarantee progress even if it's over the limit, so this config can't produce a RecordTooLargeException on the consumer at all, and hasn't been able to for years. The Javadoc on AsyncKafkaConsumer.poll() still lists it as a checked exception; I think that's a stale @throws tag nobody's gotten around to deleting.

In Spring Boot, there's no dedicated spring.kafka.producer.max-request-size. KafkaProperties.Producer has first-class fields for batch-size and buffer-memory but not this one. You set it through the generic passthrough: spring.kafka.producer.properties.max.request.size=5242880.

Size limits, chosen on purpose

The cleanest fix is usually not raising the limit at all. If you're regularly producing multi-megabyte records, ask why before you ask how big. Compression (compression.type=lz4 or zstd on the producer) often gets a 3 MB JSON payload under the default 1 MB ceiling without touching any size config, and it's free bandwidth savings on every smaller record too. For records that are genuinely large by nature (images, file attachments, big blobs), the claim-check pattern, putting the blob in object storage and producing a small message that carries its reference, keeps Kafka doing what it's fast at instead of turning it into a file transfer protocol it was never built to be.

If you do need the ceiling raised, raise it deliberately and narrowly: on the one topic that needs it, not the broker default, and document the number next to the producer config so the next person doesn't have to reverse-engineer why max.request.size is set to a strange value.

What to alert on

Alert on the per-topic BytesRejectedPerSec JMX metric (kafka.server:type=BrokerTopicMetrics,name=BytesRejectedPerSec,topic=<t>) being non-zero over any window. Since server.log stays silent, this metric is the only broker-side signal you get for free, and any non-zero reading means something is actively losing data right now. On the producer, record-size-max (kafka.producer:type=producer-metrics,client-id=<id>) tells you the largest record you've actually sent recently, so you can watch it trend toward your configured ceiling before it gets there, not after. The client-side rejection in scenario 1 doesn't move record-error-rate at all, because the record never reached the accumulator where that metric gets recorded. The exception itself is your only signal for that path, which is a good argument for never calling producer.send() without either a callback or checking the returned future.

On versions: both throw sites and both message strings are unchanged from 2.8.2 through 4.3.1. I diffed the source across five tags spanning that range and found byte-for-byte identical wording, missing space and stray parenthesis included, surviving the Scala-to-Java rewrite of the storage layer along the way. There's no "safe version" to upgrade to here and no version where this behaves differently. The one thing that has moved over that span is the consumer side. If you're running anything from before 0.10.1, the consumer-side variant of this exception was real and max.partition.fetch.bytes was the actual fix. On anything you're likely running today, it isn't.

Five things to keep in your head

• RecordTooLargeException from producer.send() directly means the client rejected it, no network trip happened, and the number in the message is max.request.size or buffer.memory.
• The same exception from a Future or callback means the broker rejected it after a real request, and the number is the topic's max.message.bytes, which defaults to roughly 1 MB plus 12 bytes of overhead.
• The broker logs nothing for either case. The confirmation is the BytesRejectedPerSec metric, not server.log.
• Fix all three levels together (producer max.request.size, buffer.memory, topic max.message.bytes) or you've only relocated the failure, not fixed it.
• Neither error is retriable. MESSAGE_TOO_LARGE is permanent, so retries and delivery.timeout.ms don't apply here.