> ## Content Index > Fetch the complete content index at: https://www.ggorantala.dev/llms.txt > Use this file to discover other available public pages before exploring further. # Kafka SerializationException: Can't Convert Value of Class Fix - URL: https://www.ggorantala.dev/kafka-cant-convert-value-of-class-fix/ - Published: 2026-02-24T14:11:00.000Z - Updated: 2026-08-29T08:19:31.000Z - Description: Kafka SerializationException: Can't convert value of class X to StringSerializer? The ClassCastException behind it, the config fix, and the safer design. - Author: Gopi Gorantala - Tags: kafka-errors, serializationexception, kafka-producer, spring-kafka, classcastexception, jsonserializer, Java ## 1\. The Error Your producer dies on the first `send()`: ```text Exception in thread "main" org.apache.kafka.common.errors.SerializationException: Can't convert value of class com.example.orders.OrderEvent to class org.apache.kafka.common.serialization.StringSerializer specified in value.serializer at org.apache.kafka.clients.producer.KafkaProducer.doSend(KafkaProducer.java:1090) at org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:1017) at org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:962) at com.example.orders.OrderProducer.main(OrderProducer.java:24) Caused by: java.lang.ClassCastException: class com.example.orders.OrderEvent cannot be cast to class java.lang.String (com.example.orders.OrderEvent is in unnamed module of loader 'app'; java.lang.String is in module java.base of loader 'bootstrap') at org.apache.kafka.common.serialization.StringSerializer.serialize(StringSerializer.java:29) at org.apache.kafka.clients.producer.KafkaProducer.doSend(KafkaProducer.java:1087) ... 3 more ``` The key-side twin is identical except for the first line: `Can't convert key of class ... specified in key.serializer`. Where you'll hit it: - **kafka-clients**: any version — the message text has been stable for years (verified against 3.x/4.x) - **Spring Boot / Spring Kafka**: Boot 3.x with spring-kafka 3.x, where `KafkaTemplate.send()` throws the same exception because Boot's autoconfigured default serializer is `StringSerializer` - **Setup**: almost always a POC or a brand-new service — this error means the producer has never successfully sent this record type, ever. It is a wiring bug, not an infrastructure incident. Note what it is **not**: it is not the consumer-side poison-pill problem (`RecordDeserializationException` inside `poll()`). This one fires in the producer, synchronously, before a single byte reaches the broker. ## 2\. How to Reproduce It (step-by-step) Single-broker KRaft cluster: ```yaml # docker-compose.yml services: kafka: image: apache/kafka:3.9.1 ports: - "9092:9092" environment: KAFKA_NODE_ID: 1 KAFKA_PROCESS_ROLES: broker,controller KAFKA_CONTROLLER_QUORUM_VOTERS: 1@kafka:29093 KAFKA_LISTENERS: INTERNAL://kafka:29092,CONTROLLER://kafka:29093,EXTERNAL://0.0.0.0:9092 KAFKA_ADVERTISED_LISTENERS: INTERNAL://kafka:29092,EXTERNAL://localhost:9092 KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: INTERNAL:PLAINTEXT,CONTROLLER:PLAINTEXT,EXTERNAL:PLAINTEXT KAFKA_CONTROLLER_LISTENER_NAMES: CONTROLLER KAFKA_INTER_BROKER_LISTENER_NAME: INTERNAL KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1 ``` ```bash docker compose up -d ``` Producer with the classic mistake — a POJO value, but the serializer config copy-pasted from a hello-world string example: ```java // build.gradle: implementation 'org.apache.kafka:kafka-clients:3.9.1' package com.example.orders; import org.apache.kafka.clients.producer.*; import org.apache.kafka.common.serialization.StringSerializer; import java.util.Properties; public class OrderProducer { record OrderEvent(String orderId, long amountCents) {} public static void main(String[] args) { Properties props = new Properties(); props.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092"); props.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class.getName()); props.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, StringSerializer.class.getName()); // Compiles without a single warning: KafkaProducer producer = new KafkaProducer<>(props); producer.send(new ProducerRecord<>("orders", "o-42", new OrderEvent("o-42", 19_99))); // 💥 throws here, synchronously producer.close(); } } ``` Run it. The `send()` call itself throws — no callback fires, no retry happens, nothing hits the broker. `kafka-console-consumer.sh --topic orders --from-beginning` shows an empty topic. Spring Boot reproduction is even easier because it's the **default**: ```java @Autowired KafkaTemplate kafkaTemplate; // Boot default: StringSerializer for both kafkaTemplate.send("orders", new OrderEvent("o-42", 1999)); // same SerializationException ``` No environment dependency: no Docker networking, no broker config, no version matrix. The record type and the serializer class disagree; that's the whole trigger. ## 3\. Why It Happens — Surface Level `value.serializer` tells the producer which class turns your value object into bytes. You configured `StringSerializer`, whose `serialize` method accepts a `String`. You handed it an `OrderEvent`. Java can't cast `OrderEvent` to `String`, throws `ClassCastException`, and `KafkaProducer.doSend()` wraps it into the `SerializationException` you're staring at — including, helpfully, both class names so you can see the exact mismatch. The fix is to make the two agree: either configure a serializer that understands your type (JSON/Avro/Protobuf), or convert the value to a `String` yourself before `send()`. Nothing about brokers, topics, or networking is involved. ## 4\. Why It Happens — Under the Hood The interesting question is why the compiler let you do this. Three mechanics stack up: **1\. Config-string wiring erases the type relationship.** When serializers are passed as class names in the config map, `KafkaProducer` instantiates them reflectively (`Utils.newInstance()` via `config.getConfiguredInstance()`). At that point the producer holds a `Serializer` and `Serializer` whose actual type parameters are unknown to the compiler — an unchecked assignment happens inside the client. `KafkaProducer` with a `StringSerializer` value serializer is nonsense, but it's runtime-only nonsense. **2\. Generics are erased; the cast happens inside the serializer.** `StringSerializer` implements `Serializer`. After erasure, the bridge method accepts `Object` and casts to `String` before calling the real `serialize(String topic, String data)`. Your `OrderEvent` sails through every compile-time check and detonates on that cast — the first line of the serializer. This is why the stack trace bottoms out at `StringSerializer.serialize`. **3\. The exception is synchronous and pre-accumulator.** In `doSend()`, key and value serialization happen **before** partitioning and before the record is appended to the `RecordAccumulator`. The catch block is explicit: ```java } catch (ClassCastException cce) { throw new SerializationException("Can't convert value of class " + record.value().getClass().getName() + " to class " + valueSerializer.getClass().getName() + " specified in value.serializer", cce); } ``` Consequences worth internalizing: the producer's retry machinery (`retries`, `delivery.timeout.ms`, idempotence) never engages — those govern the network path, and this record never got that far. The `Callback` is never invoked either; the exception surfaces on the calling thread. If your code only handles errors in the callback (the common pattern from the batch-expiry world), a fire-and-forget wrapper like `void publish(Object e) { producer.send(rec(e)); }` will still crash — but a wrapper that swallows exceptions around `send()` will silently drop every single record. In Spring, `KafkaTemplate.send()` throws before returning its `CompletableFuture`, so `.whenComplete()` handlers never see it. Why Spring Boot makes this the default trap: Boot's `KafkaAutoConfiguration` defaults both `spring.kafka.producer.key-serializer` and `value-serializer` to `StringSerializer`. Every tutorial that sends strings works out of the box; the first time you send a domain object, you get this exception. The consumer-side mirror (`JsonDeserializer` \+ trusted packages) fails differently — as a poison pill in `poll()` — which is a separate article in this series. ## 5\. The Fix ### Fix A — configure a JSON serializer (POC / internal topics) Raw client, using spring-kafka's `JsonSerializer` (or roll your own Jackson wrapper — it's \~15 lines): ```diff props.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class.getName()); -props.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, StringSerializer.class.getName()); +props.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, "org.springframework.kafka.support.serializer.JsonSerializer"); ``` Spring Boot `application.yml`: ```diff spring: kafka: bootstrap-servers: localhost:9092 producer: key-serializer: org.apache.kafka.common.serialization.StringSerializer + value-serializer: org.springframework.kafka.support.serializer.JsonSerializer ``` Minimal hand-rolled serializer if you don't want the spring-kafka dependency in a plain Java service: ```java public class JacksonSerializer implements Serializer { private static final ObjectMapper MAPPER = new ObjectMapper(); @Override public byte[] serialize(String topic, T data) { if (data == null) return null; try { return MAPPER.writeValueAsBytes(data); } catch (JsonProcessingException e) { throw new SerializationException("JSON serialization failed for topic " + topic, e); } } } ``` ### Fix B — serialize explicitly at the call site (quick unblock) ```diff -producer.send(new ProducerRecord<>("orders", "o-42", event)); +producer.send(new ProducerRecord<>("orders", "o-42", mapper.writeValueAsString(event))); ``` Fine for a spike; in a real service you want the serializer owned by producer config, not scattered across call sites. ### Fix C — constructor-injected serializers (do this everywhere) This is the fix for the *class* of bug, not just this instance. `KafkaProducer` has a constructor that takes serializer **instances**, which restores compile-time type checking: ```diff -props.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class.getName()); -props.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, StringSerializer.class.getName()); -KafkaProducer producer = new KafkaProducer<>(props); +KafkaProducer producer = new KafkaProducer<>( + props, + new StringSerializer(), + new JacksonSerializer()); ``` Now `producer.send(new ProducerRecord<>("orders", "o-42", "some string"))` is a **compile error**. The config-string style can never give you that. When to use which: Fix B to unblock a spike today; Fix A when the topic is internal to one team and JSON is acceptable; Fix C as a standing convention regardless of format. For anything crossing team boundaries or living longer than a quarter, go to schema-backed serialization (next section) — JSON-without-schema is how you end up writing the Schema Registry 409 article's incident report later. ## 6\. Best Practices & The Better Design **Precise generics, one producer per value type.** `KafkaProducer` (and `KafkaTemplate`) is the root enabler of this bug — it tells the compiler "don't check anything." Declare `KafkaTemplate` per event family. In Spring, build them explicitly instead of leaning on the autoconfigured template: ```java @Configuration public class KafkaProducerConfig { @Bean public ProducerFactory orderProducerFactory(KafkaProperties kafkaProperties) { Map props = kafkaProperties.buildProducerProperties(null); // Serializer *instances* — type-checked, and JsonSerializer gets your ObjectMapper: return new DefaultKafkaProducerFactory<>( props, new StringSerializer(), new JsonSerializer<>(new ObjectMapper().registerModule(new JavaTimeModule()))); } @Bean public KafkaTemplate orderKafkaTemplate( ProducerFactory orderProducerFactory) { return new KafkaTemplate<>(orderProducerFactory); } } ``` Injecting `KafkaTemplate` now fails at compile time if someone sends the wrong type, and fails at *context startup* if the bean wiring is wrong — both strictly better than failing on the first production `send()`. **Schema-backed serialization for anything durable.** JSON with no contract just moves the type mismatch from your producer to every consumer. The production-grade design is Avro or Protobuf with a schema registry: ```properties value.serializer=io.confluent.kafka.serializers.KafkaAvroSerializer schema.registry.url=http://schema-registry:8081 auto.register.schemas=false use.latest.version=true ``` With generated `SpecificRecord` classes, the type agreement between your code and the wire format is enforced by codegen, and evolution is governed by compatibility rules instead of hope. **One convention to ban the bug:** serializers are always passed as instances (Fix C style), never as config strings, except where an ecosystem forces strings (Connect, Streams `default.value.serde`). Put it in the team's Kafka config module and stop thinking about it. ## 7\. How to Prevent It Long-Term **Testcontainers smoke test that actually sends your real event type.** Unit tests with `MockProducer` won't catch this — `MockProducer` happily accepts any object unless you construct it with real serializers (so do that). The reliable gate is one integration test per producer: ```java @Test void publishesOrderEvent() throws Exception { kafkaTemplate.send("orders", "o-1", new OrderEvent("o-1", 500)).get(10, TimeUnit.SECONDS); // consume back with the matching deserializer and assert round-trip equality } ``` This single test catches serializer mismatch, missing trusted packages on the consumer side, and schema incompatibilities — the entire wiring class — before any deploy. **Static checks.** Forbid `KafkaProducer`, `KafkaTemplate`, and raw `KafkaTemplate` via ArchUnit or Error Prone in CI. Grep-level enforcement is enough: the offending declarations are syntactically obvious. **Fail at startup, not at first send.** A tiny `SmartLifecycle` bean that serializes a canary instance of each event type through the configured serializer at boot turns "first order of the day fails" into "deploy fails health check" — vastly cheaper. **Log-based detection.** This exception is synchronous, so it shows up in *your* service logs, not broker logs and not producer metrics (`record-error-rate` never increments — the record was never appended). Alert on `SerializationException` in application logs; don't expect Kafka-side monitoring to see it. ## 8\. Key Takeaways - `Can't convert value of class X to class Y specified in value.serializer` \= your record type and configured serializer disagree. It's a wiring bug; the broker is never involved. - The exception is **synchronous** — thrown from `send()` itself. Callbacks never fire, retries never engage, `record-error-rate` stays at zero. Handle it on the calling thread. - Spring Boot defaults both producer serializers to `StringSerializer`; the first domain object you send reproduces this exactly. Set `spring.kafka.producer.value-serializer` explicitly. - Pass serializer **instances** to the `KafkaProducer`/`DefaultKafkaProducerFactory` constructor instead of class-name strings — that turns this whole bug class into a compile error. - One Testcontainers round-trip test per producer with the real event type catches serializer mismatch, deserializer mismatch, and schema drift before production ever sees them.