Kafka SSL Handshake Failed: PKIX and Hostname Fixes

Client side, the moment your producer or consumer touches a TLS listener:

org.apache.kafka.common.errors.SslAuthenticationException: SSL handshake failed
Caused by: javax.net.ssl.SSLHandshakeException: PKIX path building failed:
    sun.security.provider.certpath.SunCertPathBuilderException:
    unable to find valid certification path to requested target
	at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:130)
	at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:378)
	at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.checkServerCerts(CertificateMessage.java:1300)

Broker side, the matching log line (this is all you get — the broker deliberately logs handshake failures at INFO with no detail):

INFO [SocketServer listenerType=BROKER, nodeId=1] Failed authentication with /172.18.0.7
    (channelId=172.18.0.4:9093-172.18.0.7:41562-0) (SSL handshake failed)
    (org.apache.kafka.common.network.Selector)

Two common variants of the client-side cause:

Caused by: java.security.cert.CertificateException: No subject alternative names
    matching IP address 10.20.30.40 found

javax.net.ssl.SSLException: Unsupported or unrecognized SSL message

Typical setup where this bites: kafka-clients or Spring Kafka 3.x connecting to a TLS-enabled listener — self-managed KRaft cluster (Kafka 3.7/3.8, apache/kafka:3.8.0 image), MSK with TLS auth, or Confluent Cloud — from a service whose JVM doesn't trust the broker's certificate chain. SslAuthenticationException is fatal and non-retriable: the client will not retry, and a Spring Kafka listener container will log it and (by default) keep attempting to re-poll, producing a log-spamming loop that looks like a retry but never succeeds.

2. How to Reproduce It (step-by-step)

Generate a self-signed CA, a broker cert, and a truststore — then deliberately don't give the client the truststore.

# 1. CA
openssl req -new -x509 -keyout ca.key -out ca.crt -days 365 -nodes \
  -subj "/CN=demo-kafka-ca"

# 2. Broker keystore + CSR, signed with a SAN for 'localhost'
keytool -genkeypair -keystore kafka.keystore.jks -storepass changeit \
  -alias broker -keyalg RSA -validity 365 -dname "CN=localhost" \
  -ext SAN=DNS:localhost,DNS:kafka
keytool -certreq -keystore kafka.keystore.jks -storepass changeit \
  -alias broker -file broker.csr
openssl x509 -req -CA ca.crt -CAkey ca.key -in broker.csr -out broker.crt \
  -days 365 -CAcreateserial \
  -extfile <(printf "subjectAltName=DNS:localhost,DNS:kafka")
keytool -importcert -keystore kafka.keystore.jks -storepass changeit \
  -alias ca -file ca.crt -noprompt
keytool -importcert -keystore kafka.keystore.jks -storepass changeit \
  -alias broker -file broker.crt -noprompt

# 3. Truststore (what the CLIENT needs and won't get)
keytool -importcert -keystore kafka.truststore.jks -storepass changeit \
  -alias ca -file ca.crt -noprompt

echo "changeit" > creds

docker-compose.yml, KRaft, one broker, TLS on 9093:

services:
  kafka:
    image: apache/kafka:3.8.0
    ports: ["9093:9093"]
    volumes: ["./secrets:/etc/kafka/secrets"]
    environment:
      KAFKA_NODE_ID: 1
      KAFKA_PROCESS_ROLES: broker,controller
      KAFKA_CONTROLLER_QUORUM_VOTERS: 1@kafka:29094
      KAFKA_LISTENERS: SSL://0.0.0.0:9093,CONTROLLER://kafka:29094
      KAFKA_ADVERTISED_LISTENERS: SSL://localhost:9093
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: SSL:SSL,CONTROLLER:PLAINTEXT
      KAFKA_CONTROLLER_LISTENER_NAMES: CONTROLLER
      KAFKA_INTER_BROKER_LISTENER_NAME: SSL
      KAFKA_SSL_KEYSTORE_FILENAME: kafka.keystore.jks
      KAFKA_SSL_KEYSTORE_CREDENTIALS: creds
      KAFKA_SSL_KEY_CREDENTIALS: creds
      KAFKA_SSL_TRUSTSTORE_FILENAME: kafka.truststore.jks
      KAFKA_SSL_TRUSTSTORE_CREDENTIALS: creds
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1

Now trigger each variant:

# Variant A — PKIX: SSL protocol, no truststore configured
cat > client-broken.properties <<'EOF'
security.protocol=SSL
EOF
kafka-console-producer.sh --bootstrap-server localhost:9093 \
  --producer.config client-broken.properties --topic demo
# -> SslAuthenticationException: SSL handshake failed / PKIX path building failed

# Variant B — hostname verification: trust the CA, connect by IP
cat > client-ip.properties <<'EOF'
security.protocol=SSL
ssl.truststore.location=./secrets/kafka.truststore.jks
ssl.truststore.password=changeit
EOF
kafka-console-producer.sh --bootstrap-server 127.0.0.1:9093 \
  --producer.config client-ip.properties --topic demo
# -> No subject alternative names matching IP address 127.0.0.1 found

Variant C is the protocol mixup: point an SSL-configured client at a PLAINTEXT port and you get Unsupported or unrecognized SSL message; point a PLAINTEXT client at the SSL port and the client just sees Connection to node -1 ... terminated during authentication or endless disconnects while the broker logs SSL handshake failed. If you're on Docker and the error mentions node -1, also rule out the advertised-listeners problem — that's a separate article.

Environment-specific triggers worth knowing: it only appears after infra rotates certificates (new CA not yet in your truststore); only from containers (the truststore path is valid on the host but not mounted into the image); only via IP-based bootstrap (SAN has DNS names, not IPs); or only after a JVM base-image upgrade replaced the cacerts file your pipeline had been appending to.

3. Why It Happens — Surface Level

TLS requires the client to validate the certificate chain the broker presents. Your JVM validates against a truststore — either ssl.truststore.location if set, or the JRE's default cacerts. A self-signed or internal-CA broker cert is in neither, so path building fails: PKIX path building failed. The handshake aborts, Kafka wraps it in SslAuthenticationException, and the connection dies before a single Kafka protocol byte is exchanged.

The SAN variant is the second gate: since Kafka 2.0 (KIP-294), ssl.endpoint.identification.algorithm defaults to https, meaning the client checks that the hostname it dialed appears in the certificate's Subject Alternative Names. Trusting the CA is not enough — the name must match too.

4. Why It Happens — Under the Hood

Kafka's network layer wraps each connection in a transport layer per listener security protocol. For SSL listeners that's SslTransportLayer, which drives the JSSE SSLEngine state machine over the same non-blocking socket Kafka uses for everything else. The handshake happens after TCP connect but before any Kafka request — API versions, metadata, everything waits behind it.

Three details explain the confusing symptoms:

The broker can't tell you what's wrong. A TLS handshake failure happens below the Kafka protocol, so the broker has no channel to return a Kafka error code — all it can do is close the socket and log Failed authentication locally. That's why a misconfigured client sees generic disconnects and why you must read both sides' logs.

Authentication errors are fatal by design. Since KIP-152 (Kafka 1.0), authentication failures are propagated to clients as AuthenticationException subclasses and are not retriable — unlike a network blip, retrying an untrusted cert will never succeed. The producer fails send() futures immediately; KafkaConsumer.poll() throws. This is deliberate fail-fast behavior; the retry loops you see in logs come from the framework above (Spring's container restarting the poll), not the client.

Hostname verification checks what you dialed, not what the broker is. The client verifies the name from the connection it opened — the bootstrap address first, then each broker's advertised listener hostname from metadata. This is why a cluster can pass bootstrap and then fail on individual brokers: your bootstrap DNS name is in the SAN, but the per-broker advertised hostnames aren't. Every name a client will ever dial — bootstrap CNAMEs, LB names, per-broker names, IPs if anyone bootstraps by IP — must be in the SAN list.

Unsupported or unrecognized SSL message is the JSSE engine receiving plaintext Kafka protocol bytes where a TLS ServerHello should be: your security.protocol doesn't match the listener's protocol map entry. It's not a certificate problem at all.

5. The Fix

Fix A — give the client the truststore (PKIX variant). Before/after on the client properties:

 security.protocol=SSL
+ssl.truststore.location=/etc/kafka/secrets/kafka.truststore.jks
+ssl.truststore.password=changeit

Spring Boot equivalent:

spring:
  kafka:
    bootstrap-servers: localhost:9093
    security:
      protocol: SSL
    ssl:
      trust-store-location: file:/etc/kafka/secrets/kafka.truststore.jks
      trust-store-password: changeit

On kafka-clients ≥ 2.7 (KIP-651) you can skip JKS entirely and use PEM — much friendlier for Kubernetes secrets:

security.protocol=SSL
ssl.truststore.type=PEM
ssl.truststore.location=/etc/kafka/secrets/ca.crt

Fix B — fix the SANs (hostname variant). The right fix is reissuing the broker cert with every dialable name:

-keytool -genkeypair ... -ext SAN=DNS:localhost
+keytool -genkeypair ... -ext SAN=DNS:localhost,DNS:kafka,DNS:kafka.prod.internal,IP:10.20.30.40

The wrong-but-popular fix is disabling verification on the client:

# Diagnostic ONLY. Never ship this: it permits trivial MITM.
ssl.endpoint.identification.algorithm=

Setting it to empty is acceptable for a five-minute test to confirm the failure is hostname matching rather than trust. If that makes the error disappear, you've proven the cert's SANs are wrong — now fix the cert, and remove the override.

Fix C — align protocols (mixup variant). Check the listener map on the broker, then set the client to the same protocol as the port you're dialing:

-security.protocol=PLAINTEXT
+security.protocol=SSL

And if the cluster uses SASL over TLS, it's SASL_SSL, not SSL — with SSL the handshake may even succeed and you'll fail later at authentication instead.

When to use which: POC/local — Fix A with PEM and a compose-mounted CA is the least ceremony. Production — Fix A plus Fix B done properly via your PKI, verification always on. Fix C isn't a tuning choice; it's simply matching reality.

6. Best Practices & The Better Design

Hand-run keytool is where TLS outages are born. The better design has three parts.

Automate issuance and rotation. On Kubernetes, cert-manager (or Strimzi, which runs its own CA and wires SANs for every listener automatically) issues broker certs with correct SANs and rotates them before expiry. Off Kubernetes, Vault's PKI engine does the same. Broker certs should be short-lived and boring.

Rotate broker keystores without restarts. Since KIP-226 (Kafka 2.0+... standard in every KRaft cluster), keystores and truststores are dynamic per-broker configs:

kafka-configs.sh --bootstrap-server localhost:9093 \
  --command-config admin.properties \
  --entity-type brokers --entity-name 1 --alter \
  --add-config listener.name.ssl.ssl.keystore.location=/etc/kafka/secrets/kafka.keystore.jks

Re---alter with the same path after replacing the file and the broker reloads the cert in place. No rolling restart, no rebalance storm as a side effect of cert renewal.

Standardize one client security snippet per environment. Every incident I've debugged in this class came from a service hand-rolling its own SSL properties. Publish a single, PEM-based, verification-on block:

# The right way — team-standard client TLS config
security.protocol=SSL
ssl.truststore.type=PEM
ssl.truststore.location=/etc/pki/kafka/ca-bundle.pem
ssl.endpoint.identification.algorithm=https   # explicit, so nobody "temporarily" blanks it

and mount /etc/pki/kafka from your secret store in every deployment. Clients then differ only in bootstrap servers.

7. How to Prevent It Long-Term

Monitor expiry, not failure. Alert on certificate notAfter at 30/14/7 days via blackbox-exporter TLS probes against every listener, or your PKI's own metrics. A handshake-failure alert fires after the outage; an expiry alert fires before it.

Probe the handshake in CI and health checks. This one-liner validates chain, dates, and SANs against a live listener and belongs in your deploy pipeline:

openssl s_client -connect kafka.prod.internal:9093 \
  -servername kafka.prod.internal -CAfile ca-bundle.pem </dev/null 2>/dev/null \
  | openssl x509 -noout -dates -ext subjectAltName

Keep a debug runbook. First move is always -Djavax.net.debug=ssl:handshake on the client — it prints the exact certificate chain received and which check failed, turning guesswork into a two-minute diagnosis. Second move is reading the broker's Failed authentication lines to confirm the failure is mutual.

Chaos-test rotation. Rotate the CA in staging on a schedule and verify clients survive. Dual-trust (old + new CA in the truststore bundle) during rotation windows is the pattern that makes CA rollovers a non-event.

Ban verification-off in CI. A grep gate for ssl.endpoint.identification.algorithm= (empty) in config repos catches the "temporary" fix before it reaches production. They're never temporary.

8. Key Takeaways

• SslAuthenticationException: SSL handshake failed is fatal and non-retriable by design (KIP-152) — retry loops in your logs are the framework, not the client, and they will never succeed.
• PKIX path building failed = your client's truststore (or JRE cacerts) doesn't contain the broker cert's CA. Mount the CA; on clients ≥ 2.7 use PEM (ssl.truststore.type=PEM) instead of JKS.
• No subject alternative names matching... = hostname verification (default https since Kafka 2.0). Fix the cert's SANs to cover every dialable name — bootstrap, per-broker advertised hostnames, and IPs. Blanking ssl.endpoint.identification.algorithm is a diagnostic, not a fix.
• Unsupported or unrecognized SSL message isn't a cert problem — your security.protocol doesn't match the listener. Check the broker's listener protocol map, and remember SASL_SSL ≠ SSL.
• The broker can only log Failed authentication ... (SSL handshake failed) — TLS fails below the Kafka protocol, so always read both client and broker logs, and reach for -Djavax.net.debug=ssl:handshake early.