Kafka InvalidTopicException: Why It Happens and How to Fix It

You ran createTopics, or a producer touched a new topic, or Kafka Streams tried to build an internal repartition topic — and instead of a partition, you got an InvalidTopicException. This is not a broker outage, not a metadata race, and not a permissions problem. The broker looked at the name you handed it, ran it through Topic.validate(), and rejected it before any partition was ever placed. This is a fast-fail validation error, and the fix is almost always in your string, not your cluster.

This article is the exact rule set behind INVALID_TOPIC_EXCEPTION (error code 17), the four distinct triggers, the ./_ collision trap that bites teams six months into production, and how to make illegal names impossible to ship.

1. The Error

The canonical form, thrown from an AdminClient call:

org.apache.kafka.common.errors.InvalidTopicException: Topic name "orders@prod" is illegal, it contains a character other than ASCII alphanumerics, '.', '_' and '-'

Wrapped in the ExecutionException you actually catch off a KafkaFuture:

java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.InvalidTopicException: Topic name "orders@prod" is illegal, it contains a character other than ASCII alphanumerics, '.', '_' and '-'
	at java.base/java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:396)
	at org.apache.kafka.common.internals.KafkaFutureImpl.get(KafkaFutureImpl.java:165)

The other message variants, all the same exception class and error code 17:

InvalidTopicException: Topic name is illegal, it can't be empty
InvalidTopicException: Topic name cannot be "." or ".."
InvalidTopicException: Topic name is illegal, it can't be longer than 249 characters, topic name: <the-249+-char-name>
InvalidTopicException: Topic 'metric.name' collides with existing topic 'metric_name'

InvalidTopicException extends ApiException — it is not a RetriableException. Retrying is pointless; the name will fail identically on every attempt. Any occurrence in logs is a real signal, not transient noise.

Where it shows up: AdminClient.createTopics() / createPartitions(), KafkaConsumer.subscribe() with an illegal literal topic, producer sends against an illegal topic when auto-creation is on, and — the one people don't expect — Kafka Streams building internal topics like <application.id>-<store>-repartition when the application.id or a Materialized store name contains illegal characters. Environment: any broker version (the rule has been stable for years, KRaft and ZooKeeper alike), kafka-clients 2.x–4.x.

2. How to Reproduce It (step-by-step)

Minimal single-broker KRaft cluster:

# docker-compose.yml
services:
  kafka:
    image: apache/kafka:3.9.0
    container_name: kafka
    ports:
      - "9092:9092"
    environment:
      KAFKA_NODE_ID: 1
      KAFKA_PROCESS_ROLES: broker,controller
      KAFKA_CONTROLLER_QUORUM_VOTERS: 1@localhost:9093
      KAFKA_LISTENERS: PLAINTEXT://:9092,CONTROLLER://:9093
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9092
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,CONTROLLER:PLAINTEXT
      KAFKA_CONTROLLER_LISTENER_NAMES: CONTROLLER
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
      KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1
      KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1

docker compose up -d

Trigger it straight from the CLI — the fastest way to see the exact message:

# illegal character
docker exec -it kafka /opt/kafka/bin/kafka-topics.sh \
  --bootstrap-server localhost:9092 \
  --create --topic 'orders@prod' --partitions 1 --replication-factor 1
# => InvalidTopicException: Topic name "orders@prod" is illegal, it contains a
#    character other than ASCII alphanumerics, '.', '_' and '-'

# too long (250 chars)
docker exec -it kafka /opt/kafka/bin/kafka-topics.sh \
  --bootstrap-server localhost:9092 \
  --create --topic "$(printf 'a%.0s' {1..250})" --partitions 1 --replication-factor 1
# => InvalidTopicException: Topic name is illegal, it can't be longer than 249 characters

Reproduce the collision — create two names that differ only by . vs _:

docker exec -it kafka /opt/kafka/bin/kafka-topics.sh \
  --bootstrap-server localhost:9092 \
  --create --topic 'metric_name' --partitions 1 --replication-factor 1
# OK

docker exec -it kafka /opt/kafka/bin/kafka-topics.sh \
  --bootstrap-server localhost:9092 \
  --create --topic 'metric.name' --partitions 1 --replication-factor 1
# => InvalidTopicException: Topic 'metric.name' collides with existing topic 'metric_name'

And from Java, the shape most apps hit — an illegal name assembled from config or user input:

// kafka-clients 3.9.0
try (Admin admin = Admin.create(Map.of("bootstrap.servers", "localhost:9092"))) {
    String env = "prod:eu";                 // colon from an env label, oops
    NewTopic topic = new NewTopic("orders." + env, 3, (short) 1);
    admin.createTopics(List.of(topic)).all().get();   // throws here
} catch (ExecutionException e) {
    // e.getCause() instanceof InvalidTopicException
    throw e;
}

Environment-specific trigger worth knowing: the collision case only fires when the other name already exists on the cluster. On a fresh cluster both metric.name and metric_name succeed individually; the loser is whichever is created second. That's why this passes every local test and detonates in a shared environment where someone else already owns the sibling name.

3. Why It Happens — Surface Level

Kafka enforces a strict allow-list for topic names: ASCII letters and digits, plus ., _, and -. Nothing else. No spaces, no @, :, /, #, no unicode, no emoji. The name must be non-empty, at most 249 characters, and cannot be the reserved . or ... Break any of those and Topic.validate() throws InvalidTopicException synchronously, before the broker does any real work.

The 249 limit and the ./_ restriction aren't arbitrary. Topic names become filesystem directory names (<topic>-<partition> under each log dir), and they become JMX metric names. The character rules exist so a topic name is always a safe path segment and a safe metric label.

4. Why It Happens — Under the Hood

The validation lives in org.apache.kafka.common.internals.Topic.validate(String), shared by the CLI, AdminClient request handling, and the broker. Conceptually it is:

static final int MAX_NAME_LENGTH = 249;

public static void validate(String topic) {
    if (topic.isEmpty())
        throw new InvalidTopicException("Topic name is illegal, it can't be empty");
    if (topic.equals(".") || topic.equals(".."))
        throw new InvalidTopicException("Topic name cannot be \".\" or \"..\"");
    if (topic.length() > MAX_NAME_LENGTH)
        throw new InvalidTopicException(
            "Topic name is illegal, it can't be longer than " + MAX_NAME_LENGTH +
            " characters, topic name: " + topic);
    if (!containsValidPattern(topic))   // regex [a-zA-Z0-9._-]+ over the whole string
        throw new InvalidTopicException("Topic name \"" + topic +
            "\" is illegal, it contains a character other than ASCII alphanumerics, '.', '_' and '-'");
}

The collision check is separate and lives in the create path (the AdminManager/ReplicationControlManager on the broker). Kafka computes a normalized form of each name by mapping every . and _ to the same character, then compares against existing topics. metric.name and metric_name normalize to the same string, so the second one to be created collides. The reason is the JMX metric namespace: metric names use . as a hierarchy separator, so metric.name and metric_name would produce indistinguishable metric MBeans. Kafka refuses to let both exist to keep per-topic metrics unambiguous.

On the request path this all happens client-side or at the controller before any TopicRecord is written. Under KRaft, the QuorumController runs Topic.validate() inside ReplicationControlManager.createTopics() before appending anything to __cluster_metadata; the response carries error code 17 for just the offending topic in the batch (per-topic results, so a batch of five with one bad name creates the other four). That's why you never see half-created state from this error — validation is a pure function of the string, evaluated up front.

The Streams angle is the same rule reached indirectly. Streams derives internal topic names from application.id plus operator/store names: ${application.id}-${name}-repartition, -changelog, etc. If application.id contains a space or a colon (a very common "I named it after my service URL" mistake), the derived internal topic name is illegal and Streams dies at startup building its topology — the stack trace points at internal topic creation, not at anything you obviously wrote.

5. The Fix

The fix is to make the name legal. What "legal" means depends on which of the four rules you tripped.

Illegal characters — sanitize the name to the allow-list. The usual culprits are separators borrowed from other systems (:, /, @) and human-friendly labels with spaces.

- NewTopic topic = new NewTopic("orders." + "prod:eu", 3, (short) 1);
+ String env = "prod-eu";                       // '-' instead of ':'
+ NewTopic topic = new NewTopic("orders." + env, 3, (short) 1);

Too long — you are over 249 characters, almost always because a name is machine-composed from several fields. Shorten the scheme; don't try to raise the limit (you can't).

- "events." + tenantId + "." + region + "." + serviceName + "." + eventType + "." + version
+ // hash or abbreviate the low-value segments; keep the name well under 249
+ "events." + tenantId + "." + region + "." + shortCode(serviceName, eventType) + ".v" + version

Empty name — you passed "" or null-derived-to-empty, usually from an unset property. Fail fast at config load instead of at the broker (see §6).

Collision — pick one separator convention and standardize. Do not create a.b when a_b exists (or vice versa). The clean fix is a team-wide rule: dots for hierarchy, never underscores (or the reverse), and never mix.

- topic.name = order_events        # some services use underscores
- topic.name = order.events        # others use dots -> collides
+ topic.name = order.events        # ONE convention everywhere

For Kafka Streams, fix the application.id (and any explicit store/repartition names) so the derived internal topics are legal:

- props.put(StreamsConfig.APPLICATION_ID_CONFIG, "orders svc:v2");   // space + colon
+ props.put(StreamsConfig.APPLICATION_ID_CONFIG, "orders-svc-v2");

There is no server-side config to relax these rules. message.max.bytes, retention, replication — none of them apply. The name is the contract.

6. Best Practices & The Better Design

The better design removes the possibility of an illegal name reaching a broker at all, by validating at the earliest boundary and by owning topic names as declared artifacts rather than runtime strings.

Validate names where they're born. If a topic name comes from configuration or user input, check it against the exact Kafka rule at startup and fail loudly, with a message a developer can act on — not a broker round-trip later:

private static final Pattern LEGAL = Pattern.compile("[a-zA-Z0-9._-]+");

static String requireLegalTopic(String name) {
    if (name == null || name.isEmpty())
        throw new IllegalStateException("Topic name is unset/empty");
    if (name.length() > 249)
        throw new IllegalStateException("Topic name > 249 chars: " + name);
    if (name.equals(".") || name.equals(".."))
        throw new IllegalStateException("Topic name cannot be '.' or '..'");
    if (!LEGAL.matcher(name).matches())
        throw new IllegalStateException(
            "Topic name has illegal chars (allowed: a-z A-Z 0-9 . _ -): " + name);
    return name;
}

Own topics as code, with one naming convention. Declare topics in Strimzi KafkaTopic CRDs, Terraform, or a checked-in manifest — not by string-building at runtime. A declared topic name is code-reviewed, so an illegal character or a ./_ collision is caught in a pull request, not in production logs. Standardize on a single separator: pick . for namespacing (domain.entity.event, e.g. orders.payment.captured) and ban _ in topic names entirely, or the reverse — but write it down and lint it.

In Spring Kafka, declare NewTopic beans so provisioning goes through KafkaAdmin and names live in one place:

@Bean
NewTopic ordersTopic() {
    return TopicBuilder.name("orders.payment.captured")   // reviewed, legal, one convention
        .partitions(6)
        .replicas(3)
        .build();
}

For Streams, treat application.id as a topic-name prefix and hold it to the same rule — because it is one. Keep it short (it's prepended to every internal topic, so it eats into the 249 budget), lowercase, and separator-consistent.

The throughline: an illegal topic name is a naming-discipline failure, and naming discipline is enforced in code review and CI, not by hoping the right string reaches the broker.

7. How to Prevent It Long-Term

Put a CI gate on topic names. If topics are declared as code, add a test that runs every declared name through the exact Topic.validate() regex and length/collision checks, and additionally asserts your team convention (e.g. "only . as a separator, no _"). A five-line unit test kills this class of bug permanently.

Add a collision detector across your whole topic inventory: normalize every declared name by mapping . and _ to a single character and assert the set has no duplicates. This catches the cross-team a.b/a_b clash before it's created, which is the one failure mode that passes local testing.

Lint application.id and any explicit Streams store/repartition names with the same rule, since Streams turns them into topic names you never typed.

Alert on error code 17 at the broker (InvalidTopicExceptionRate via request metrics) — with topics-as-code, a legitimate app should never produce this error, so any occurrence means something is bypassing your provisioning path (an app auto-creating topics from raw input, a rogue producer with auto.create.topics.enable on). Turn auto.create.topics.enable=false on production clusters so a bad name from application code fails visibly at create time instead of silently spawning garbage topics.

Finally, run a Testcontainers smoke test that provisions your real declared topics against a throwaway broker. It exercises the actual validation path — including the collision check against topics your manifest creates earlier in the same batch — which a pure regex test can miss.

8. Key Takeaways

• Topic names allow only a-z A-Z 0-9 . _ -, must be non-empty, ≤ 249 chars, and can't be . or ... Anything else throws InvalidTopicException (error code 17). It's not retriable — the name is wrong, retrying won't help.
• . and _ collide. metric.name and metric_name can't coexist because they map to the same JMX metric namespace. Standardize on one separator team-wide and ban the other.
• The collision only fails when the sibling already exists, so it passes local tests and detonates in shared environments. Detect it across your whole topic inventory in CI, not at runtime.
• Kafka Streams turns application.id and store names into internal topic names, so an illegal application.id (spaces, colons) kills startup with a confusing internal-topic stack trace.
• Own topic names as code (Strimzi/Terraform/NewTopic beans), validate them in CI against the exact rule, and disable broker auto-creation so illegal names fail in review, not in production.