Java IllegalStateException: Duplicate Key in Collectors.toMap

1. The Error

On JDK 9 and later, the two-argument Collectors.toMap fails like this:

Exception in thread "main" java.lang.IllegalStateException: Duplicate key payments (attempted merging values Ada and Grace)
	at java.base/java.util.stream.Collectors.duplicateKeyException(Collectors.java:135)
	at java.base/java.util.stream.Collectors.lambda$uniqKeysMapAccumulator$1(Collectors.java:182)
	at java.base/java.util.stream.ReduceOps$3ReducingSink.accept(ReduceOps.java:169)
	at java.base/java.util.AbstractList$RandomAccessSpliterator.forEachRemaining(AbstractList.java:722)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
	at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682)
	at Repro.main(Repro.java:18)

On Java 8 the same program prints something that has confused a decade of engineers:

Exception in thread "main" java.lang.IllegalStateException: Duplicate key Ada
	at java.util.stream.Collectors.lambda$throwingMerger$0(Collectors.java:133)
	at java.util.HashMap.merge(HashMap.java:1255)
	at java.util.stream.Collectors.lambda$toMap$58(Collectors.java:1320)
	at java.util.stream.ReduceOps$3ReducingSink.accept(ReduceOps.java:169)

Ada is not the key. It is the value already sitting in the map. Java 8's throwingMerger() is a BinaryOperator<V> — it only ever sees the two colliding values, so it printed one of them under the label "Duplicate key". That was JDK-8040892 ("Misleading exception message from Collectors.toMap()"), fixed in JDK 9 build 12; JDK-8173464 was closed as a duplicate of it. If you are on 8, ignore the identifier in the message — grep your key mapper instead.

The format strings, read straight out of the bytecode:

$ javap -p -c java.util.stream.Collectors | grep -i "Duplicate key"
# JDK 8:        // String Duplicate key %s
# JDK 11/17/21: // String Duplicate key %s (attempted merging values %s and %s)

Two neighbours throw a different exception for the same shape of mistake, and searchers often land here by accident:

java.lang.IllegalArgumentException: duplicate key: payments
	at java.base/java.util.ImmutableCollections$MapN.<init>(ImmutableCollections.java:1196)
	at java.base/java.util.Map.ofEntries(Map.java:1680)

Map.of / Map.ofEntries (Java 9+) throw IllegalArgumentException, lowercase duplicate key:. Collectors.toMap and Collectors.toUnmodifiableMap throw IllegalStateException, capital Duplicate key.

2. How to Reproduce It

Single file, runs on Java 11+ with the source launcher (JEP 330); records need 16+:

import java.util.List;
import java.util.Map;
import java.util.stream.Collectors;

public class Repro {

    record Employee(String id, String dept, String name) {}

    static List<Employee> staff() {
        return List.of(
            new Employee("e1", "payments", "Ada"),
            new Employee("e2", "risk",     "Linus"),
            new Employee("e3", "payments", "Grace"));
    }

    public static void main(String[] args) {
        Map<String, String> byDept = staff().stream()
            .collect(Collectors.toMap(Employee::dept, Employee::name));
        System.out.println(byDept);
    }
}

$ java Repro.java          # Java 11+ single-file launcher
# Java 8: replace the record with a class, then
$ javac Repro.java && java Repro

Environment-specific triggers worth knowing:

• Data-dependent. Unit tests with three curated rows pass; production explodes on the first tenant that has two rows sharing a business key. This is the single most common "worked in staging" failure of toMap.
• Parallel streams change the frame and sometimes the exception. With parallelStream() the collision can be detected in the combiner instead of the accumulator, and ForkJoinTask rethrows a reconstructed copy — so you get a nested message:

Exception in thread "main" java.lang.IllegalStateException: java.lang.IllegalStateException: Duplicate key d1629 (attempted merging values n1629 and n6629)
	at java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:62)
	at java.base/java.util.concurrent.ForkJoinTask.getThrowableException(ForkJoinTask.java:540)
	...
	at java.base/java.util.stream.Collectors.lambda$uniqKeysMapMerger$0(Collectors.java:157)

Which of the two colliding values is reported is not deterministic under parallelism — it depends on how the fork/join split lands.

• Comparator-defined equality. With a supplier like TreeMap::new and String.CASE_INSENSITIVE_ORDER, "Payments" and "payments" collide even though equals says they differ — and with a merge function present they collapse silently.

3. Version Behaviour Matrix

Line numbers in Collectors.java drift between releases (133 on 11, 135 on 17/21); the method names duplicateKeyException and lambda$uniqKeysMapAccumulator$1 are the stable fingerprint.

4. Why It Happens — Surface Level

Collectors.toMap(keyMapper, valueMapper) is a one-to-one contract. The Javadoc is explicit: "If the mapped keys contain duplicates (according to Object.equals(Object)), an IllegalStateException is thrown when the collection operation is performed. If the mapped keys might have duplicates, use toMap(Function, Function, BinaryOperator) instead."

So the exception is not a bug in your stream. It is the JDK telling you that the cardinality you assumed — one employee per department, one order per customer, one config per environment — is not the cardinality your data actually has. The collector refuses to pick a winner for you.

5. Why It Happens — Under the Hood

Since JDK 9 the two-argument toMap is built on a private accumulator, not on merge:

// java.util.stream.Collectors (JDK 9+), private
private static <T, K, V> BiConsumer<Map<K,V>, T> uniqKeysMapAccumulator(
        Function<? super T, ? extends K> keyMapper,
        Function<? super T, ? extends V> valueMapper) {
    return (map, element) -> {
        K k = keyMapper.apply(element);
        V v = Objects.requireNonNull(valueMapper.apply(element));
        V u = map.putIfAbsent(k, v);
        if (u != null) throw duplicateKeyException(k, u, v);
    };
}

Three consequences fall straight out of that shape, and all three are observable:

The key is now in scope, so the message improved. throwingMerger on 8 was a BinaryOperator<V> handed to HashMap.merge — by the time it ran, the key had been left behind on the stack of the caller. uniqKeysMapAccumulator holds k, u and v together, which is exactly what duplicateKeyException(k, u, v) formats.

putIfAbsent makes "already present" and "mapped to null" indistinguishable, so a null value had to be banned outright. Hence the Objects.requireNonNull — and hence this asymmetry, verified on 21:

2-arg toMap, null KEY   -> {null=x}                      // accepted, silently
2-arg toMap, null VALUE -> NullPointerException at java.base/java.util.Objects.requireNonNull(Objects.java:233)
                             at Collectors.lambda$uniqKeysMapAccumulator$1(Collectors.java:180)

A HashMap happily stores a null key, so toMap inherits that. A null value throws an NPE with no message at all — a bare java.lang.NullPointerException — because Objects.requireNonNull(Object) is the one-arg overload. If you have ever chased a message-less NPE inside a stream collect, this is very often the culprit. Note the JDK 8 difference: there the same input throws from java.util.HashMap.merge(HashMap.java:1226) instead, because 8 routed everything through merge.

The three- and four-argument overloads still use merge, which is why their null-value NPE has a different frame:

3-arg toMap, null value -> NullPointerException
	at java.base/java.util.HashMap.merge(HashMap.java:1363)
	at java.base/java.util.stream.Collectors.lambda$toMap$68(Collectors.java:1636)

And it is why a merge function that returns null does not store null — it deletes the entry, per Map.merge's contract ("If the remapping function returns null, the mapping is removed"). Verified on 8, 11, 17 and 21:

staff.stream().collect(Collectors.toMap(Emp::dept, Emp::name, (x, y) -> null));
// => {risk=Linus}   -- "payments" has vanished entirely

Parallel collection has a second detection site. uniqKeysMapMerger() walks the right-hand partial map and does the same putIfAbsent check while combining, so with parallelStream() the throw can originate in ReduceOps$3ReducingSink.combine on a ForkJoinPool.commonPool-worker-N thread. ForkJoinTask.getThrowableException then tries to rebuild the exception via Constructor.newInstance so the stack trace reaches the caller — producing the IllegalStateException: java.lang.IllegalStateException: ... double-wrap shown above.

Collision is equals/hashCode, or the supplied Comparator. With the default HashMap, two keys collide when hashCode puts them in the same bucket and equals returns true. A key type with a broken or mutable equals/hashCode will therefore either duplicate silently (missing collisions you expected) or blow up on rows you thought were distinct. With a TreeMap supplier, equals is irrelevant — compare(...) == 0 defines the collision, which is how "Payments" and "payments" merge under CASE_INSENSITIVE_ORDER. This is the same contract surface as IllegalArgumentException: Comparison method violates its general contract! in TimSort: a comparator that disagrees with equals breaks SortedMap uniqueness the same way it breaks sorting.

6. The Fix

Diff — the quick, honest fix when a collision is legitimate:

  Map<String, String> byDept = staff().stream()
-     .collect(Collectors.toMap(Employee::dept, Employee::name));
+     .collect(Collectors.toMap(Employee::dept, Employee::name,
+                               (first, second) -> first));   // first-wins, explicit

Pick from these, in this order of preference:

1. The key is not unique — model it as one-to-many. Collectors.groupingBy is almost always what you actually meant:

   Map<String, List<String>> namesByDept = staff().stream()
       .collect(Collectors.groupingBy(Employee::dept,
                Collectors.mapping(Employee::name, Collectors.toList())));
   // {payments=[Ada, Grace], risk=[Linus]}

2. You picked the wrong key. toMap on a genuinely unique column is fine and stays fine:

   Map<String, Employee> byId = staff().stream()
       .collect(Collectors.toMap(Employee::id, e -> e));

3. You really do want to collapse. Supply a merge function and say which one wins. (a, b) -> a is first-wins, (a, b) -> b is last-wins, (a, b) -> a.merge(b) is a real reduction. Add a LinkedHashMap::new supplier if "first" needs to mean encounter order — plain toMap returns a HashMap (getClass() on 21 confirms class java.util.HashMap) with no ordering guarantee at all:

   Map<String, String> firstSeenByDept = staff().stream()
       .collect(Collectors.toMap(Employee::dept, Employee::name,
                (first, second) -> first, LinkedHashMap::new));
   // {payments=Ada, risk=Linus}

4. Java 8 codebases: all four overloads exist in 8. Only toUnmodifiableMap (10+) and Stream.toList() (16+) are unavailable; use Collections.unmodifiableMap(...) around the result.

Do not "fix" this by swallowing the exception, and do not reach for (a, b) -> b reflexively — silently discarding a row is a data-correctness bug that will surface much later, in a reconciliation report, rather than at the collect site.

7. Best Practices & The Better Design

The message gives you the key and the two values — not the two source elements. In a financial pipeline, "attempted merging values 4711.00 and 5120.00" tells you almost nothing about which two records to go look at, and it happily writes business data into your logs. Fail with a diagnostic that names the offending elements instead:

import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import java.util.function.Function;
import java.util.stream.Collector;
import java.util.stream.Collectors;

public class Guard {

    record Employee(String id, String dept, String name) {}

    /** toMap that reports every duplicate key and the elements behind it. */
    static <T, K, V> Collector<T, ?, Map<K, V>> strictToMap(
            Function<? super T, ? extends K> keyMapper,
            Function<? super T, ? extends V> valueMapper) {
        return Collectors.collectingAndThen(
            Collectors.groupingBy(keyMapper, LinkedHashMap::new, Collectors.toList()),
            grouped -> {
                List<String> dupes = grouped.entrySet().stream()
                    .filter(e -> e.getValue().size() > 1)
                    .map(e -> e.getKey() + " -> " + e.getValue())
                    .toList();
                if (!dupes.isEmpty()) {
                    throw new IllegalStateException("Duplicate keys: " + String.join("; ", dupes));
                }
                return grouped.entrySet().stream().collect(
                    Collectors.toMap(Map.Entry::getKey,
                                     e -> valueMapper.apply(e.getValue().get(0)),
                                     (a, b) -> a,
                                     LinkedHashMap::new));
            });
    }

    public static void main(String[] args) {
        List<Employee> staff = List.of(
            new Employee("e1", "payments", "Ada"),
            new Employee("e2", "risk",     "Linus"),
            new Employee("e3", "payments", "Grace"));

        System.out.println(staff.stream().collect(strictToMap(Employee::id, Employee::name)));
        System.out.println(staff.stream().collect(strictToMap(Employee::dept, Employee::name)));
    }
}

{e1=Ada, e2=Linus, e3=Grace}
Exception ... Duplicate keys: payments -> [Employee[id=e1, dept=payments, name=Ada], Employee[id=e3, dept=payments, name=Grace]]

Beyond that:

• Make key types records. Records give you equals/hashCode derived from all components, so a compound key (record LedgerKey(String account, LocalDate valueDate) {}) cannot drift out of sync with itself. Never key a map on a mutable object whose fields you later change — the entry becomes unreachable without any exception at all.
• Prefer toUnmodifiableMap (Java 10+) for lookup tables built once at startup. Its Javadoc, unlike toMap's, explicitly states that null keys and values are rejected — the null-handling contract is documented rather than emergent.
• Never rely on toMap's return type or ordering. Pass a supplier when you need LinkedHashMap, TreeMap or EnumMap.
• Enforce uniqueness upstream. A unique index in the database, or a DISTINCT ON in the query, turns a runtime IllegalStateException into a constraint violation at write time, which is where it belongs.
• Treat a null-value NPE inside a collect as this bug's sibling. Message-less NullPointerException at Objects.requireNonNull two frames under ReduceOps$3ReducingSink.accept is toMap rejecting a null value, not a null element.

8. How to Prevent It Long-Term

• Static analysis. Error Prone ships StreamToString and related stream checks; more usefully, NullAway plus JSpecify annotations on your value mappers catch the null-value NPE before the duplicate-key one ever fires. Turn on -Xlint:all -Werror in CI.
• Property tests over curated fixtures. The failure is data-shaped. A test that feeds toMap a generated list with a deliberately repeated key is worth more than ten hand-written rows.
• Run the collect in a parallel-stream test too. Duplicate detection moves from uniqKeysMapAccumulator to uniqKeysMapMerger, and the ForkJoinTask rewrap changes what your error handling sees. If you catch IllegalStateException and inspect getMessage(), the parallel path will surprise you — inspect getCause()/getSuppressed() as well.
• JFR in production. jcmd <pid> JFR.start settings=profile records jdk.JavaExceptionThrow events; filtering those for java.lang.IllegalStateException shows you which pipelines are throwing before the on-call ticket arrives.
• Log the key, never the values. If you must catch and report, re-derive the offending key from your own data rather than echoing the JDK message, which contains customer values verbatim.
• Convention: ban bare two-argument toMap in code review unless the key is a primary key or a validated unique index. Everything else uses groupingBy or an explicit merge function.

9. Key Takeaways

• IllegalStateException: Duplicate key means your assumed one-to-one cardinality is wrong — the JDK is refusing to silently drop a row.
• On Java 8 the message prints a value, not the key. Duplicate key Ada where Ada is a name, not a department. Fixed in JDK 9 (JDK-8040892); the message has been stable through 25.
• The two-argument toMap uses putIfAbsent and therefore rejects null values with a message-less NPE while silently accepting a null key; the three- and four-argument overloads use HashMap.merge and throw from a different frame.
• A merge function returning null removes the entry rather than storing null — a silent data-loss trap.
• Fix by intent: groupingBy when the key repeats legitimately, a different key when you picked the wrong one, an explicit merge function plus LinkedHashMap::new when you deliberately collapse.