ForkJoinPool in Java: Work-Stealing and the Common Pool

Forty lookups, eight seconds

sleep    parallelism=1  40 x 200 ms  wall= 8011 ms  peak pool size=1

Forty FX-rate lookups, each one a 200 ms wait. Eight seconds, for work that was nearly all waiting. That's ForkJoinPool.commonPool() on a 2-vCPU box running OpenJDK 21.0.12.1, and it's the symptom I see most often when a team says "we made it async and it got slower".

The common pool had exactly one worker, so the forty waits ran one after another. Nothing was broken. The pool did what it's built to do, and what it's built to do is not "wait on the network".

I used to believe the common pool noticed blocked workers and quietly added threads. That's half true, and the half that's true depends on which blocking call you make, in a way that changed between JDK 11 and JDK 14. The labs below map out which half is which.

Fork/join is a scheduler for work that splits

The short version: a ForkJoinPool is a fixed set of worker threads, each with its own double-ended queue (a deque) of tasks. A worker pushes the subtasks it forks onto its own deque and pops them back off the same end, newest first. A worker with nothing to do steals from the opposite end of someone else's deque, which is where the oldest and usually biggest pieces of work sit. That's work-stealing.

The guarantee you get is about throughput for CPU-bound, divide-and-conquer work: the pool tries to keep parallelism workers busy. It makes no promise about blocking. If a task sleeps, the worker running it is gone for that long, unless the task tells the pool it's about to block. Some JDK classes do that for you. Others don't.

Five experiments in one file

What you need. Any JDK 11 or newer. My numbers come from two machines:

• OpenJDK 21.0.12.1 (Ubuntu build), x86_64, 2 vCPU, Linux.
• OpenJDK 11.0.32.1 (Ubuntu build), aarch64, 4 cores, a Linux VM on an ARM laptop.

I didn't have a JDK 25 binary available for this run. Where I say something about 25, it comes from reading the jdk-25-ga source and Javadoc, not from running it.

Get a JDK. Use whatever you've got, or a container if you want to match versions without touching your machine:

docker run --rm -it -v "$PWD":/lab -w /lab eclipse-temurin:21-jdk bash
java -version

The code. One file, no build tool. It compiles on JDK 11 and 21 unchanged, so you can compare versions with the same source. Save it as Lab.java:

import java.util.*;
import java.util.concurrent.*;
import java.util.concurrent.atomic.*;

// One file, five experiments. Compiles and runs on JDK 11 and JDK 21: java Lab.java <mode>
public class Lab {

    public static void main(String[] args) throws Exception {
        String mode = args.length > 0 ? args[0] : "info";
        if (mode.equals("info")) info();
        else if (mode.equals("skew")) skew(args.length > 1 ? Integer.parseInt(args[1]) : Runtime.getRuntime().availableProcessors());
        else if (mode.equals("block")) block(args[1], args.length > 2 ? Integer.parseInt(args[2]) : 200,
                                                args.length > 3 ? Integer.parseInt(args[3]) : 40);
        else if (mode.equals("order")) { order(false); order(true); }
        else System.out.println("modes: info | skew [p] | block sleep|managed|take|latch|cf [ms] [tasks] | order");
    }

    // ---------- info: what the common pool looks like on this machine ----------
    static void info() throws Exception {
        ForkJoinPool common = ForkJoinPool.commonPool();
        Thread w = common.submit(() -> Thread.currentThread()).get();
        System.out.println("java.version         = " + System.getProperty("java.version"));
        System.out.println("availableProcessors  = " + Runtime.getRuntime().availableProcessors());
        System.out.println("common parallelism   = " + ForkJoinPool.getCommonPoolParallelism());
        System.out.println("worker name / daemon = " + w.getName() + " / " + w.isDaemon());
    }

    // ---------- skew: static split vs work-stealing on uneven work ----------
    // A risk aggregation where the first 25% of positions are 12x more expensive to price.
    static final int N = 4_000_000;
    static double price(int i) {
        int rounds = i < N / 4 ? 240 : 20;              // the skew: options book vs plain equities
        double x = i;
        for (int r = 0; r < rounds; r++) x = Math.sqrt(x + r) * 1.0000001;
        return x;
    }

    static final ConcurrentHashMap<String, LongAdder> leavesByWorker = new ConcurrentHashMap<>();

    static class Exposure extends RecursiveTask<Double> {
        final int lo, hi;
        Exposure(int lo, int hi) { this.lo = lo; this.hi = hi; }
        @Override protected Double compute() {
            if (hi - lo <= 20_000) {
                leavesByWorker.computeIfAbsent(Thread.currentThread().getName(), k -> new LongAdder()).increment();
                double s = 0;
                for (int i = lo; i < hi; i++) s += price(i);
                return s;
            }
            int mid = (lo + hi) >>> 1;
            Exposure left = new Exposure(lo, mid);
            left.fork();                                  // push left onto MY deque; an idle worker may steal it
            double right = new Exposure(mid, hi).compute(); // work on the right half myself
            return right + left.join();                   // join last: by now left is often already done
        }
    }

    static void skew(int p) throws Exception {
        for (int round = 1; round <= 3; round++) {
            // static split: p equal chunks on a fixed pool
            ExecutorService fixed = Executors.newFixedThreadPool(p);
            long t0 = System.nanoTime();
            List<Future<Double>> parts = new ArrayList<>();
            int chunk = N / p;
            for (int c = 0; c < p; c++) {
                final int lo = c * chunk, hi = (c == p - 1) ? N : lo + chunk;
                parts.add(fixed.submit(() -> { double s = 0; for (int i = lo; i < hi; i++) s += price(i); return s; }));
            }
            double a = 0; for (Future<Double> f : parts) a += f.get();
            long staticMs = (System.nanoTime() - t0) / 1_000_000;
            fixed.shutdown();
            fixed.awaitTermination(10, TimeUnit.SECONDS);

            // fork/join: split until 20k positions, let idle workers steal
            leavesByWorker.clear();
            ForkJoinPool pool = new ForkJoinPool(p);
            t0 = System.nanoTime();
            double b = pool.invoke(new Exposure(0, N));
            long fjMs = (System.nanoTime() - t0) / 1_000_000;
            long steals = pool.getStealCount();
            pool.shutdown();
            pool.awaitTermination(10, TimeUnit.SECONDS);

            System.out.printf("round %d  p=%d  static split %5d ms | fork/join %5d ms | steals=%d | leaves per worker %s | same total: %b%n",
                    round, p, staticMs, fjMs, steals, new TreeMap<>(leavesByWorker), Math.abs(a - b) < 1e-3 * Math.abs(a));
        }
    }

    // ---------- block: N tasks (default 40) that each wait ~200 ms, in the common pool ----------
    static final ScheduledExecutorService timer = Executors.newSingleThreadScheduledExecutor(r -> {
        Thread t = new Thread(r, "fx-feed"); t.setDaemon(true); return t; });

    static void waitFx(String kind, int ms) throws Exception {
        switch (kind) {
            case "sleep":                                  // plain sleep: the pool can't tell you're stuck
                Thread.sleep(ms);
                break;
            case "managed": {                              // same sleep, but announced as blocking
                final long until = System.nanoTime() + ms * 1_000_000L;
                ForkJoinPool.managedBlock(new ForkJoinPool.ManagedBlocker() {
                    public boolean block() throws InterruptedException {
                        long left = until - System.nanoTime();
                        if (left > 0) TimeUnit.NANOSECONDS.sleep(left);
                        return true;
                    }
                    public boolean isReleasable() { return System.nanoTime() >= until; }
                });
                break;
            }
            case "take": {                                 // BlockingQueue.take(): Condition.await() inside
                ArrayBlockingQueue<Double> q = new ArrayBlockingQueue<>(1);
                timer.schedule(() -> q.offer(1.0842), ms, TimeUnit.MILLISECONDS);
                q.take();
                break;
            }
            case "latch": {                                // CountDownLatch.await(): plain AQS park
                CountDownLatch l = new CountDownLatch(1);
                timer.schedule(l::countDown, ms, TimeUnit.MILLISECONDS);
                l.await();
                break;
            }
            case "cf": {                                   // CompletableFuture.join(): a ManagedBlocker inside
                CompletableFuture<Double> f = new CompletableFuture<>();
                timer.schedule(() -> f.complete(1.0842), ms, TimeUnit.MILLISECONDS);
                f.join();
                break;
            }
            default: throw new IllegalArgumentException(kind);
        }
    }

    static void block(String kind, int ms, int n) throws Exception {
        ForkJoinPool common = ForkJoinPool.commonPool();
        AtomicInteger peak = new AtomicInteger();
        Thread sampler = new Thread(() -> {
            while (!Thread.currentThread().isInterrupted()) {
                peak.accumulateAndGet(common.getPoolSize(), Math::max);
                try { Thread.sleep(2); } catch (InterruptedException e) { return; }
            }
        });
        sampler.setDaemon(true);
        sampler.start();

        long t0 = System.nanoTime();
        List<ForkJoinTask<?>> tasks = new ArrayList<>();
        for (int i = 0; i < n; i++) {
            tasks.add(common.submit(() -> {
                try { waitFx(kind, ms); }
                catch (InterruptedException e) { Thread.currentThread().interrupt(); }  // restore, don't swallow
                catch (Exception e) { throw new RuntimeException(e); }
            }));
        }
        for (ForkJoinTask<?> t : tasks) t.join();
        long wall = (System.nanoTime() - t0) / 1_000_000;
        sampler.interrupt();
        System.out.printf("%-8s parallelism=%d  %d x %d ms  wall=%5d ms  peak pool size=%d%n",
                kind, ForkJoinPool.getCommonPoolParallelism(), n, ms, wall, peak.get());
    }

    // ---------- order: forked-but-never-joined tasks, LIFO vs asyncMode FIFO ----------
    static void order(boolean asyncMode) throws Exception {
        ForkJoinPool pool = new ForkJoinPool(1, ForkJoinPool.defaultForkJoinWorkerThreadFactory, null, asyncMode);
        List<Integer> ran = Collections.synchronizedList(new ArrayList<>());
        pool.submit(() -> {
            for (int i = 1; i <= 6; i++) {
                final int id = i;
                ForkJoinTask.adapt(() -> ran.add(id)).fork();   // forked from inside a worker: goes to its own deque
            }
        }).get();                                               // the parent is done; the six children are still queued
        while (ran.size() < 6) Thread.sleep(10);                // one worker, so they run after the parent returns
        System.out.println("asyncMode=" + asyncMode + "  execution order " + ran);
        pool.shutdown();
    }
}

Step 1: look at your common pool.

$ java Lab.java info

java.version         = 21.0.12.1
availableProcessors  = 2
common parallelism   = 1
worker name / daemon = ForkJoinPool.commonPool-worker-1 / true

Two CPUs, one worker. The default is availableProcessors() - 1, never less than 1, on the assumption that the thread calling into the pool will help. On the 4-core ARM box with JDK 11 it said common parallelism = 3, and then the last line said main / false. Five runs out of five. I'll come back to that, because it's the JDK 11 pool doing something the 21 pool didn't in my runs.

Step 2: the opening number, then the fix nobody wrote.

$ java Lab.java block sleep
$ java -Djava.util.concurrent.ForkJoinPool.common.parallelism=4 Lab.java block sleep
$ java -Djava.util.concurrent.ForkJoinPool.common.parallelism=4 Lab.java block managed

sleep    parallelism=1  40 x 200 ms  wall= 8011 ms  peak pool size=1
sleep    parallelism=4  40 x 200 ms  wall= 2004 ms  peak pool size=4
managed  parallelism=4  40 x 200 ms  wall=  212 ms  peak pool size=41

Forty waits divided by the worker count, times 200 ms. With ManagedBlocker the pool started a spare thread for each blocked worker and the whole batch took one wait plus noise.

Step 3: the same 200 ms wait, through three JDK classes.

$ P=-Djava.util.concurrent.ForkJoinPool.common.parallelism=4
$ for k in take latch cf; do java $P Lab.java block $k; done

On JDK 21, x86, three runs each, all within a few ms of these:

take     parallelism=4  40 x 200 ms  wall=  212 ms  peak pool size=41
latch    parallelism=4  40 x 200 ms  wall= 2008 ms  peak pool size=4
cf       parallelism=4  40 x 200 ms  wall=  216 ms  peak pool size=41

On JDK 11, ARM, two runs each:

take     parallelism=4  40 x 200 ms  wall= 2040 ms  peak pool size=4
latch    parallelism=4  40 x 200 ms  wall= 2032 ms  peak pool size=4
cf       parallelism=4  40 x 200 ms  wall=  211 ms  peak pool size=41

Read those twice. BlockingQueue.take() gets compensated on 21 and doesn't on 11. CountDownLatch.await() gets nothing on either. CompletableFuture.join() gets a spare thread on both.

Step 4: push past the spare limit.

$ java $P Lab.java block cf 200 300
$ java $P Lab.java block take 200 300

Exception in thread "main" java.lang.RuntimeException: java.lang.RuntimeException: java.util.concurrent.RejectedExecutionException: Thread limit exceeded replacing blocked worker
...
take     parallelism=4  300 x 200 ms  wall=  413 ms  peak pool size=260

Same on both machines for cf and managed. Three hundred tasks blocked in CompletableFuture.join() inside the common pool, and some of them failed. Not slowed down. Failed. take on 21 hit the same ceiling of 260 threads (4 workers plus 256 spares) and then carried on without compensation, so it took two waits instead of one.

Step 5: work-stealing on uneven work.

$ java Lab.java skew 2

round 1  p=2  static split  2350 ms | fork/join  1403 ms | steals=2 | leaves per worker {ForkJoinPool-1-worker-1=157, ForkJoinPool-1-worker-2=99} | same total: true
round 2  p=2  static split  2369 ms | fork/join  1361 ms | steals=2 | leaves per worker {ForkJoinPool-2-worker-1=157, ForkJoinPool-2-worker-2=99} | same total: true
round 3  p=2  static split  2323 ms | fork/join  1359 ms | steals=2 | leaves per worker {ForkJoinPool-3-worker-1=157, ForkJoinPool-3-worker-2=99} | same total: true

On the ARM box with skew 4: static split 1,315 to 1,351 ms, fork/join 425 to 522 ms, 6 to 9 steals per round.

Step 6: queue order.

$ java Lab.java order

asyncMode=false  execution order [6, 5, 4, 3, 2, 1]
asyncMode=true  execution order [1, 2, 3, 4, 5, 6]

My first version of this mode called pool.awaitQuiescence(5, SECONDS) straight after submit and printed [] for the LIFO pool while the FIFO pool printed all six. I spent a while convinced LIFO mode was dropping tasks. It wasn't; my wait was wrong, and the version above waits for the parent and then for the six children explicitly. I never pinned down exactly why awaitQuiescence returned early there, so I stopped trusting it in labs.

Forcing levers. The common.parallelism property makes the 2-core and 4-core machines behave the same; use it instead of comparing raw defaults. -XX:ActiveProcessorCount=8 changes availableProcessors() and therefore the default. Raising the task count past parallelism + 256 is what turns compensation into exceptions. In production that number isn't exotic: a request fan-out of 30 lookups times 10 concurrent requests is 300.

How you know you saw it. The block results were the most stable numbers I've ever had in a concurrency lab: every run landed within about 30 ms of the figures above. If take comes back at ~2,000 ms on a JDK newer than 14, check you didn't change it to poll(timeout). The timed variant doesn't compensate (more on that below). If skew shows fork/join slower than the static split, your machine probably has fewer free cores than p.

When results differ. Containers with CPU limits change availableProcessors(), so the default parallelism is whatever the JVM thinks the quota is. The skew timings depend on core speed, not on the JDK; the ratios held on both machines. The ARM and x86 results for cf, managed, sleep and latch matched, so the take difference is the JDK version, not the CPU. The source confirms that below.

Cleanup. Nothing persists except any jcmd dump files you saved.

Deques, steals, and a worker that won't wait

Here's what happened in step 5, the one place this pool's design pays off.

worker-1 deque (top = newest)              worker-2 deque
 ┌────────────────────────┐                 ┌───────────┐
 │ [3.0M..3.5M)  <- pop   │   top           │  (empty)  │
 │ [2.0M..3.0M)           │                 └───────────┘
 │ [0..2.0M)     <- steal │   base  ──────────────▶ worker-2 takes the oldest,
 └────────────────────────┘                          biggest piece

1. invoke puts the root Exposure(0, 4M) in the pool. Worker 1 takes it.
2. Worker 1 splits it: forks [0, 2M) onto its own deque, then computes [2M, 4M) itself, splitting again and again. Each split pushes the left half on top.
3. Worker 2 is idle. It scans, finds worker 1's deque non-empty, and takes from the base, the opposite end. The base holds the first thing worker 1 forked: [0, 2M), half the whole job, and in my skew it's the expensive half.
4. Worker 2 now splits that into its own deque. Worker 1 finishes its cheap half, finds its own deque empty, and steals from worker 2's base. That's steal number two.
5. Everything else is local push and pop. left.join() mostly finds left already done, or pops it off its own deque and runs it in place.

Two steals for 256 leaf tasks. That number confused me at first. I expected hundreds. But steals are the expensive, contended operation, and taking from the base means each steal grabs a big subtree, so a thief rarely needs a second one. The static split had no such option: chunk 1 held the whole options book and took 65 "units" of work against chunk 2's 10. Fork/join split it 37.5 each. 65 / 37.5 is 1.73, and 2,350 / 1,359 is 1.73. Rarely does a lab agree with arithmetic that cleanly.

Step 6 shows the other side of the deque. A worker pops its own tasks newest-first, so forked-and-never-joined tasks run backwards. That's right for recursion (the newest piece is the smallest and its data is in cache), and wrong for event-style tasks where order matters. asyncMode = true (Java 7) flips the worker's own queue to FIFO for exactly that case. The virtual-thread scheduler runs a ForkJoinPool in this mode; the virtual threads article covers that pool, which is separate from the common pool.

Now the blocking. A worker that calls Thread.sleep is a thread asleep, nothing more. The pool counts it as active. It has no idea the worker is useless for 200 ms, so it starts nobody, and your 40 tasks queue behind 4 sleepers. ForkJoinPool.managedBlock(blocker) is the protocol for telling it: "I'm about to block; if that leaves you short of parallelism running workers, start or wake a spare". The pool calls blocker.isReleasable() first, and if blocking is still needed it compensates and calls blocker.block(). When the wait ends, the extra capacity winds down; spare threads idle out (the common pool's keep-alive is 60 seconds on 21, from javap of its constructor).

So the rule isn't "the pool handles blocking". The rule is: the pool handles blocking that goes through managedBlock. Everything in step 3 follows from which JDK classes go through it.

Where the JDK compensates, version by version

The common pool's settings, as javap -c -p java.util.concurrent.ForkJoinPool shows them in the JDK 21 constructor: parallelism from the java.util.concurrent.ForkJoinPool.common.parallelism property, else max(1, availableProcessors() - 1); spare threads from ...common.maximumSpares, else 256; keep-alive 60,000 ms; no saturation predicate. The JDK 25 Javadoc still documents 256 spares and says every worker has isDaemon set to true.

The 256 is where step 4's exception comes from. tryCompensate in ForkJoinPool.java reactivates an idle worker if there is one, creates a new one while the total is under the limit, and otherwise asks the pool's saturate predicate. The common pool has none, so it throws RejectedExecutionException("Thread limit exceeded replacing blocked worker"). You can pass a predicate yourself through the 10-argument constructor (Java 9); the common pool doesn't let you.

Then the per-class behaviour. This part cracked open when I put the take thread dump next to the JDK 11 one:

"ForkJoinPool.commonPool-worker-1" #19 daemon prio=5 ... waiting on condition
   java.lang.Thread.State: WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@21.0.12.1/Native Method)
	- parking to wait for  <0x00000000a87d7ae0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(java.base@21.0.12.1/LockSupport.java:371)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionNode.block(java.base@21.0.12.1/AbstractQueuedSynchronizer.java:519)
	at java.util.concurrent.ForkJoinPool.compensatedBlock(java.base@21.0.12.1/ForkJoinPool.java:3740)
	at java.util.concurrent.ForkJoinPool.managedBlock(java.base@21.0.12.1/ForkJoinPool.java:3723)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@21.0.12.1/AbstractQueuedSynchronizer.java:1746)
	at java.util.concurrent.ArrayBlockingQueue.take(java.base@21.0.12.1/ArrayBlockingQueue.java:420)
	at Lab.waitFx(Lab.java:112)

ConditionNode is a ManagedBlocker. In the JDK 21 class file, AbstractQueuedSynchronizer$ConditionNode implements ForkJoinPool$ManagedBlocker, and ConditionObject.await() and awaitUninterruptibly() call ForkJoinPool.managedBlock. awaitNanos (and so every timed wait built on a Condition) calls LockSupport.parkNanos directly and gets no spare. The exception table on await() catches RejectedExecutionException and falls back to a plain block(), which is why take survived 300 tasks and cf didn't.

The JDK 11 source (jdk-11+28) has no managedBlock anywhere in AbstractQueuedSynchronizer.java; await() parks with LockSupport.park. The jdk-14-ga source has the ManagedBlocker version, and jdk-25-ga still does, RejectedExecutionException fallback included. That change came with Doug Lea's AQS refresh, JDK-8229442, which the related CSR places in JDK 14.

CountDownLatch.await() never goes near a Condition. It's a shared acquire on AQS that parks the thread. No managedBlock, no spare, on any version I looked at. javap of the JDK 21 AbstractQueuedSynchronizer class itself (not its ConditionObject) has no managedBlock call at all, so Semaphore.acquire() and ReentrantLock.lock() park the same way. I didn't lab those two.

CompletableFuture waits through its own Signaller, which implements ManagedBlocker. javap on 21 shows managedBlock in both its untimed and timed get paths, and the JDK 11 lab compensated the same way. It doesn't catch the exception, so it's also the one that blows up at spare 257.

And the 11-vs-21 info result: in jdk-11+28, ForkJoinTask.get() from a non-worker first tries tryExternalHelp(), which unpushes the task from the common pool's submission queue and runs it on the calling thread if no worker has taken it yet. My main thread ran its own task five times out of five. The jdk-21-ga code still has a path that removes and runs a still-queued task (tryRemoveAndExec), but in my 21 runs a worker always got there first. Don't build anything on either behaviour; I mention it because a thread name of main in a log line from "pool code" sent me looking for a bug that wasn't there.

The "since" tags from JDK 19 onward come from the JDK 25 Javadoc.

The common pool is shared, and that's the trap

Every parallel stream in your JVM and every CompletableFuture.supplyAsync without an executor lands in the same pool, including the ones inside your libraries. One worker on a 2-CPU pod. Three on a 4-CPU pod. Your code doesn't own it, and neither does anything else.

The patterns that hurt, roughly in the order I find them:

supplyAsync(() -> httpClient.send(...)) with no executor. Each call blocks a common-pool worker for the length of the call. No compensation, because socket I/O on a platform thread isn't a managed block. A single slow downstream now delays every parallel stream in the process. One wrinkle: when the common pool's parallelism is below 2 (any 2-CPU machine), CompletableFuture's async methods don't use the common pool at all; they start a new thread per task. The daemon threads article shows what that does to JVM exit. So the same code behaves differently on a 2-CPU pod and an 8-CPU node.

future.join() inside a common-pool task, at scale. This is the one that surprised me most. It works at low volume because the pool compensates. Then a burst arrives, the 257th waiter can't get a spare, and the task fails with RejectedExecutionException: Thread limit exceeded replacing blocked worker. Your error logs show a rejection from a pool you never configured. Before that point you also had 260 threads where you thought you had 3.

CountDownLatch.await() or Semaphore.acquire() in a pool task. No compensation. Enough of them and every worker is parked waiting for work that's queued behind them. That's pool-induced starvation, and the starvation article walks through what it looks like in a dump.

Treating the steal count as a health metric. In step 5, two steals meant perfect balance. A high steal count often means your tasks are too small and workers are fighting over crumbs. A low one tells you nothing on its own.

Using ForkJoinPool as a general executor because it "is faster". For independent, non-splitting tasks that block, a ThreadPoolExecutor with a bounded queue gives you back-pressure and a rejection policy you chose. The fork/join pool's queues grow until they hit a capacity in the millions (javap on 21 shows a 16777216 limit and a Queue capacity exceeded message in WorkQueue), which in practice means unbounded.

Keeping blocking out of ForkJoinPool, and what each choice costs

The order I'd try things in:

How I'd write the FX enrichment today, on 21:

import java.util.List;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Semaphore;
import java.util.stream.Collectors;

final class FxEnricher implements AutoCloseable {
    private final ExecutorService io = Executors.newVirtualThreadPerTaskExecutor(); // JDK 21+
    private final Semaphore inFlight = new Semaphore(50);   // the FX service's real concurrency limit
    private final FxClient client;                            // blocking client, your own type

    FxEnricher(FxClient client) { this.client = client; }

    List<Double> rates(List<String> pairs) {
        List<CompletableFuture<Double>> fs = pairs.stream()
                .map(p -> CompletableFuture.supplyAsync(() -> lookup(p), io)) // never the common pool
                .collect(Collectors.toList());
        return fs.stream().map(CompletableFuture::join).collect(Collectors.toList()); // join on the caller
    }

    private double lookup(String pair) {
        try {
            inFlight.acquire();
            try {
                return client.rate(pair);
            } finally {
                inFlight.release();
            }
        } catch (InterruptedException e) {
            Thread.currentThread().interrupt();                 // restore the flag, then fail the stage
            throw new IllegalStateException("interrupted looking up " + pair, e);
        }
    }

    @Override public void close() { io.close(); }               // waits for in-flight lookups
}

interface FxClient { double rate(String pair) throws InterruptedException; }

The join() happens on the request thread, not inside a pool task, so there's nothing to compensate for. The semaphore is there because "unlimited virtual threads" turns straight into "unlimited calls to the FX service"; the Semaphore article has the numbers. On 17 or earlier, swap the virtual-thread executor for a named, bounded ThreadPoolExecutor.

What isn't a fix: raising common.parallelism to 64 because things are slow. You've made every parallel stream in the JVM contend for 2 CPUs with 64 threads. Raising maximumSpares to 10,000 turns an exception into a thread explosion. And wrapping CountDownLatch.await() in managedBlock works, but it's patching the symptom; the real problem is waiting inside a pool built not to wait.

If you only take one thing from the most-of-two-days I spent here, make it this: decide which pool each *Async call runs in, explicitly, every time.

What to watch, and what changes when you upgrade

Pool size versus parallelism. ForkJoinPool.commonPool().getPoolSize() above getCommonPoolParallelism() means compensation is happening right now, which means some code is blocking in the pool. Export both as gauges. A pool size sitting near parallelism + 256 is a page; you're one burst away from rejections. getQueuedSubmissionCount() growing for minutes is the uncompensated case: workers stuck, submissions piling up.

Thread dumps. Take three, a few seconds apart, and grep for commonPool-worker. ForkJoinPool.managedBlock / compensatedBlock frames mean compensated waits. Workers in TIMED_WAITING (sleeping), socket reads, or CountDownLatch$Sync parks under ForkJoinTask.doExec mean uncompensated ones. A dozen workers named commonPool-worker-37 and up on a 4-CPU box is the spare pool in action. The thread states article covers reading the state lines.

The test that would have caught it. Run your fan-out code under -Djava.util.concurrent.ForkJoinPool.common.parallelism=1 and with a fake downstream that sleeps 200 ms. If the wall time scales with the number of calls, something is blocking in the common pool. Then run it with 300 concurrent requests and look for Thread limit exceeded in the output.

Across versions.

• JDK 11 (lab): CompletableFuture.join()/get() and explicit managedBlock compensate. BlockingQueue.take() and every other Condition wait don't. I'd expect JDK 8 to match; I didn't run it.
• JDK 14 onward, including 17, 21 and 25 (by source): untimed Condition.await() compensates too, and falls back quietly at the spare limit. Timed waits (poll(timeout), awaitNanos) still don't.
• Moving 11 to 17/21: code that blocked on take() in the common pool may suddenly create dozens of extra threads. Faster, and a surprise on your thread-count dashboard.
• JDK 25: the pool can now schedule delayed tasks itself. I haven't run 25; re-run steps 3 and 4 there before trusting this list.

Before you reach for the common pool again

• The common pool on a 2-CPU machine has one worker. Size assumptions from your laptop don't survive the pod.
• Work-stealing earns its keep on uneven, splittable CPU work: 1.7x over a static split on 2 cores in my lab, 2.5x to 3.1x on 4.
• CompletableFuture.join() and, from JDK 14, take() get spare threads. Thread.sleep and CountDownLatch.await() don't.
• Spares stop at 256. After that, CompletableFuture.join() inside the pool throws Thread limit exceeded replacing blocked worker.
• Pass an executor to every *Async call that does I/O. Leave the common pool to code that computes.

Questions people search about ForkJoinPool

What is the default parallelism of ForkJoinPool.commonPool()? Runtime.getRuntime().availableProcessors() - 1, with a minimum of 1, unless you set -Djava.util.concurrent.ForkJoinPool.common.parallelism. On a 2-CPU container that's one worker. A pool you create with new ForkJoinPool() defaults to availableProcessors() instead.

Can I block inside a ForkJoinPool task? You can, but the pool only replaces the blocked worker when the wait goes through ForkJoinPool.managedBlock, which CompletableFuture does and, since JDK 14, untimed Condition.await() does. Other blocking holds a worker for its full duration. For I/O, use a separate executor or virtual threads (JDK 21+).

Is ForkJoinPool faster than ExecutorService? For recursive, CPU-bound work with uneven pieces, usually yes, because idle workers steal. For independent blocking tasks, no: a ThreadPoolExecutor gives you bounded queues and a rejection policy, and the fork/join pool gives you neither.

Are ForkJoinPool worker threads daemon threads? Yes, all of them, which is documented in the JDK 25 Javadoc and matched my lab on 11 and 21. If main returns while common-pool work is still running, that work is abandoned.