OutOfMemoryError: Direct buffer memory: Causes and Fixes

java -Xmx2g -XX:+DisableExplicitGC -jar reconciler.jar

That one flag was in the launch script for years. Somebody added it after a System.gc() call in a vendor library caused a 900 ms pause, and it was a reasonable thing to do. Then a nightly job started dying with OutOfMemoryError: Direct buffer memory while the heap graph sat at 10% and every dashboard said the JVM was fine.

Here's what the error looks like on JDK 21 (OpenJDK 21.0.12.1, G1, -Xmx256m -XX:MaxDirectMemorySize=64m, from my lab):

Exception in thread "http-worker-7" java.lang.OutOfMemoryError: Cannot reserve 8388608 bytes of direct buffer memory (allocated: 62914560, limit: 67108864)
	at java.base/java.nio.Bits.reserveMemory(Bits.java:178)
	at java.base/java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:111)
	at java.base/java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:360)
	at Lab.lambda$leak$1(Lab.java:45)
	at java.base/java.lang.Thread.run(Thread.java:1583)

Older JDKs print just java.lang.OutOfMemoryError: Direct buffer memory, no numbers. If that's what your logs say, you're on an older build, and you'll have to work out the limit yourself (it's in section 5). The newer wording came from JDK-8048192; JBS lists backports to 8u491 and 11.0.31, and I confirmed it on 21.0.12.1. I did not check which mainline release got it first, so don't read anything into "JDK 21 and up" from this post.

Both wordings, and the thread that printed them

Look at the thread name in that trace: http-worker-7. It asked for 8 MiB, one buffer, which is nothing. It wasn't the problem. Another thread (batch-loader in my lab) had already staged 60 MiB and never let go, and the request thread was just the one that came along when the counter ran out. This happens constantly with direct buffer errors. Don't go hunting in the code of the thread in the trace until you've looked at who holds the memory.

Also notice the message tells you the whole story if you read it slowly: "allocated: 62914560, limit: 67108864" means 60 MiB reserved out of 64, and "8388608" is the request that didn't fit. Subtract and you know the cap was 4 MiB short. On the old wording you get none of that.

Which counter ran out: heap, direct, or the container?

This error is thrown by Java code, not by the JVM's allocator and not by the kernel. It comes from java.nio.Bits.reserveMemory, a plain bookkeeping check inside the JDK: "total reserved direct bytes + this request > limit?". Nothing has been asked of the OS yet when it fires. That tells you three things:

• Your heap can be nearly empty. The check doesn't look at it.
• The machine can have gigabytes free. The check doesn't look at that either.
• It's a different failure from the kernel killing the process. If the pod restarted with exit code 137 and you saw no Java error at all, that's the kernel (see OOMKilled vs OutOfMemoryError and the exit 137 article).

To check how much direct memory a live JVM is holding, the BufferPoolMXBean named direct gives you count and bytes. From a running process with Native Memory Tracking on (-XX:NativeMemoryTracking=summary at startup, there's no switching it on later):

jcmd <pid> VM.native_memory summary | grep -E "Total|Other"
jcmd <pid> GC.class_histogram | grep -E "DirectByteBuffer|Cleaner"

On my lab holding 60 buffers of 1 MiB each, NMT printed Other (reserved=61440KB, committed=61440KB). In JDK 21, direct buffers land in the "Other" category. And the class histogram showed this:

  23:            60           3840  java.nio.DirectByteBuffer (java.base@21.0.12.1)
  28:            60           2400  jdk.internal.ref.Cleaner (java.base@21.0.12.1)

3,840 bytes of heap for 60 MiB of native memory. That's 64 bytes of heap per megabyte, and it's the entire reason this error is so confusing. A heap dump won't show it, a heap graph won't show it, and the garbage collector sees almost no pressure from it. Keep that number in your head for section 5.

(Quick note on what doesn't fire: with -XX:+HeapDumpOnOutOfMemoryError and -XX:+ExitOnOutOfMemoryError set, my lab run still exited with code 0 and wrote no dump. Both are covered in the heap dump setup and fail-fast posts. If direct buffers are how your service dies, neither flag is saving you.)

Getting the same crash in three minutes

You need JDK 21 and a shell. I ran everything on OpenJDK 21.0.12.1, x86_64, two cores, no container. I had no Docker daemon this session, so none of this ran in a container with a cgroup limit; I'll say where that matters. The lab is one class with a few modes:

import java.lang.management.*;
import java.nio.*;
import java.util.*;

public class Lab {
    static final int MIB = 1024 * 1024;

    public static void main(String[] a) throws Exception {
        switch (a[0]) {
            case "leak" -> leak();
            case "garbage" -> garbage(Integer.parseInt(a[1]));
            case "probe" -> probe();
        }
    }

    // batch loader keeps every staging buffer; the request thread is the one that dies
    static void leak() throws Exception {
        List<ByteBuffer> staged = new ArrayList<>();
        Thread loader = new Thread(() -> {
            for (int i = 0; i < 60; i++)
                staged.add(ByteBuffer.allocateDirect(MIB)); // never released: the leak
        }, "batch-loader");
        loader.start(); loader.join();
        Thread handler = new Thread(() -> {
            ByteBuffer.allocateDirect(8 * MIB); // innocent request
        }, "http-worker-7");
        handler.start(); handler.join();
    }

    // nothing retained: every buffer is garbage as soon as the loop body ends
    static void garbage(int n) {
        long t0 = System.nanoTime();
        int i = 0;
        try {
            for (; i < n; i++) ByteBuffer.allocateDirect(MIB).put(0, (byte) 1);
            System.out.println("ok: " + n + " x 1 MiB, " + (System.nanoTime() - t0) / 1_000_000 + " ms");
        } catch (OutOfMemoryError e) {
            System.out.println("OOM at iteration " + i + " after " + (System.nanoTime() - t0) / 1_000_000 + " ms: " + e.getMessage());
        }
    }

    // keep allocating until it fails; shows what the limit really is
    static void probe() {
        List<ByteBuffer> keep = new ArrayList<>();
        System.out.println("Runtime.maxMemory = " + Runtime.getRuntime().maxMemory() / MIB + " MiB");
        try { while (true) keep.add(ByteBuffer.allocateDirect(MIB)); }
        catch (OutOfMemoryError e) { System.out.println("fit " + keep.size() + " x 1 MiB\n" + e); }
    }
}

Compile it and run the first case:

javac Lab.java
java -Xmx256m -XX:MaxDirectMemorySize=64m Lab leak

You get the trace from the top of the post, in a bit over half a second. The forcing lever is -XX:MaxDirectMemorySize=64m; in production it's whatever you set, or more likely, nothing at all, which brings us to the limit.

What the limit is when you don't set it. Run the probe with a 100 MiB heap and no direct flag:

$ java -Xmx100m Lab probe
Runtime.maxMemory = 100 MiB
fit 100 x 1 MiB
java.lang.OutOfMemoryError: Cannot reserve 1048576 bytes of direct buffer memory (allocated: 104857600, limit: 104857600)

Same with -Xmx256m: 256 buffers fit, limit 268435456. With -XX:MaxDirectMemorySize=32m and the same 256m heap: 32 fit. So by default the cap equals the maximum heap size. -XX:+PrintFlagsFinal shows MaxDirectMemorySize = 0 on this build, and 0 means "use max heap". That's worth knowing because it means setting -Xmx to 4 GB has quietly given you a 4 GB direct-memory budget too, on top of the heap.

Now the one that took me a while.

Nothing retained, still dies. Run the garbage mode, which allocates 2,000 buffers of 1 MiB and drops each one immediately, through a 64 MiB limit:

$ java -Xmx2g -XX:MaxDirectMemorySize=64m Lab garbage 2000
ok: 2000 x 1 MiB through the limit, 472 ms, direct count=16 used=16384 KiB

$ java -Xmx2g -XX:MaxDirectMemorySize=64m -XX:+DisableExplicitGC Lab garbage 2000
OOM at iteration 64 after 542 ms: Cannot reserve 1048576 bytes of direct buffer memory (allocated: 67108864, limit: 67108864)
direct count=64 used=65536 KiB

Same code, same limit, same heap. One flag, and a program with zero leaks dies at the 65th buffer. If you see "OOM at iteration 64" you've reproduced it. If instead you see GC overhead limit exceeded or a Java heap space message, you're running something else; this lab has no heap pressure.

Teardown is trivial: nothing wrote a file. The lab writes no files. Even with -XX:+HeapDumpOnOutOfMemoryError set, I got no .hprof out of it.

Why it broke, the simple version

A direct buffer is a small Java object on the heap pointing at a chunk of native memory. The native chunk is only returned when the little heap object gets garbage-collected. And the little heap object is tiny, so the heap can fill up with ten thousand dead ones and not feel it.

So there are two ways to run out: you're really holding too much (a leak, or too many threads each caching a buffer), or you're holding nothing but the garbage collector hasn't gotten around to collecting the dead handles. The JDK has a safety net for the second case, and -XX:+DisableExplicitGC cuts it.

Reserve, collect, sleep, throw: the 500 ms I could measure

I found the safety net by reading Bits.reserveMemory in the jdk-21+35 source (I read it on GitHub, I can't paste it here, and the line numbers in the trace are from the Ubuntu build, not upstream). The order of operations, as I read it:

1. Try to reserve. If the total stays under the limit, done. This is the normal path and it's cheap: a compare-and-set on a counter.
2. If not, ask the reference-processing machinery whether the cleaner thread is already working through dead buffers, and retry after each answer.
3. Call System.gc() yourself. This is the safety net. The idea: the JDK knows direct memory is pressured and the heap isn't, so it forces a collection to find the dead handles.
4. Retry in a loop with sleeps of 1 ms, then 2, then 4, doubling up to 256 ms, nine sleeps in all. The code comment says roughly 511 ms total.
5. If it still doesn't fit, throw. That's the one OutOfMemoryError in the method.

I measured step 5. In the leak lab, the failing allocation took 520 ms from call to exception, which matches the sleeps. That's a number worth knowing, because it means a service that's out of direct memory doesn't fail fast: every request that needs a buffer parks for half a second before dying. Under load that looks like latency, then thread pool exhaustion, then errors.

Step 3 is where -XX:+DisableExplicitGC bites. That flag turns System.gc() into a no-op. So the JDK calls it, nothing happens, the dead handles sit there, and after the 511 ms of retries it throws. My garbage run proves it from the other side. Without the flag, the GC log shows 31 Pause Full (System.gc()) entries for those 2,000 MiB, one roughly every 64 buffers, each reclaiming the lot:

[0.085s][info][gc] GC(0) Pause Full (System.gc()) 1M->1M(10M) 5.039ms
[0.104s][info][gc] GC(1) Pause Full (System.gc()) 1M->1M(8M) 3.756ms

Those are 3 to 5 ms in my lab because the live set is nothing. On a service with a multi-gigabyte heap, that same System.gc() is a full stop-the-world collection. I did not measure that, but it's what the vendor library's System.gc() was doing when someone added the flag, and it's why the flag exists. The flag fixes the pause and breaks direct buffers. You've traded one for the other without knowing.

Two details I checked. First, a small heap doesn't make it go away: I ran the garbage loop with -Xmx64m -XX:+DisableExplicitGC and got the identical failure at buffer 64, with no collection logged at all. The heap never filled because 64 bytes per buffer is nothing. Second, -XX:+ExplicitGCInvokesConcurrent is fine: the job completed, with the explicit GCs becoming concurrent mark cycles (about 60 of them in that run). The buffers got freed.

The wrong turn I took. My first theory for the garbage case was that the young generation wasn't big enough, so I tried to trigger GC by shrinking the heap. It did nothing. The memory the garbage collector tracks and the memory that's piling up are different pools, and the first one is never under pressure. Once I'd noticed the 64-bytes-per-MiB ratio in the histogram, the rest followed. That ratio, the System.gc() call in Bits.java, and the DisableExplicitGC flag finally lined up, and the whole thing took me most of two days to get straight. I'm writing it down so you don't spend yours.

The other way to fill it: threads. NIO has a hidden consumer of direct memory. When you write a heap ByteBuffer to a channel, the JDK copies it into a temporary direct buffer first, and it caches that temporary buffer per thread so the next write is cheaper. The cache has no limit by default. In my lab, 8 threads each wrote one 16 MiB heap buffer to /dev/null and went idle:

after 8 writers of 16 MiB: direct count=8 used=131072 KiB

That's 128 MiB of direct memory pinned by eight threads that did one write. With -Djdk.nio.maxCachedBufferSize=262144 the same run showed direct count=0 used=0 KiB: buffers over 256 KiB aren't cached. And with 64 such threads under a 256 MiB cap, the unfixed run threw Cannot reserve 16777216 bytes of direct buffer memory (allocated: 268435456, limit: 268435456) in several threads. With the cache capped at 256 KiB and the same 64 writers, nobody failed. If your service writes big heap buffers from a large thread pool, this is a prime suspect and it's invisible in your own code.

From jcmd to the guilty allocation, and the fix

Work out which of the three shapes you have before changing a flag.

1. Holding too much (a real leak or over-sized pool). The direct pool's used grows and doesn't come back after GC. Sample the BufferPoolMXBean (direct and mapped) over time. If it's a staircase, find who allocates. A heap dump won't show the 60 MiB, but it will show DirectByteBuffer instances and what references them: in the histogram, count of java.nio.DirectByteBuffer going up with nobody dropping them. Follow the references to the holder (a list, a map, a queue). The fix is to release them: pool and reuse a fixed set, or stop holding them past the batch.

2. Dead handles not collected (the safety net is off). used is high but the retained DirectByteBuffer count is low, and everything works for hours until a burst. Look for DisableExplicitGC in your flags and in JAVA_TOOL_OPTIONS and your launcher script (the launch line is not the only place). The fix is to remove the flag. If a library's System.gc() was the reason it went in, either replace that library call or use -XX:+ExplicitGCInvokesConcurrent, which kept the cleanup working in my test without a full stop-the-world collection. Don't take my word for the latency win on your heap; measure the pause.

3. Thread-local temp buffers. used scales with the number of threads that have ever done big writes. The fix is -Djdk.nio.maxCachedBufferSize=<bytes> (JDK property, set on the command line), or writing from direct buffers you own so NIO never makes temporaries.

And the fix you shouldn't reach for first: raising -XX:MaxDirectMemorySize. It's legitimate when the live set is real, and I'll say when (next paragraph). It's a bad fix for shapes 1 and 2, because all it does is delay the same error. It's also a native-memory commitment: on a container with a 1 GiB limit, an "-Xmx700m plus 512m direct" configuration can't all be used, and then the kernel kills you instead (that's the OOMKilled article, and I haven't run this one under a real cgroup limit).

Raising it is right when you've measured the pool plateauing: stable used, no growth after GC, peak under, say, 80% of the cap. Then size the cap above the peak and set it explicitly so it isn't silently tied to the heap.

I didn't run Netty in this lab; I'd rather say that than guess. If you use it, its pooled allocator manages its own arenas of direct memory and a separate leak detector (-Dio.netty.leakDetection.level), and a stuck pool shows up as a flat, high used that never falls. That's a Netty-side issue, not a JDK one.

Buffers with an owner, and a number on every pool

The structure that avoids all of this: nothing allocates direct buffers per request. Allocate a fixed set at startup (or use a pool with a hard cap), borrow and return with try/finally, and let the pool's size be the memory budget. A direct buffer that's allocated and dropped on a hot path is a code smell, because the allocation is expensive (the JDK zeroes it) and the release is up to the GC.

// one budget, set up front; borrowers must give it back
final class ExportBuffers {
    private final BlockingQueue<ByteBuffer> free = new ArrayBlockingQueue<>(32);
    ExportBuffers() {
        for (int i = 0; i < 32; i++) free.add(ByteBuffer.allocateDirect(1 << 20)); // 32 MiB, total
    }
    ByteBuffer borrow() throws InterruptedException { ByteBuffer b = free.take(); b.clear(); return b; }
    void giveBack(ByteBuffer b) { free.add(b); }
}

Set -XX:MaxDirectMemorySize explicitly to that budget plus headroom, so the number in your launch script means something, and so it doesn't ride along with -Xmx when someone resizes the heap.

Keeping it from coming back

Alert on the thing the heap graph can't show: BufferPoolMXBean direct used bytes against the cap. In Micrometer that's jvm.buffer.memory.used{id="direct"} (I didn't re-verify the metric name on the current Micrometer; check yours). Alert when it passes about 80% of your MaxDirectMemorySize and stays there. The full alert set is in the alerting post (coming up: java-memory-alerting-before-oom).

Add a soak test (see leak-hunting with heap dumps for the method) that runs with a tight MaxDirectMemorySize and fails the build if the pool grows after warm-up. A 64 MiB cap would have caught the 60 MiB leak above in a minute.

Version safety, plainly: the newer message with allocated and limit I verified only on 21.0.12.1 (JBS says 8u491 and 11.0.31 got it too; I did not run those). The default cap equals max heap, and the System.gc()-and-retry in Bits was what the JDK 21 source showed me; I didn't diff it against 8, 11 or 25. Re-read your own JDK's Bits.reserveMemory if the limit matters to you. If you move to a JDK where jdk.nio.maxCachedBufferSize or the flag defaults changed, java -XX:+PrintFlagsFinal -version | grep -i direct on the new JDK takes five seconds.

I also did not run any of this inside a container or on Alpine, on cgroup v2, or on JDK 25. The numbers here are one machine's.

What to remember

• Direct buffer memory is a JDK counter check, not a kernel or malloc failure. The heap can be empty and the host can have gigabytes free.
• Default limit = max heap. Setting -Xmx set a direct budget too. Set -XX:MaxDirectMemorySize on purpose.
• A failing allocation spends about 511 ms retrying before it throws. Services don't fail fast here.
• If -XX:+DisableExplicitGC is in your flags anywhere, remove it before you debug anything else.
• Thread pools doing large writes pin direct memory per thread. -Djdk.nio.maxCachedBufferSize is the knob.

Questions people type after pasting the error

How do I fix java.lang.OutOfMemoryError: Direct buffer memory? First work out why the pool is full: a real leak, dead handles the GC hasn't collected (check for -XX:+DisableExplicitGC), or per-thread NIO cache. Then fix that cause. Raising -XX:MaxDirectMemorySize is correct only if the pool plateaus under the new cap.

What is the default MaxDirectMemorySize? On JDK 21.0.12.1 the flag shows 0, which means "equal to the max heap size". I measured it: -Xmx100m gave a limit of 104857600 bytes, and -Xmx256m gave 268435456.

Does -Xmx limit direct memory? Only indirectly, by being the default for the cap. Direct memory isn't taken out of the heap; it's on top of it, so your process can use heap plus direct plus everything else. See heap sizing in containers for how that interacts with a container limit.

Why doesn't a heap dump show the problem? Because 60 MiB of buffers is 3,840 bytes of heap in my histogram. The memory is native. Use BufferPoolMXBean or NMT's "Other" category, and see the full OutOfMemoryError catalogue for the other messages.