> ## Content Index > Fetch the complete content index at: https://www.ggorantala.dev/llms.txt > Use this file to discover other available public pages before exploring further. # CodeCache Is Full: Why Java Stops Compiling Hot Code - URL: https://www.ggorantala.dev/codecache-full-tuning/ - Published: 2026-11-06T11:37:00.000Z - Updated: 2026-11-06T11:37:42.000Z - Description: CodeCache is full. Compiler has been disabled. What the warning means on JDK 21, the throughput cliff I measured after it, and how to size the code cache. - Author: Gopi Gorantala - Tags: java-memory-management, jit-compiler, jvm-tuning ```log OpenJDK 64-Bit Server VM warning: CodeCache is full. Compiler has been disabled. OpenJDK 64-Bit Server VM warning: Try increasing the code cache size using -XX:ReservedCodeCacheSize= ``` Two lines on stderr and no stack trace. Nothing pages anyone. The service keeps answering requests. It answers them about 40% slower than it did an hour ago, and it'll stay that way until the next restart. That's why I like this failure and hate it at the same time. The JVM doesn't crash. It quietly stops turning new hot code into machine code, and everything that hadn't been compiled yet runs in the interpreter for good. If you only watch error rates, you'll never see it. If you grep the logs for `CodeCache is full`, you'll find it at 3 a.m. on a Tuesday, eleven days after the deploy that caused it. Here's what I measured on JDK 21, what the cliff looks like on a graph, and how I'd size the thing so it stops being a surprise. I thought it'd be a half-day lab. The first two versions of it never produced the warning at all, which turned out to be the most useful result. ## What the warning really says The code cache is the memory region where HotSpot keeps the machine code its JIT compilers produce, plus the interpreter and a few stubs. It has a fixed reserved size. When the compilers need room for a new compiled method and there's no contiguous free space, the JVM prints the warning above and *turns the compilers off*. Methods that are already compiled keep running. Methods that weren't, stay interpreted. In my lab the compiler never came back on once it stopped (more on that below). Two things people get wrong. First, it isn't a memory leak and it isn't `OutOfMemoryError`; the heap is irrelevant. Second, "full" is per-region when the cache is segmented. On JDK 21 with default settings the cache is split into three heaps (non-method code, profiled methods, non-profiled methods), and one of them can be full while the total looks half empty. The warning text changes when that happens, and so does the flag you need. If you haven't read the layout in my [native memory tracking post](https://ggorantala.dev/jvm-native-memory-tracking/?ref=ggorantala.dev), that's where the code cache sits among the JVM's other non-heap regions. This article only cares about one question: is the compiler still running? ## A fraud-rules engine that compiles its own classes A real way to fill a code cache is to generate code at runtime. Picture a fraud-rules engine that compiles every customer's rule set into its own class, and a few thousand customers whose rules all get hot. That's the lab. What you need: OpenJDK 21.0.12.1 (the Ubuntu build, `21.0.12.1+1-1-24.04.4-Ubuntu`) and nothing else. No JMH, no external libraries, since the JDK includes the compiler API. My box is a 2-vCPU Intel Xeon at 2.10 GHz, 8 GB, kernel 6.18, no container limits. Small. Relative comparisons only; I'll say what might not transfer. `Gen.java` writes N classes, each with one large scoring method, and compiles them once with `javax.tools`: ```java import javax.tools.*; import java.io.*; import java.nio.file.*; import java.util.*; public class Gen { public static void main(String[] a) throws Exception { int n = Integer.parseInt(a[0]), m = Integer.parseInt(a[1]); // classes, statements per method Path d = Paths.get("gen"); Files.createDirectories(d); List args = new ArrayList<>(List.of("-d", "gen", "-nowarn")); for (int i = 0; i < n; i++) { StringBuilder sb = new StringBuilder("public class G" + i + " implements java.util.function.IntUnaryOperator { public int applyAsInt(int x){ int h=x;\n"); for (int j = 0; j < m; j++) sb.append("h = h*31 + (h>>>" + (j % 7 + 1) + ") ^ " + (i * 131 + j * 17 + 1) + ";\n"); sb.append("return h; } }"); Path f = d.resolve("G" + i + ".java"); Files.writeString(f, sb.toString()); args.add(f.toString()); } System.out.println("rc=" + ToolProvider.getSystemJavaCompiler().run(null, null, null, args.toArray(new String[0]))); } } ``` `Run.java` loads them and calls them round-robin, printing calls per second once a second. For the first 15 seconds it only uses the first half of the classes. Then it switches to all of them. That switch is the point: it models a service whose hot code changes after the first minute. ```java import java.net.*; import java.nio.file.*; import java.util.function.IntUnaryOperator; public class Run { public static void main(String[] a) throws Exception { int n = Integer.parseInt(a[0]), secs = Integer.parseInt(a[1]), phase = Integer.parseInt(a[2]); URLClassLoader cl = new URLClassLoader(new URL[]{Paths.get("gen").toUri().toURL()}); IntUnaryOperator[] ops = new IntUnaryOperator[n]; for (int i = 0; i < n; i++) ops[i] = (IntUnaryOperator) cl.loadClass("G" + i).getDeclaredConstructor().newInstance(); int half = n / 2, sink = 0, k = 0; long t0 = System.nanoTime(), end = t0 + secs * 1_000_000_000L, lastT = t0, calls = 0, lastCalls = 0; StringBuilder out = new StringBuilder(); while (System.nanoTime() < end) { int lim = ((System.nanoTime() - t0) / 1_000_000_000L < phase) ? half : n; // hot set grows at t=phase for (int r = 0; r < 2000; r++) { sink += ops[k].applyAsInt(sink); if (++k >= lim) k = 0; } calls += 2000; long now = System.nanoTime(); if (now - lastT >= 1_000_000_000L) { out.append(String.format("t=%2ds %,.0f calls/s%n", (now - t0) / 1_000_000_000L, (calls - lastCalls) * 1e9 / (now - lastT))); lastT = now; lastCalls = calls; } } System.out.print(out); System.out.println("total calls/s " + String.format("%,.0f", calls * 1e9 / (System.nanoTime() - t0)) + " sink=" + sink); } } ``` Steps, each a literal command: ```bash # 1. generate 1,500 classes with 400-statement methods java Gen 1500 400 # 2. one run: 40 s, hot set grows at t=15 s; warnings to a file, cache summary at exit java -XX:ReservedCodeCacheSize=8m -XX:-UseCodeCacheFlushing \ -Xlog:codecache=warning:file=run.ulog -XX:+PrintCodeCache Run 1500 40 15 # 3. while it runs, from another shell jcmd Compiler.codecache ``` Expected shape: a per-second line that drops at t=15 for every configuration, then recovers for the healthy ones and stays low for the ones that ran out. The `-XX:+PrintCodeCache` summary at exit ends with a `Compilation:` line that says `enabled` or `disabled (not enough contiguous free space left)`. That one line is the diagnosis. Forcing lever: 8 MB is absurdly small. The default on this JVM is 240 MB. I shrank it so the failure shows up in 25 seconds instead of in production in 11 days. The mechanism is the same; the time scale isn't. Clean up afterwards: `rm -rf gen run.ulog`. Nothing else is left running. ## The cliff, and the two versions of the lab that didn't have one My first generator made 600 classes with 60-statement methods. Reserved cache of 3 MB, and the run filled it to 2,866 KB of 3,072 KB with `full_count=0` and the compiler enabled. I bumped it to 4,000 small classes at 4 MB: `full_count=0`, `max_used=3,860 KB`, `used=1,608 KB` at exit. Used went *down*. I turned off `-XX:+UseCodeCacheFlushing` and it was still `full_count=0`. I'd assumed that a cache smaller than the code I asked it to hold would fill up. It doesn't. The JVM evicts cold compiled methods, and `-Xlog:codecache=info` shows why: lines like `Triggering threshold (10.775%) GC due to allocating 28.165% since last unloading`. The code cache can trigger its own unloading pass. If your hot set is big but your code is *small and cold-ish*, you get churn instead of a full cache. To get a real full cache, every method has to be big and hot at the same time. That's what the 400-statement version does. A fixed 15 seconds of the first half, then all 1,500 classes, three interleaved runs per configuration, 40 seconds each: | Config | Reserved | Flushing | Whole run (k calls/s) | t=5–15 s | t=30–39 s | Warning at | | ------ | ---------------- | -------- | --------------------- | --------------- | --------------- | -------------------- | | A | default (240 MB) | on | 414 / 426 / 426 | 482 / 492 / 483 | 507 / 492 / 499 | none | | B | 8 MB | on | 283 / 292 / 297 | 321 / 330 / 336 | 275 / 289 / 288 | 37.6 / 36.8 / 36.4 s | | C | 8 MB | off | 252 / 264 / 266 | 278 / 287 / 288 | 248 / 257 / 258 | 24.3 / 22.1 / 25.4 s | | D | 32 MB | off | 385 / 390 / 396 | 485 / 488 / 497 | 394 / 400 / 413 | none | | E | 8 MB, C1 only | off | 390 / 397 / 401 | 506 / 515 / 519 | 358 / 365 / 361 | 5.0 / 4.9 / 4.5 s | *OpenJDK 21.0.12.1, G1 default, 2 vCPU Xeon 2.10 GHz, 8 GB, no container limits, generator and measured code in the same JVM. 3 runs per row, listed in run order, interleaved. A to B to C to D to E, repeated three times.* Read it like this. Run-to-run spread inside a row is about 3 to 8%, so A's 414 versus 426 is noise and B versus A is not. C, the textbook case (flushing off, small cache), is the worst row: roughly 40% below A for the whole run. B kept flushing on and still hit the warning, later, at around 37 seconds, and was already well under A before it did. I don't have a decomposition of B's early slowness. Eviction and recompilation churn is the obvious suspect; I didn't count recompilations, so that's a guess. Here's the per-second view of one C run around the switch at t=15\. Every row dips, because 750 new classes arrive cold: ```log t=14s 295,861 calls/s t=15s 284,762 calls/s t=16s 250,868 calls/s t=17s 239,457 calls/s t=19s 211,802 calls/s t=22s 230,701 calls/s ... never recovers; warning had printed at 24.294 s ``` In A, the same dip lasted about eleven seconds and then throughput climbed back to roughly 500k as the compilers caught up. In C there's no climb. That plateau is the cliff. Config E is the odd one, and I'll come back to it in the flags section. It's the *fastest* row for the first 15 seconds and the warning prints within five seconds of starting. ## How to tell CodeCache is full and not something else Four ways, in the order I'd try them. All of them work on JDK 21; check the flag spellings if you're on something much newer or older. **The log line.** Grep for `CodeCache is full` or `CodeHeap '` plus `is full` (the segmented variant). The unified-logging version looks like this, with a timestamp: ```log [24.294s][warning][codecache] CodeCache is full. Compiler has been disabled. [24.294s][warning][codecache] Try increasing the code cache size using -XX:ReservedCodeCacheSize= ``` Notice the message appears *twice* in a default run: once as the unified-logging line and once as the old `OpenJDK 64-Bit Server VM warning:` form on stderr. If you're shipping stderr and the JVM log to different places, you'll see it in both. **`jcmd Compiler.codecache`.** The last line is the one to read. This is from a live C-config process at t=38 s: ```log CodeCache: size=8192Kb used=7453Kb max_used=8041Kb free=738Kb total_blobs=1528, nmethods=1133, adapters=302, full_count=1 Compilation: disabled (not enough contiguous free space left), stopped_count=0, restarted_count=0 ``` `full_count=1` and `Compilation: disabled` are the whole story. Note `free=738Kb`. A cache can say "free" and still not have a contiguous block big enough for the next nmethod. Don't read free space as headroom. **`-XX:+PrintCodeCache` at exit.** Same data, once, at shutdown. Useful in a canary where you can't attach. **JFR.** A default recording contains `jdk.CodeCacheFull`. In one recording the event looked like this (trimmed): ```log jdk.CodeCacheFull { startTime = 02:00:01.372 entryCount = 2678 methodCount = 2221 adaptorCount = 360 unallocatedCapacity = 35.3 kB fullCount = 1 codeCacheMaxCapacity = 8.0 MB eventThread = "C2 CompilerThread0" } ``` The `eventThread` is the compiler thread that failed to allocate. That's a nice detail when someone asks which compiler hit the wall. The `codeBlobType` field said `"Unused"` in my recording; I don't know why and didn't chase it. Set up the recording and thresholds as in my [JFR profiling write-up](https://ggorantala.dev/java-flight-recorder-profiling/?ref=ggorantala.dev). ### The segmented variant If you size the cache above the segmentation cut-off, the message names the heap. I forced it with explicit sizes on JDK 21: ```bash java -XX:+SegmentedCodeCache -XX:-UseCodeCacheFlushing -XX:ReservedCodeCacheSize=24m \ -XX:ProfiledCodeHeapSize=6m -XX:NonProfiledCodeHeapSize=10m -XX:+PrintCodeCache Run 1500 45 15 ``` ```log [10.485s][warning][codecache] CodeHeap 'non-profiled nmethods' is full. Compiler has been disabled. [10.485s][warning][codecache] Try increasing the code heap size using -XX:NonProfiledCodeHeapSize= CodeHeap 'non-profiled nmethods': size=10240Kb used=10230Kb max_used=10230Kb free=9Kb CodeHeap 'profiled nmethods': size=6144Kb used=6135Kb max_used=6135Kb free=8Kb CodeHeap 'non-nmethods': size=8192Kb used=1382Kb max_used=1382Kb free=6809Kb CodeCache: size=24576Kb, used=17747Kb, max_used=17747Kb, free=6826Kb ``` Almost 7 MB free in total and the compiler is off, because the free space is in the wrong heap. If you follow a blog's advice to raise `ReservedCodeCacheSize` and the message names `NonProfiledCodeHeapSize`, you'll need to raise that one (or leave the sizes ergonomic, as below). ### What the defaults are here Quoted from `java -XX:+PrintFlagsFinal -version` on this box (2 CPUs, 8 GB): ```log uintx ReservedCodeCacheSize = 251658240 {pd product} {ergonomic} (240 MB) uintx InitialCodeCacheSize = 2555904 {pd product} {default} uintx CodeCacheExpansionSize = 65536 {pd product} {default} bool SegmentedCodeCache = true {product} {ergonomic} uintx NonNMethodCodeHeapSize = 5826188 {product} {ergonomic} uintx ProfiledCodeHeapSize = 122916026 {product} {ergonomic} uintx NonProfiledCodeHeapSize = 122916026 {product} {ergonomic} bool UseCodeCacheFlushing = true {product} {default} ``` With the default config A, the 1,500-class run used 33,652 KB of 245,760 KB. That's 14% of the cache. When I set `ReservedCodeCacheSize=8m`, the JVM chose a single unsegmented `CodeCache` on its own; with `-XX:+SegmentedCodeCache` forced at 24 MB I got three heaps. I haven't pinned down the exact size where ergonomics flips, so I won't quote one. ## Why it doesn't free itself When the compiler can't allocate, it stops; the stop isn't automatically reversed just because some space frees up later. In every one of the nine sweep runs that hit the warning, `restarted_count` stayed at 0, and in an earlier run `used` went *down* from 8,130 KB to 7,563 KB after the stop without compilation coming back (free space exists, but not in a block the compiler could use). The cache does try to restart compilation after an unloading pass in JDK 21's code (I read `CodeCache::flush_unlinked_nmethods` at `jdk-21+35`); I never saw it succeed. The JIT mechanics belong to a sibling post: the tier mechanics are in the JVM series, and I'll link it when it's published. ## The flags people paste in **`-XX:ReservedCodeCacheSize=512m` "just in case".** Cheap, and I measured it. Reserved is virtual address space; committed memory grows in 64 KB steps (`CodeCacheExpansionSize`). Same 1,500-class workload, RSS read from `/proc` at 18 s, two runs each: | Reserved | RSS (MB), run 1 / run 2 | | -------- | ----------------------- | | 240 MB | 104.5 / 104.2 | | 1 GB | 102.8 / 104.3 | (From VmRSS in kB: 106,984 / 106,724 and 105,268 / 106,816.) No visible cost for a large reservation on this box. That's one workload and one JDK; I didn't test a container with a memory limit that counts virtual reservations differently, so check yours before you raise it everywhere. **`-XX:-UseCodeCacheFlushing`.** I turned it off to force the warning and the cache did fill faster (C hit it at about 24 s versus B's 37 s). Turning it off is how I *produced* the failure. I don't recommend it in production. **`-XX:TieredStopAtLevel=1` to "save code cache".** This is config E, and it is the interesting row. C1-only code is smaller, so it fills slower per method, but my C1-only lab hit the wall in five seconds (the methods are large; the C1 versions are still big), and then ran 358–365k in the last ten seconds. Higher than C's 248–258k, because C1 code is faster than interpreted code and 750 methods were already compiled, but about 28% below A. You pay for C1-only forever in peak speed on everything else. I'd only use it where startup matters more than peak (short-lived processes) and say so; that trade-off belongs to the warm-up post. **Raising it until the message goes away.** D (32 MB, flushing off) never printed the warning and still ended 20% under A in the last ten seconds, with 26,791 KB of 32,768 KB used. I don't know why. The cache wasn't full and the compiler was enabled; maybe thresholds or queueing, maybe my box. I didn't explain it and I won't pretend to. If you see this, don't count "no warning" as "no problem." ## Size it from the peak, then add room Measure first. Run a warmed-up canary under real traffic and read `max_used` (or the `jcmd Compiler.codecache` line) at its peak. For my lab, A's peak was about 33.7 MB. Then size to a multiple of that peak: I'd start at roughly 2x, because a hot set that changes (my phase switch is the toy version) or a new feature can add code you haven't seen yet. Leave segmented sizes ergonomic unless the message names a specific heap. One more reason to prefer headroom over a tight fit: **C2 compile threads compete for CPU with your requests**, and under a CPU quota a burst of compiles can show up as throttling. That's the failure mode in my [CPU throttling write-up](https://ggorantala.dev/kubernetes-cpu-throttling-java/?ref=ggorantala.dev); it is not this one, but the symptoms (slow after a deploy, fine later) rhyme, and the discriminator is whether `Compilation:` says disabled. The decision I'd write down: | What you see | Do | | ---------------------------------------------------------------- | ------------------------------------------------------------------------------- | | Compilation: enabled, full\_count=0, throughput low after deploy | Not this. Check warm-up and CPU throttling first. | | Warning names CodeCache | Raise ReservedCodeCacheSize to \~2x measured peak. | | Warning names one CodeHeap | Raise that heap's flag, or drop the explicit sizes and let ergonomics split it. | | No warning, cache near full, throughput odd | Look at used vs max\_used over time; don't trust "no warning." | The cost of the fix is a larger virtual reservation (nothing visible in my RSS test) and one more number to remember when someone moves to a container. The cost of not fixing it is a service running partly interpreted until the next restart. ## What to watch, and what to redo on a new JDK Alert on the log line (`CodeCache is full` or `CodeHeap '` with `is full`) and, if you export it, on code cache used versus reserved. I'm not giving Micrometer metric names here because I didn't run Micrometer; verify them in its JVM binders docs. Use a threshold you've measured: mine would be well under the point where `max_used` approaches the reserve, say 70% of reserved for a week, then investigate. A regression guard that costs nothing: in your load-test job, run `-XX:+PrintCodeCache` and fail if the last `Compilation:` line says anything but `enabled`. Details in my upcoming CI regression-testing post. Version safety. Everything here is OpenJDK 21.0.12.1\. I did not run JDK 25, or JDK 17 or earlier. What I did read: the `jdk-21+35` source does its cleanup of dead compiled methods in a GC-triggered unloading step rather than a sweeper thread, which is why the log says `Triggering threshold ... GC due to allocating ... since last unloading`. Older JDKs (the sweeper era) flushed differently, so advice about `StartAggressiveSweepingAt` and the sweeper from those years may not describe 21\. That flag still prints in `PrintFlagsFinal` on 21 (default 10); I didn't test what it does. When you move LTS to LTS, re-run the sweep above and compare the `Compilation:` line and `max_used` for the same load; I'd re-measure rather than trust my defaults. ## Before you close the ticket - The warning is the end of compilation, not the end of the service. Throughput falls; nothing else tells you. - Small and cold code gets evicted; big and hot code fills the cache. If you can't reproduce it, your code is probably too small or too cold. - Free space isn't headroom. Contiguous space per heap is. A cache can be 28% empty and off. - Size from a measured peak, with room. A big reservation cost me nothing in RSS on this box. - "No warning" isn't "no problem." One of my configurations lost 20% and never complained. ## Quick answers on the code cache warning **What does "CodeCache is full. Compiler has been disabled." mean in Java?** The JIT couldn't allocate room for another compiled method, so HotSpot turned compilation off. Already-compiled code keeps running; anything uncompiled stays interpreted. On JDK 21 I saw it with `-XX:ReservedCodeCacheSize=8m`, and the compiler never came back on in the nine runs that hit it. **What is the default ReservedCodeCacheSize?** On OpenJDK 21.0.12.1 on my 2-CPU, 8 GB box it printed 251658240 bytes (240 MB), ergonomic, split into three heaps. Defaults can depend on the JDK and flags, so run `java -XX:+PrintFlagsFinal -version | grep CodeCache` on yours. **Does a bigger code cache use more memory?** Not in my test: 240 MB and 1 GB reservations both sat near 105 MB RSS on the same workload, since the cache is committed in small steps as it fills. Check in your own container. **How do I check the code cache of a running JVM?** `jcmd Compiler.codecache`, and read the last line (`Compilation: enabled` or `disabled`). A JFR recording also carries `jdk.CodeCacheFull`.