Summary

Improve startup and warmup time by making optimized native code for an application instantly available when the HotSpot Java Virtual Machine starts. Achieve this by compiling application code to native code in a training run, storing the native code in the AOT cache for use in subsequent production runs. If the workload changes in production, regenerate native code dynamically for continued peak performance, providing the best of both ahead-of-time (AOT) and just-in-time (JIT) compilation.

Goals

• Enable applications to achieve peak performance more quickly.

• Enable applications to sustain peak performance even as workloads change.

• Do not require any change to the code of applications, libraries, or frameworks.

• Do not require any change to the configuration of HotSpot, beyond requesting the use of the AOT cache.

• Continue to support the Serial, Parallel, G1, and ZGC garbage collectors.

• Do not introduce new AOT workflows, but, rather, extend the existing AOT cache creation workflow.

• Ensure that shifting from AOT-compiled code to JIT-compiled code is invisible to applications.

• Support the AArch64 and x64 processor architectures.

Non-Goals

• It is not a goal to provide an AOT-only mode. Applications will use both AOT-compiled code and JIT-compiled code in the same run, automatically transitioning between them as needed.

• It is not a goal to support cross-compilation. Code compiled in a training run must run on the same CPU architecture, with the same set of features, in subsequent production runs.

• It is not a goal to support all CPU architectures currently supported by HotSpot. We expect normal porting activities to eventually add support for all major architectures.

Motivation

When a Java application is run by the HotSpot JVM, it passes through three phases: It starts up, then it warms up, and then it reaches peak performance.

During startup, HotSpot invokes the application's main method and loads, links, and initializes classes on demand. Initially, it runs both application and JDK library code via the bytecode interpreter, which is slow. Inside the interpreter, HotSpot profiles the application's behavior by counting events such as method invocations and loop iterations. It uses the profile data to select frequently-invoked methods, or hot spots, which it compiles to native code via the basic C1 compiler. This native code is only modestly optimized.

During warmup, the application settles into its workload and the loading, linking, and initialization of classes ramps down. HotSpot continues to profile the application, both in the bytecode interpreter and via instrumentation code inserted by C1. It collects richer profile information, including not just method-invocation and loop-iteration counts but also the types of objects encountered. As the profile data accumulates over time, it becomes more statistically useful. Eventually, HotSpot uses the data to select the hottest methods, which it compiles to native code via the advanced C2 compiler. This native code contains no instrumentation and is highly optimized.

Profiling the application and generating native code is not free. Not only is the bytecode interpreter slow, but instrumented native code is slower than uninstrumented native code. Compiling methods to native code requires both CPU time and memory that could otherwise be used by the application, even though HotSpot only compiles methods to native code when profile data indicates that doing so will be worth the effort. Gradually, however, JIT compilation catches up to the application's emerging hot spots and the application runs faster. Eventually, all hot methods are compiled to fully optimized native code and the compilers go idle.

The application remains in this state of peak performance as long as its hot spots do not change. The application's hot spots may change, however, in response to changes in its workload. When that happens, HotSpot can dynamically deoptimize, by discarding previously generated native code as needed, and reoptimize, by generating new native code for newly-hot methods. For example, if an application initially receives two types of requests then HotSpot dynamically optimizes the code for those two request types. If the application starts receiving a third type of request, HotSpot can dynamically deoptimize and then reoptimize the code for all three types of requests. The application can pass, in effect, through another warmup phase, sustaining performance as the application's workload changes.

What about static compilation?

Static compilation has sometimes been proposed as an alternative to the dynamic compilation of Java code. A static compiler converts entire applications to native code ahead-of-time, prior to run time.

Static compilation has some advantages over dynamic compilation. A statically compiled application starts up and reaches peak performance immediately, without a warmup phase. At run time there is no need for a bytecode interpreter, profiling, or compilation. Peak performance can even be competitive with HotSpot if the static compiler's optimization work is guided by accurate profiles gathered during prior runs.

Dynamic compilation, however, has three key advantages over static compilation.

First, dynamic compilation makes applications agile because it responds to changes in the application's hot spots. It deoptimizes and reoptimizes as needed, sustaining performance as the application's workload changes. A statically compiled application cannot respond in this way — by its nature, it can be optimized for only one set of hot spots.

Second, dynamic compilation makes applications portable across varying hardware and software because it generates native code at run time that is specific to the run-time environment. If an application is redeployed on a different processor architecture, a processor with a different feature set, a different operating system, or a different version of the JDK, HotSpot will achieve peak performance for that environment without requiring any change to the application. A statically compiled application must be recompiled in the face of such changes.

Finally, dynamic compilation is compatible with the dynamic nature of the Java Platform. Features such as dynamic class loading, dynamic linkage, dynamic dispatch, and dynamic reflection bring vast expressive power, and have been fundamental to the platform's success. HotSpot handles these features naturally, while static compilers struggle with them. Even heroic amounts of static analysis cannot make up for the fact that these features require many decisions to be made at run time. Implementors of static compilers for Java code have therefore resorted to incompatible constraints, such as closed-world assumptions, and to putting significant burdens on developers, such as having to identify in advance the classes eligible for reflection.

Shifting compilation work to training runs

Throughout the startup and warmup phases, HotSpot continuously juggles multiple balls: It runs application and JDK library code; it loads, links, and initializes classes on demand; it profiles the application's execution; and it compiles hot methods to native code with varying degrees of optimization, guided by the profile data.

The thesis of Project Leyden is that the key to improving startup and warmup time is to do some of this work earlier, ahead of time, rather than just in time. We shift work earlier in time by doing it in a training run, storing the results of the work in the AOT cache for instant use in subsequent production runs.

We shifted class loading and linking work earlier in time via JEP 483, delivered in JDK 24. The AOT cache stores the loaded and linked forms of classes from the training run, thereby improving startup time.

We shifted profiling work earlier in time via JEP 515, delivered in JDK 25. The AOT cache stores the execution profiles of methods invoked in the training run, enabling the C2 compiler to run immediately at the start of production runs, thereby improving warmup time.

These improvements laid the foundation for our ultimate goal, which is to shift compilation and optimization work earlier in time. The AOT cache will store optimized native code compiled in the training run, enabling HotSpot to load that code instantly, rather than having to recompile it at the start of each production run. This will improve both startup and warmup time.

HotSpot will not always use the cached code; if the application's workload changes then HotSpot can, as usual, deoptimize and reoptimize, generating new native code for newly-hot methods in order to sustain performance. Thus Java applications will gain some of the benefits of static compilation while retaining the agility, portability, and compatibility of dynamic compilation.

Description

We extend the existing AOT cache to store optimized native code generated in a training run. Such cached code is known as AOT code. During a production run, a request for optimized code for a method can be fulfilled instantly if matching AOT code is found in the cache. If AOT code is unavailable, incompatible, otherwise unsuitable, or later deoptimized, execution falls back to the existing interpreter and JIT mechanisms. AOT code and JIT code can coexist and are completely interoperable since they are created by the same compilers, C1 and C2.

To create an AOT cache, use the AOTCacheOutput option to do a training run of your application and generate AOT code:

$ java -XX:AOTCacheOutput=app.aot -cp app.jar com.example.App ...

This workflow is unchanged from previous releases. The AOT cache in the file app.aot, however, now contains not just pre-linked classes and profiling data but also AOT code for selected hot methods. Subsequently, in production, you can run the application with the cache:

$ java -XX:AOTCache=app.aot -cp app.jar com.example.App ...

No additional options or settings are required to generate or use AOT code. HotSpot creates AOT code and stores it in the cache by default. It continues to store profile data in the cache as well, to be used for sequencing the loading of AOT code and for guiding the subsequent generation of JIT code.

Performance

To evaluate the startup benefit of AOT code, we ran five benchmark applications built with popular Java frameworks. We ran them on a two-core Linux/x64 system so as to emulate a microservice setting in which the JIT compiler is likely to compete with the application for CPU time, thereby increasing startup time:

Without AOT code, the AOT cache reduces the startup time of these applications by around 50% to 70%; with AOT code, the cache reduces their startup time by around 65% to 80%.

To evaluate the warmup benefit of AOT code, we ran a javac benchmark application which repeatedly compiles the same 50 source files, twenty times, measuring the time required for each iteration:

In each curve, the first iteration shows the startup-time improvement: The AOT cache without AOT code improves startup time by about 30%; adding AOT code brings an additional 45% improvement, for a total of about 75%. Successive iterations show the warmup phase, during which HotSpot compiles the hottest methods: The iteration time tends to decrease and then reach a steady state as the quality of the native code improves and the compilers finish their work. The curve for the AOT cache without AOT code decreases more quickly than the curve for no AOT cache, eventually reaching roughly the same steady state. The curve for the cache with AOT code is already close to the steady state by the fourth iteration. The area between the top curve and the bottom curve represents the total warmup-time improvement.

Information on all these benchmarks, including run instructions and links to source code, is available here.

Differences between AOT code and JIT code

AOT code and JIT code can be different, since training runs and production runs can be different.

One source of differences is the fact that the order in which classes are initialized can differ between training and production runs, especially if the workload differs. A method that accesses a static field or invokes a static method in another class must ensure that the class is initialized. When generating AOT code with C2, HotSpot therefore compiles two versions of such methods: A slow version contains extra code to ensure the initialization of referenced classes, while a fast version does not contain that code and thus can be better optimized. HotSpot uses the slow version initially, and then switches to the fast version once all referenced classes are initialized.

Another source of differences is the fact that a static final field's value can vary from run to run; it might, for example, be initialized with the current date and time. When just-in-time compiling a method that refers to such a field, the class containing the field will have been initialized, so the field's value will be known and C2 can treat it as a compile-time constant, embedding it directly in native code. When ahead-of-time compiling the same method, however, no classes will have been initialized, so the field's value will not be known and C2 cannot treat it as a compile-time constant; it must generate code that explicitly loads the field.

Despite these differences, AOT code still delivers significant performance benefits in a manner that is transparent to applications. As always, at run time the compilers can generate JIT code to replace AOT code that fails to age well.

Consistency of training and production runs

To enjoy the benefits of the AOT cache generated in a training run, the training run and all subsequent production runs must be essentially similar, as described in JEP 483.

If an AOT cache contains AOT code, that code is used when two additional constraints are met:

• All runs use CPUs of the same architecture and with the same features. For example, AOT code generated for an x64 CPU with the AVX-512 vector-instruction feature will not run on an x64 CPU without that feature.

• All runs use the same garbage collector, since AOT code contains GC-specific read/write barriers.

If these constraints are not met then HotSpot issues a warning message and does not load the AOT code, falling back to the usual interpreter and JIT mechanisms. It still uses the other information in the AOT cache, namely the loaded and linked classes and the profiling data. In that case, the application may start up and warm up more slowly, but its execution will still be correct and it will still, eventually, achieve peak performance.

Observing AOT cache usage in production

You can observe whether AOT code is loaded in a production run via the existing HotSpot option PrintCompilation, which now reports on both AOT code loading and JIT compilation:

$ java -XX:+PrintCompilation \
  -XX:AOTCache=app.aot -cp app.jar com.example.App ...

You can also check whether an AOT cache containing AOT code is usable in a particular production environment. The option AOTMode=required causes HotSpot to report an error and exit if the AOT cache violates any constraint:

$ java -XX:AOTMode=required \
  -XX:AOTCache=app.aot -cp app.jar com.example.App ...

(This option was originally AOTMode=on, later renamed for clarity to AOTMode=required.)

Finally, you can disable the loading of AOT code via the diagnostic option AOTCodeCaching:

$ java -XX:+UnlockDiagnosticVMOptions -XX:-AOTCodeCaching \
  -XX:AOTCache=app.aot -cp app.jar com.example.App ...

You can use this option to evaluate the performance effects of AOT code in a production run, or to avoid constraint-violation errors when using AOTMode=required and the AOT code is unusable but the other information in the cache is usable.

Controlling the generation of AOT code in training

We have, thus far, spoken of an AOT cache being created in a training run, in a single step via the AOTCacheOutput option as shown above. In fact, creating an AOT cache takes two distinct steps: HotSpot runs once in record mode to store observations of your application's behavior in an AOT configuration, and then again in create mode to assemble an AOT cache from that configuration, which includes compiling AOT code. Using the AOTCacheOutput option runs the second step transparently, but you can invoke each step explicitly via the AOTMode and AOTConfiguration options:

$ java -XX:AOTMode=record -XX:AOTConfiguration=app.aotconf \
       -cp app.jar com.example.App ...
$ java -XX:AOTMode=create -XX:AOTConfiguration=app.aotconf \
       -XX:AOTCache=app.aot

HotSpot provides many options to control the behavior of its compilers, C1 and C2. These options uniformly govern the generation of both JIT code, at run time, and AOT code, in the assembly step. For example, this command line reports all JIT compilation activity during the training run proper and all AOT compilation activity as the cache is created:

$ java -XX:+PrintCompilation \
  -XX:AOTCacheOutput=app.aot -cp app.jar com.example.App ...

To see AOT compilation activity but not JIT compilation activity, use the existing environment variable JDK_AOT_VM_OPTIONS to pass the PrintCompilation option only to the assembly step:

$ JDK_AOT_VM_OPTIONS='-XX:+PrintCompilation' \
  java -XX:AOTCacheOutput=app.aot -cp app.jar com.example.App ...

Alternatively, you can run the two steps yourself, specifying the PrintCompilation option only in the second step.

Finally, you can disable the creation of AOT code in a training run via the diagnostic option AOTCodeCaching:

$ java -XX:+UnlockDiagnosticVMOptions -XX:-AOTCodeCaching \
  -XX:AOTCacheOutput=app.aot -cp app.jar com.example.App ...

You can use this option to evaluate the size impact of AOT code on the AOT cache — cached code can make the AOT cache significantly larger.

For more information on all of the AOT-related command-line options, see the manual page for the java command.

Future Work

• Investigate minimizing bytecode interpretation and JIT compilation in favor of near-total reliance on AOT code. Initial experiments suggest that minimizing interpreter use results in overly large AOT cache files, which can take more time to load than just running the interpreter. Likewise, minimizing JIT compilation often leads to lower peak performance. This approach may have limited applicability if, like static compilation, it fails to live up to user expectations.

• HotSpot has a wide range of fine-grained options for controlling its compilers. Based on experience with this feature, tune the default values of existing options to apply more gracefully to AOT code. Also consider defining new options, such as options for explicit AOT cache size management.

• Consider an option that would enable giving up some performance, or accepting larger AOT cache files, or both, in order to gain portability across processors of the same architecture but with different feature sets.

Testing

• We will create new unit tests for this feature. They will ensure that AOT code, if present, behaves correctly.

• We will run existing AOT cache tests with this feature enabled and ensure that they pass.

• Initially, only AArch64 and x64 processors will be supported. Unit tests will be adjusted appropriately to allow for the absence of AOT code on other architectures.

Risks and Assumptions

• There are no new risks beyond those already noted in JEP 483.

• We assume that HotSpot's organizing principle is still sound: A Java application should be compiled, at run time, to favor the application's actual behavior, exploiting all of the processor features available. Because of this principle, Java code automatically runs better on new hardware.

• The base assumption of the AOT cache also remains operative: A training run is assumed to be a good source of observations that, when passed through an AOT cache to a production run, will benefit the performance of the production run. This assumption applies fully to AOT code, which benefits similar production runs while not doing harm to divergent production runs, which can use JIT compilation to generate different code.