HotSpot (Temurin) vs. OpenJ9 (Semeru): A Comparative Performance Study in Quarkus
In this blog post, the performance of various Java runtime environments (OpenJ9/Semeru versus Temurin with Leyden AOT) and specific Quarkus configurations is evaluated. The primary focus is placed on startup times, memory usage, and throughput to answer the question if Java and Quarkus can be run with a memory usage closer to that of .NET while still having superior throughput and startup times.
Spoiler: almost, further investigation and tuning with OpenJ9 is required
See https://github.com/itbh-at/quarkus-dotnet-benchmarks for the code, raw results and the reports.
Insights from Runs 11–14
Quarkus 3.35.2 is used across all four runs on the same AMD EPYC-Genoa host (16 vCPUs, Hetzner), each with 7 iterations. Two comparison groups are formed:
- Runs 11 & 14: OpenJ9/Semeru with Shared Class Cache (SCC), before and
after SCC tuning (
-Xscmx128M -Xtune:virtualized). - Runs 12 & 13: Temurin 25 + Quarkus Leyden AOT, with and without Quarkus tree-shaking.
| Run | Purpose |
|---|---|
| 11 | Quarkus 3.35.2 + tree-shaking on IBM Semeru OpenJ9 (SCC, optthruput GC). quarkus-jvm, quarkus-semeru, quarkus-virtual, and quarkus-virtual-semeru are covered. → Report |
| 12 | The same application and configuration, but executed on Temurin 25 + Leyden AOT (SerialGC). This serves as the "HotSpot counterpart" to run 11 — identical tree-shaking, different JVM. → Report |
| 13 | Run 12 is repeated without tree-shaking — a pure Leyden baseline. Purpose: The contribution of tree-shaking is isolated by holding all other variables constant (same JVM, same GC, same heap). → Report · Comparison: tree-shaking on vs. off |
| 14 | Run 11 is repeated with tuned SCC flags (-Xscmx128M, -Xtune:virtualized), which are passed during both recording and runtime. A flaw from run 11 is fixed, where the SCC size was uncapped and the JVM was not informed about the CPU-pinned environment. → Report · Comparison: run 11 vs. run 14 |
Startup is 3× faster with Leyden AOT compared to Semeru SCC
The difference between Leyden and Semeru at startup is the most significant result observed across all four runs:
| Runtime | TTFR (ms) |
|---|---|
| quarkus-leyden (run 12) | 853 |
| quarkus-virtual-leyden (run 12) | 868 |
| quarkus-semeru (run 11/14) | ~3,248 |
| quarkus-jvm Temurin (run 12) | ~2,538 |
The first response is reached in under a second by Leyden's AOT compilation; no warmup cycle is required. Startup is accelerated by Semeru SCC compared to the standard Temurin JVM, but the performance gap to Leyden remains substantial.
Semeru delivers comparable throughput at lower memory cost
OpenJ9/Semeru's Shared Class Cache reduces RSS under load while sustaining throughput on par with Temurin. The difference is consistent across both platform and virtual thread variants (run 11, 7 iterations):
| Runtime | Throughput (rps) | Load RSS (MiB) | Density (rps/MiB) |
|---|---|---|---|
| quarkus-jvm | 9,871 | 351 | 28.1 |
| quarkus-semeru | 9,713 | 334 | 29.1 |
| quarkus-virtual | 10,134 | 306 | 33.1 |
| quarkus-virtual-semeru | 9,958 | 297 | 33.5 |
| dotnet-aspnet-ef | 6,601 | 227.2 | 29.06 |
Semeru trims 17–19 MiB of resident memory per instance while keeping throughput within 2% of the Temurin baseline. The throughput density — requests per second per MiB of RAM — is higher for both Semeru variants. For deployments where instance count is constrained by memory rather than CPU, this translates directly into more capacity per host.
Almost no measurable effect is observed from tree-shaking
Runs 12 and 13 are identical, except tree-shaking is enabled in run 12. The resulting metrics are really close:
| Metric | With tree-shaking (run 12) | Without (run 13) |
|---|---|---|
| quarkus-jvm TTFR | 2,538 ms | 2,636 ms |
| quarkus-leyden TTFR | 853 ms | 853 ms |
| quarkus-virtual throughput | 15,051 rps | 14,832 rps |
| quarkus-jvm load RSS | 449 MiB | 454 MiB |
For an application of this scale, tree-shaking is considered negligible and is not recognized as an effective tuning mechanism - especially when project Leyden comes into play.
Disclaimer: Compairing binary sizes wasn't done. Maybe tree-shaking causes smaller binaries that result in smaller container images.
No significant impact is achieved by SCC tuning
When run 11 (default SCC) is compared to run 14 (tuned SCC), the metrics are
essentially flat. A slight RSS improvement under load is observed for
quarkus-semeru (334 → 315 MiB) in the tuned run, but throughput is also
marginally decreased (9,713 → 9,398 rps). No meaningful advantage is gained from
the tuning flags.
Throughput and memory density are consistently improved by virtual threads
Across all four runs, a clear throughput gain with lower RSS under load is achieved when switching from platform threads to virtual threads:
| Run 12 | Throughput | Load RSS |
|---|---|---|
| quarkus-jvm | 12,369 rps | 449 MiB |
| quarkus-virtual | 15,051 rps | 414 MiB |
| quarkus-leyden | 11,383 rps | 417 MiB |
| quarkus-virtual-leyden | 14,160 rps | 376 MiB |
A 20–25% throughput increase with lower memory consumption is provided by
virtual threads. This is systematically confirmed across all runs by the
load_throughput_density metric (rps/MiB).
Conclusion
Exceptionally good runtime characteristics are achieved when a Quarkus application is run on a JVM with default values. However, memory resources tend to be utilized wastefully by the JVM (in favor of high throughput). Outstanding performance is maintained and memory consumption is significantly reduced by selecting the appropriate JVM for the task and applying command-line optimizations.
An ideal scenario would involve combining OpenJ9 with Leyden. Through this combination, memory usage would be brought closer to the .NET version, startup times would be vastly improved, and superb throughput would be retained. This is considered an optimal solution for containerized workloads when native compilation is not an option. And yes, I know that Leyden isn't possible with OpenJ9 because of the differences in the JVM architecture. I keep on dreaming.
Comments