Quarkus has great performance – and we have new evidence

[giscus[bot]]

Quarkus has great performance – and we have new evidence

Quarkus: Supersonic Subatomic Java

https://quarkus.io/blog/new-benchmarks/

[krzysztof-miksa]

Regarding parity principle, specifically:

"If a change is made that changes the architecture of an application (i.e. moving blocking to reactive, using virtual threads, etc), then these changes should be applied to all the versions of the applications"

which approach do you acctually measure? Reactive, blocking or both of them?

[holly-cummins]

You can see every 'variation' that's measured on https://quarkus.io/benchmarks/spring-quarkus-perf-comparison/2026-02-05_12-30-36/metrics-ootb-composite-for-all-dark.svg. In terms of your question, there are two main variations that affect the coding style:

• "default", which is a blocking programming model – see, for example, https://github.com/quarkusio/spring-quarkus-perf-comparison/tree/main/quarkus3
• "with virtual threads enabled"

We don't currently measure anything using a reactive programming style, just because every combination we add means each run takes more processing time in the lab, and at some point too many variations becomes overwhelming.

[edeandrea]

I'd also say that the use case of building a CRUD application using a relational database with a reactive paradigm is very uncommon in practice, so its not a use case we're testing with this set up.

[cescoffier]

@edeandrea I think you mean reactive with a blocking database driver (JDBC, Hibernate ORM classic).

[edeandrea]

Yes that's exactly what I mean. Thank you for clarifying!

[vsevel]

hello. interesting. great job. do you publish the characteristics of your infrastructure, beside "4 cpus"?

[edeandrea]

It's bare metal. I don't think it would matter if it were a vm. The resources allocated are the resources allocated.

[vsevel]

out of curiosity, how do you limit tthe java process to 4 cpus? using cgroup quotas? with taskset?
I suppose the answer is here?

I had an idea about a different type of visual representation. in the early days you had expressed efficiency as the throughput per Mb of RAM. I liked the fact that we did not stop just at throughput, but took into account the associated cost.
so one representation could be to show all the different runs with their associated configs as dots in a single 2 dimensional table with throughput on x-axis, and memory on y-axis. the top efficiency would be at the bottom right, and the minimal efficiency would be at the top left. so you could pick if you want absolute throughput (on the far right), or absolute memory savings (bottom), or something balanced. there could some additional info on each dot to show their rank in terms of efficiency (as defined by throughput/memory), at least show the winner.
that way we could see everything at once. native vs jvm, spring 3 vs spring 4 vs quarkus, virtual threads vs blocking vs reactive, in the future different versions of quarkus, ...

[vsevel]

here is a quick attempt with nano banana. it can be improved but you see the idea:

[franz1981]

out of curiosity, how do you limit tthe java process to 4 cpus? using cgroup quotas? with taskset?
I suppose the answer is here?

Yep, we're using taskset which doesn't have NUMA-awareness: currently we have not installed numactl on the perf lab or I would have preferred to use such instead, as it allows to constrain where memory is allocated, but we added the JVM arg UseNUMA to mitigate this lack of control.
In an ideal world cgroug could be better, but not with quota despite been more realistic for the container use case: the cpu throttling can create a pretty severe noise which won't improve the quality of data observed.
As far as I understand, the purpose of the benchmark is to show the relative performance diff between the 2 frameworks, in the best conditions for both, including having enough CPU time available, for both.
That said, is still possible to modify the scripts and use other mechanisms to observe how the 2 frameworks behave with this additional constraint

[holly-cummins]

here is a quick attempt with nano banana. it can be improved but you see the idea:

That makes a lot of sense, I like it! Do you mind raising a "new visualisation" issue over on https://github.com/quarkusio/benchmarks?

[Adilbek-Akhmet]

Could you provide benchmarks comparing Reactive SQL Client vs. Standard JDBC when running on Virtual Threads?

[edeandrea]

Hi @Adilbek-Akhmet at this time we don't have a test that uses reactive SQL clients. There are a number of reasons for this:

1. We're attempting to look at use cases we often find "in the wild." Using reactive clients with relational databases isn't a use case we come across very often.
2. The way Spring & Quarkus implement reactive SQL clients is very different. Quarkus uses Hibernate Reactive, which is a fully-fledged ORM, whereas Spring uses Spring Data R2DBC, which is not a fully-fledged ORM. It's a mapping layer on top of a reactive driver. Comparing them against each other would not be a fair comparison.
   • A way to get a closer comparison would be to have Quarkus use the Vert.x reactive drivers, but in that situation it wouldn't be a fair comparison either, since Spring would still be using a mapping layer on top if it.

[yuzihao99]

• A way to get a closer comparison would be to have Quarkus use the Vert.x reactive drivers, but in that situation it wouldn't be a fair comparison either, since Spring would still be using a mapping layer on top if it.

Vert.x SQL Client Templates provide object mapping functionality and can be used in Quarkus, but it requires some tricks.
https://vertx.io/docs/vertx-sql-client-templates/java/

[vsevel]

it would be interesting to have the response time also (e.g. min, p50, avg, p95, p99, max). usually there is a tradeoff between throughput and response time.

[franz1981]

@vsevel with "those charts" you mean

?

Because If the TPS is "sustainable" and there is no saturation, the response time is dominated by:

1. RTT to the DBMS (which is the same across frameworks)
2. Framework overhead back and forth

So, it really depends by the ratio between these twos timings.
e.g. If the DMBS RTT is much higher than the time spent in the framework, the two frameworks will likely have similar response times.
But...CPU usage, for the lighter one, will be much lower

[vsevel]

with "those charts" you mean

yes. thanks for the answer. it would be interesting to have a measure of the framework overhead

[wfouche]

yes. thanks for the answer. it would be interesting to have a measure of the framework overhead

@vsevel , this is the CPU usage difference between Quarkus and Spring at 10,000 RPS in the benchmarks that I've ran.

Total Elapsed Time:

• Warmed-up for 90 seconds.
• Benchmark run for 5 minutes.

Total CPU usage measured during the 5 minute benchmark:

Quarkus 3 -> 6 minutes and 58.2 seconds
Spring 3 -> 8 minutes and 0.7 seconds
Spring 4 -> 8 minutes and 1.6 seconds

| Quarkus - Spring | = +/- 64 seconds of CPU time.

• Quarkus uses 15% fewer CPU resources than Spring at the same 10,000 RPS rate.

I'll see where I can upload the benchmark reports to, and will share a link here.

[franz1981]

Thanks @wfouche
Now the interesting thing is that for sustained load if quarkus go idle more frequently than Spring, the former will pay the tax of been idle/woken up more frequently, which is why you have "just" 15% saving.
If I would be an SRE I would size differently Quarkus with less CPU cores to save this to happen, and this instance consolidation (which usually led to use less and cheaper instances on the cloud) translates in much more saving than 15% in my xp

[vsevel]

without the latency shown, one could argue that it does not matter that quarkus has much better throughput, if it has a much worse response time for instance. it depends what you are looking for. the server team will try to maximize the throughput while keeping the cost down, and keeping the client happy. the client is usually very sensitive to latency, and less on throughput (unless he is one of the big callers). so not showing latency is missing one side of the coin - the client perspective.

e.g. If the DMBS RTT is much higher than the time spent in the framework, the two frameworks will likely have similar response times.

at the very least you should be able to show that the big throughput is not achieved at the expense of a degraded response time.

Framework overhead back and forth

and I would expect quarkus's overhead to be smaller than springboot (or is it?). in that case response time should be at least equal to springboot.

But...CPU usage, for the lighter one, will be much lower
If I would be an SRE I would size differently Quarkus with less CPU cores to save this to happen,

another approach would be to constrain the environment so that the different compared configurations get equal response times and throughput, but some show lower operational costs.

in any case, not showing latency is missing something in the story. back to the opening statement, as a client, I do not care too much that you can double your throughput if you degrade my experience.

[neoludo]

Hi there,

did you include SPRING AOT mode in your benchmark, as It should have consequences on metrics ?

Ludovic

[edeandrea]

They're there in the metrics. See https://github.com/quarkusio/benchmarks/blob/main/images/spring-quarkus-perf-comparison/tuned/results-latest-tuned-composite-for-all-light.svg for example.

[neoludo]

your link seems to talk about JDK AOT, not Spring AOT

[vsevel]

I believe the relevant parts are here:
https://github.com/quarkusio/spring-quarkus-perf-comparison/blob/main/scripts/perf-lab/main.yml#L99
https://github.com/quarkusio/spring-quarkus-perf-comparison/blob/main/scripts/perf-lab/main.yml#L131

[holly-cummins]

As @edeandrea says, we didn't show all of the variations we measured in our graphics on the front page, because it's a bit overwhelming to understand. We've measured Spring AOT. I confirm it is the Spring AOT feature that was measured, not JVM AOT. The JVM AOT measurements are coming soon. :) I agree that the line break in the chart makes it a bit ambiguous which it is, and that's going to be a worse problem once we also have JVM AOT in the charts.

This is the chart showing AOT. It's fairly neutral on throughput, slightly improves footprint, more noticeable improvement on start time, but still twice the start time of Quarkus:

[cdhermann]

Thank you, for your hard work and the deep insights. These benchmarks are impressive.

In my daily business life I am regularly confronted with decision makers who hear Java and think slow, old, why not .NET - especially when dealing with the Azure Cloud. My developer joy doesn't impress them too much.

To help bridge the gap in corporate discussions where Java is still often viewed as 'slow' or 'memory-heavy' compared to .NET, would it be possible to include ASP.NET Core (Native AOT) in future comparisons?

Having direct head-to-head numbers would be a powerful tool to prove that Quarkus makes Java just as cloud-native and efficient as its competing non JVM languages which are using some form of byte code as well.

[cdhermann]

Hi @edeandrea,

I completely understand that comparing to .NET isn't a current priority for the team. However, inspired by this discussion, I’ve spent some time porting the benchmark to .NET 10 to see how it stacks up from a developer experience perspective.

You can find the implementation (with a lot of help from Sonnet, Gemini and Google) here: https://github.com/cdhermann/spring-quarkus-perf-comparison

While I started this to compare performance, the most revealing findings were actually about productivity:

Native Compilation: Unlike Quarkus, which handles GraalVM configuration out-of-the-box, .NET’s NativeAOT currently struggles with its standard ORM EF Core, making true "Native" comparisons impossible.

Boilerplate & Config: The .NET version required significantly more "plumbing" code (approx. 35% more) for DI, JSON context, and infrastructure wiring that Quarkus extensions handle automatically.

Testing Integration: Quarkus features like @InjectMock and @TestTransaction significantly reduce test-suite friction compared to the manual setup required in the .NET ecosystem.

Completing this port made me appreciate the practical advantages of the extension model - no version management hell. The out-of-the-box testing infrastructure does make a big difference. Being able to use the same codebase for both JVM and native modes without manual configuration and code changes is a significant differentiator. And these are good arguments for the decision makers.

Back to performance testing: running just stress.sh on a laptop doesn't give any clue about the real performance and throughput. Nonetheless, I find the differences interesting (always about the same magnitude after several runs on a macOS laptop):

Quarkus

Running 20s test @ http://localhost:8080/fruits
  2 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     9,74ms    4,33ms  77,59ms   77,62%
    Req/Sec   9781,43   2508,41  11765,00     95,24
  205410 requests in 20.01s,   1,07GB read
Requests/sec: 10265,37
Transfer/sec:  54,93MB

.NET 10

Running 20s test @ http://localhost:8080/fruits
  2 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    15,68ms    9,42ms 201,33ms   85,97%
    Req/Sec   6074,38   1818,96  7575,00     85,71
  127562 requests in 20.011s, 692,06MB read
Requests/sec: 6374,59
Transfer/sec:  34,58MB

[holly-cummins]

Oh nice!

Another data point you could show your colleagues is the TechEmpower benchmark results. It's designed to cover a huge range of languages and platforms. Because it's so huge, and there's so many variants even for the same platform, and there are multiple benchmarks within each category, it's a bit overwhelming to try and pull a single answer out of. But looking at https://www.techempower.com/benchmarks/#section=data-r23, which happens to be the one I have bookmarked, you can see that the the highest-scoring entries are all C++ and Rust (look on the right, at the 'language' and 'platform' columns). But if you look down the column, the first 'higher level' language is Java, at #13. And the next slot is Java too, at #14. You have to look a bit further down to find Go, at #17. .Net and C# are all the way down at #35. The higher entries tend to be stripped-down setups without higher-level libraries, so there are other .Net and Java entries all the way down in the table. For example, if you wanted MVC, ASP MVC is at #122.

I just did that analysis for the Fortunes benchmark, but the trend is similar across most of the others. For example, for 'Multiple Queries', the best aspnetcore can do is #25. For 'Data Updates', if I'm reading it right, the best .NET entry is what looks like a specially-crafted setup, and it's #83. The one exception is the 'Cached Queries' test, where .NET seems to do consistently well. .NET also just beats Java on the 'Plain Text' test. But on aggregate, Java is leaving .NET in the dust. :)

[edeandrea]

@cdhermann this is very cool! Honestly I wouldn't even really know how to build any of this. Do you have a list of requirements needed in order to run the full benchmark on a linux host? If we could modify the scripts (scripts/perf-lab/run-benchmarks.sh & scripts/perf-lab/main.yml) then I could take this and run it in our controlled lab, even if its only on your branch.

[cdhermann]

Do you have a list of requirements needed

@edeandrea, yes in the README.md there is a requirements section. But honestly speaking, I'd like to try it first on Linux. At the moment, I tried it only on macOS and Linux is theory. I also have to do slight changes in 1strequest.sh to make it work with .NET. And I should improve the current stress.sh to be callable for .NET more like the Java version.

I'll try my best to provide this soon.

[edeandrea]

Sounds like a plan! And no rush at all!

running just stress.sh on a laptop doesn't give any clue about the real performance and throughput.

That is 100% true, and something we tried to call out in our documentation. At the end of the day though we know people will do it, so we try and give people the "least bad" way to do it. macOS doesn't give you any way to isolate CPU cores to different processes.

[omatheusmesmo]

Well done! 👏🏻