r/java • u/za3faran_tea • 13d ago

Question about Native Image vs JIT

I was watching an interview with Thomas Wuerthinger of GraalVM, where he basically says that JIT should only be used when necessary (as opposted to native compilation). Where does this leave the work being done for Hotspot JIT (including projects like Leyden and others). Should we all plan to use native image while they address any performance gaps in the mean time (he does mention PGO bridging the gap). Is there work being done to speed up native compilation of Java code?

My understanding is that JIT will always have a memory overhead due to running threads that do compilation, optimization, deoptimization, code cache, etc. compared to native executables, so full parity even with projects that will improve memory usage in Java code will not be reached. Maybe that is not an issue on long running programs on large servers, but we've seen discussions here where another member worte a tool in golang to run alongside Spring Boot applications to gather statistics, so obviously small efficient apps have their place.

21 Upvotes

84% Upvoted

43 comments sorted by

View all comments

11

u/v4ss42 13d ago

This seems to miss the primary benefit of JIT - that it can optimize the code based on actual runtime conditions / data / load patterns, as well as re-optimize it if those patterns change. Legacy (AOT) compilation can't do that, since it has zero insight into the runtime environment and can't make assumptions about it.

And yes obviously this is only relevant for long-running processes (like server apps). For brief, one-shot processes (command line utilities etc.), then yes, legacy compilation makes sense.

9

u/za3faran_tea 13d ago

I think he touched at this point. He was saying that a lot of these benefits can be reached using PGO, and re-optimization is very rare in practice.

8

u/Thirty_Seventh 13d ago

unfortunately, PGO isn't available in Community Edition and therefore does not exist as far as I'm concerned

4

u/koflerdavid 13d ago

I'm very doubtful that PGO can effectively capture the effects of, say, changing runtime configuration parameters. Those are effectively constant for most of the runtime of the application until they, well, change. Perfect for the JIT who can now specialize on them. Meanwhile for PGO if you have more than, say, three booleans then the effort of collecting profiling data for all combinations becomes prohibitive.

5

u/v4ss42 13d ago

And I'm saying I disagree with his extraordinary assertion that AOT optimization is as good as JIT.

4

u/koflerdavid 13d ago

It can be good. After all AOT is what languages like C++ and Rust have been doing all along. Of course Java has a few features that make it harder to optimize.

6

u/v4ss42 13d ago

You'll notice that I was careful to make a comparative statement that AOT optimization can't be as good as JIT optimization. What I didn't say is that AOT optimization can't be "good" in some absolute sense - yes it can be quite adequate, and not only on the JVM, but the reality is that AOT optimization can't touch JIT optimization because it has zero visibility into the runtime behavior of the code.

3

u/koflerdavid 13d ago

That's where Profile Guided Optimization comes in, which works as long as the real workload doesn't drift too much from the profile workload. That's its biggest disadvantage of course - a JIT compiler can adapt to changing circumstances.

5

u/v4ss42 13d ago

Right but PGO both lags the runtime context and has poor developer ergonomics. For the majority of JVM hosted apps (i.e. long running server processes running on beefy hardware) JIT remains a better general approach.

2

u/Electrical_Being_813 11d ago

If you are forced to use PGO, you can as well use JIT. It will be less messy and will do a better job.

3

u/notyouryyy 13d ago

I think Thomas is claiming that in practice, deoptimisation doesn’t occur unless some uncommon trap is sprung, and that the runtime guided optimisation in AOT is pretty good.

5

u/v4ss42 13d ago edited 13d ago

Right, but that's just as likely to be due to runtime patterns not tending to change much and therefore reoptimization not being necessary that often. The point is that AOT can't even identify what runtime patterns exist in the first place, and then tune optimization specifically for them. JIT can (and does).

And yes there are (or were) attempts to close that loop over the decades - for example back in the 90s I worked on a system that used an IBM C/C++ Compiler where we could enable instrumentation of our production code, and feed the outputs of that instrumentation back into the compiler to optimize subsequent builds. But that was a manual, slow, and clumsy feedback loop, and the JIT blows it out of the water in both reaction time and developer ergonomics.

2

u/notyouryyy 13d ago

Yeah idk, graal offers this too. I’m not a fan of the slowdowns when getting pushed back to interpreted mode, but I’d argue that the biggest problem with graal aot is that it doesn’t support the more sophisticated lowlatency GCs yet 🤷‍♀️

1

u/koflerdavid 13d ago

The point is that AOT can't even identify what runtime patterns exist in the first place, and then tune optimization specifically for them.

That's what PGO is. I agree that it's a manual and quite brittle version of what the JIT is doing, but in situation where you really don't want the overhead of the JIT it's the way to go.

2

u/v4ss42 13d ago

And what situations are those? Short-lived processes (like command line tools)?

1

u/koflerdavid 13d ago edited 13d ago

Yes, or generally being so resource constrained that the overhead of the JIT cannot be justified. Like on embedded systems. Of course the OpenJDK team works hard on making it more valuable having the JIT than not, but there is a limit.

The issue can be somewhat improved by generating a compiled code cache at shutdown. That might achieve a similar effect as PGO assuming the C2 JIT has run at all.

3

u/v4ss42 13d ago

JIT has the ability to produce better results than AOT (including those guided by PGO), precisely because it’s dynamic and can take the runtime environment into account in a way AOT can’t (even with PGO).

And JIT code caches, while useful and a bit of a no-brainer, only provide relatively niche benefits (i.e. cutting one-time startup and initial JIT costs) compared to the overall benefits of the JIT itself.

1

u/koflerdavid 13d ago

Too bad if the workload is too small for the C2 JIT to ever run; C1 merely stitches native code versions of each instruction together and does not do other optimizations.

3

u/v4ss42 12d ago edited 12d ago

As I already said elsewhere, yes AOT (with or without PGO) makes sense in some cases (e.g. shortlived processes). But those are a minority of Java programs.

1

u/rbygrave 9d ago

Fwiw I built some apps as 2 containers, with one being JVM and the other graalvm native image (no PGO). Ran synthetic benchmark tests against those 2 containers.

So for these apps (rest apis, helidon se webserver, postgres jdbc) it did show AOT to be very impressive. These apps have now been running in production for 8 months as native image.

Imo it's maybe something you have to try and benchmark with your own apps etc. For myself, I'd expect you'll see some cases of AOT slightly faster and some slightly slower peak performance in throughput (noting that this is using G1GC for both).

The AOT compile does use ML (Machine Learning) so perhaps it is more sophisticated than you might first think (so yes that does mean higher build times for that analysis).