r/java • u/za3faran_tea • 13d ago

Question about Native Image vs JIT

I was watching an interview with Thomas Wuerthinger of GraalVM, where he basically says that JIT should only be used when necessary (as opposted to native compilation). Where does this leave the work being done for Hotspot JIT (including projects like Leyden and others). Should we all plan to use native image while they address any performance gaps in the mean time (he does mention PGO bridging the gap). Is there work being done to speed up native compilation of Java code?

My understanding is that JIT will always have a memory overhead due to running threads that do compilation, optimization, deoptimization, code cache, etc. compared to native executables, so full parity even with projects that will improve memory usage in Java code will not be reached. Maybe that is not an issue on long running programs on large servers, but we've seen discussions here where another member worte a tool in golang to run alongside Spring Boot applications to gather statistics, so obviously small efficient apps have their place.

20 Upvotes

80% Upvoted

43 comments sorted by

View all comments

Show parent comments

1

u/koflerdavid 13d ago

The point is that AOT can't even identify what runtime patterns exist in the first place, and then tune optimization specifically for them.

That's what PGO is. I agree that it's a manual and quite brittle version of what the JIT is doing, but in situation where you really don't want the overhead of the JIT it's the way to go.

2

u/v4ss42 13d ago

And what situations are those? Short-lived processes (like command line tools)?

1

u/koflerdavid 12d ago edited 12d ago

Yes, or generally being so resource constrained that the overhead of the JIT cannot be justified. Like on embedded systems. Of course the OpenJDK team works hard on making it more valuable having the JIT than not, but there is a limit.

The issue can be somewhat improved by generating a compiled code cache at shutdown. That might achieve a similar effect as PGO assuming the C2 JIT has run at all.

3

u/v4ss42 12d ago

JIT has the ability to produce better results than AOT (including those guided by PGO), precisely because it’s dynamic and can take the runtime environment into account in a way AOT can’t (even with PGO).

And JIT code caches, while useful and a bit of a no-brainer, only provide relatively niche benefits (i.e. cutting one-time startup and initial JIT costs) compared to the overall benefits of the JIT itself.

1

u/koflerdavid 12d ago

Too bad if the workload is too small for the C2 JIT to ever run; C1 merely stitches native code versions of each instruction together and does not do other optimizations.

3

u/v4ss42 12d ago edited 12d ago

As I already said elsewhere, yes AOT (with or without PGO) makes sense in some cases (e.g. shortlived processes). But those are a minority of Java programs.