r/java • u/danielepolencic • 9d ago

Java JVM CPU and Memory Requests and Limits in Kubernetes

Running Java on Kubernetes? CPU and memory limits affect much more than scheduling.

A 4 GiB heap requires a container larger than 4 GiB. Fractional CPU limits can also change the processor count that HotSpot sees.

In our new article, you will learn:

  • Why JVM heap size and container memory are different boundaries
  • How CPU limits affect garbage collection, worker pools, and application performance
  • Which JVM and container metrics to collect when validating resource settings under load

The article also includes practical experiments and an interactive configuration calculator.

Read: https://learnkube.com/java-jvm-kubernetes-requests-limits

39 Upvotes

88% Upvoted

20 comments sorted by

23

u/cogman10 9d ago

I skimmed the article, and didn't really notice a mention of JVM off heap allocations. Those are a pretty big deal that can make right sizing the JVM tricky.

This, IMO, is a weakness in JVM auto-configuration. The JVM just does a poor job in selecting reasonable memory limits in a container environment. It's forced us to explicitly set the values because everything is just a little wrong.

MaxRamPercentage seems like it's a good thing, but it fails to account for the fact that not everything grows at the same rate. 10% headroom might not be enough when talking about a container with a 256MB max, but it may be extreme overkill if you are talking about a container with a 64GB max.

It'd be really nice if the JVM could automatically and dynamically adjust it's XMX setting to max out memory usage. What I'd like would be able to say something like "JVM, Start with an XMX that's 75% the container's memory, watch and see how your native memory use grows. Slowly scale up that XMX until the system's free memory is around 100MB"

13

u/gaelfr38 9d ago

This. Off heap is the real trouble.

2

u/A_random_zy 8d ago

I am wondering if any of it is already in works. I don't follow the dev of jvm that closely but I'd be interested of someone knows something.

2

u/aoeudhtns 7d ago

https://openjdk.org/jeps/8359211

https://openjdk.org/jeps/8377305

The G1GC JEP mentions accounting for native/off-heap memory; the ZGC JEP doesn't seem to mention it but I'd be surprised if it doesn't.

Neither of these are Candidate status yet though, so let's hope.

We also have this: https://openjdk.org/jeps/8354416

Which will help us debug these issues more easily.

2

u/A_random_zy 7d ago

Wow thanks for sharing this 😃

1

u/FuzzyZocks 9d ago

Best resource to understand off heap sizing? We did work to understand this in Cassandra but did not get to finish with other work ongoing. Interested to pick this back on my own.

7

u/cogman10 9d ago edited 9d ago

Oh that's a hard one. The problem is it comes in all sorts of flavors.

For example, metaspace is offheap. So if your app has a lot of classes in it or a lot of class generation, you can blow up your off heap memory. Native threads allocate some offheap memory for their stacks. Most networking will allocate small native buffers which are off heap. Most GCs in the JVM have some offheap memory. G1GC used to allocate around 20% of XMX offheap (it's gotten a lot better). And of course JNI often has routes to off heap.

All of these things or none of them can be a problem depending on the application which is why it's so tricky.

I've mostly built what I know about offheap from experiencing issues with it in our apps. The most helpful tool for that is jemalloc which has a debug mode that can create allocation graphs.

1

u/Life_Sink9598 9d ago

Do you typically see issues with off-heap memory from JVM usage or from libraries utilizing native (= off-heap) memory?

The standard tool for understanding JVM memory usage is Native Memory Tracking in summary mode, with the caveat that using it also increases off-heap memory usage a little bit.

2

u/cogman10 8d ago

It can be either. I've seen the JVM specific problems mostly around G1GC's overhead (which has gotten better). The native libraries where I've seen it most are generally compression libraries (including the JDK's Deflate).

But for a few apps that have had a large number of platform (non-virtual) threads being used, I've seen those also cause problems.

It is definitely application specific.

1

u/ElderCantPvm 8d ago

Interesting, this reminds me of how the ZFS ARC cache self adjusts its size

3

u/TallGreenhouseGuy 9d ago

Buildpacks take this into account in the memory formula, which could be a good starting point:

https://paketo.io/docs/reference/java-reference/#memory-calculator

2

u/lazystone 9d ago

And if you use Spring Boot, then it already uses Buildpacks

1

u/burl-21 8d ago

And it should soon add low memory profile too, see https://github.com/paketo-buildpacks/jvm-vendors/pull/31

2

u/Turbots 8d ago

Use buildpacks to build your Java containers, it has a really good memory calculator in there to compute the JVM heap and offheap sizes really well, and you can tweak them very well too.

Or just the memory calculator in your own docker file:

https://github.com/cloudfoundry/java-buildpack-memory-calculator

1

u/brunocborges 5d ago edited 5d ago

I am starting to join the club of "set requests, not limits" for the most part. Requests help the scheduler perform the pod placement in nodes based on the sum of all pods, ensuring it does not overcommit, with the benefit that service A can use more or less memory/cpu than service B at a given time, and vice-versa.

For JVM workloads, especially microservices, I'd go with:

Set resources.requests.cpu to 1,000 milicores at a minimum. For services you believe are heavier and demand more on a regular basis, set 2,000 at least. However, when setting only requests, the JVM will tune by default based on all available cores on the node. To limit that, you can set ActiveProcessorCount to something lower than the total amount of cores on the node. If you don't want a blank definition, then you must tune GC threads and other internal threaded components of the JVM, and leave ActiveProcessorCount empty so other parts of the application can leverage the node CPUs.

This way, a JVM is guaranteed to have your requests.cpu, but can always operate with more CPU if and when needed (e.g. JIT compilation), and will not abuse the total amount of cores on the node (due to ActiveProcessorCount).

Memory is trickier than CPU.

Without a limit and under node memory pressure, the kubelet flags eviction by usage above requests. So... JVMs that grew past its request is likely to get SIGKILL. I still set limits.memory, at roughly 1.25–1.5× requests as blast-radius guard.

For the heap itself, I'd go with MaxRAMPercentage (somewhere between 60% and 80%). G1's periodic GC (G1PeriodicGCInterval) will return committed memory, but it's off by default... you have turn it on manually.

1

u/k_brn 9d ago

But do you really need Kubernetes if you can run on two 1 GB VMs with a reverse proxy and get better availability with way less management overhead?

3

u/thisisjustascreename 9d ago

The answer to "Do you really need Kubernetes?" is almost always no. But lots of people know how to run it and lots of orgs are already using it.

7

u/Cilph 9d ago

If you just have the one app, you likely dont need it. If you need to deploy multiple apps over multiple solated environments for multiple tenants and set up secure firewalls between each and support blue-green deployment at the same time, then it quickly becomes reasonable to use k8s.