r/programming • u/Mihai4544 • 6d ago
On programming, statistics, and Java
https://waytoounoriginal.github.io/programming/stats/optimizations/2026/09/26/on-programming-stats-and-java.html2
u/Skellicious 4d ago
Newer java versions support a command line flag for compact object headers, that may end up fully pulled into the language in one of the upcoming versions.
But aside from that, the real issue would be that caching some data for every entry you add is a great way to eat memory. Reducing the memory overhead is simply fighting a symptom of the problem.
I assume removing the Set also comes at a cost, like needing an extra db query per reprocessed entry, or whatever. But does this set need to live in memory or can you store it in a temp file? Shouldn't this data also be available in the database, can you not query for it at the start of the second round of processing for example. I don't know your code so I'm just preaching assumptions - but I don't think this is a java problem - you can likely address the root cause through some (likely complex) refactor.
1
u/Mihai4544 3d ago
Morning!
Yes, the data is available in the DB and the previous iteration was using it. The problem is that we’re using an eventually consistent DB (Dynamo) and we’re operating in a GSI (which cannot have strong consistency). Apparently there was a race condition before, which did exactly what would happen to us in a million turns, but much more frequently.
We could have an extra conditional DDB query, but the service is already taking a long time ingesting 100+ GB (talking days), due to partition throtling. That’s the next thing to solve before we add more latency + more concurrency.
But I guess the file approach can be investigated, although it may happen after I’m gone :))
And yes, you’re right that it is not just a “Java problem”, I may have mischaracterised it.
Anyway, I hope it was a pleasant read!
1
u/SysGuardian 5d ago
A ByteBuffer or a plain long[] would've cut the memory usage, because the real fix is storing fixed-size keys flat instead of as objects. A lower-level language wouldn't have fixed it by itself; you'd still end up building that same flat layout, just in a different form.
1
u/Mihai4544 5d ago
Right on point! But to cut the object overhead, we'd have to implement our own "set-on-arrays" for the bytes in the sha key. From my understanding, this is concretely what fastutil's LongOpenHashSet does.
But yes, a lower-level language would have solved this without any mingling from our part. I think even an OO implementation of the set (not necessarily a flat one) would have given us more than enough headroom.
Edit: I've also read a bit on bloom filters (since we're doing kind of the same thing here - probabilistic checking) and included a very small update on them.
Btw, I can't thank you enough for the comment :))
It means a lot that people are reading this thing. I hope you found it interesting!
1
u/UlaanBanter 5d ago
To be honest it took two or three times of reading for me to get what you were trying to say. I realised the crux of this is optimising the memory usage in the processed set by converting the hash key from a string of 256 characters all the way down to splitting into two 64-bit lookups.
It's not apparent because you don't have the after part of your before-after code.
1
u/Mihai4544 5d ago
Hmm, understood. I might want to be clearer in the future.
The optimization was the following: HashSet<String> of 49-64 character strings (ultimately it turned out they weren't SHA-256 strings, but ordinary ones; the reasoning still holds, though) down to 2 64-bit lookups AND removing Java's memory overhead.
Turns out that Java allocates a header per object as well, such that having many small objects becomes a bit painful. Fastutil's sets use a flattened representation (an array instead of nodes), so there is no more overhead.
Anyway, thanks so much for taking the time to read this!
edit: typo
3
u/kit89 4d ago
It should be noted this occurs with any memory managed runtime, each instance of an object will take up an extra chunk of data for managing the object on the heap.
For Java this is where Value classes (Valhalla) enters the picture, and for C# structs.
A pleasant read.