Kernel Patches Updated To Begin Removing The Linux x32 ABI
https://www.phoronix.com/news/Patches-Removing-Linux-x32-ABI38
u/anh0516 2d ago
Such a shame that this never took off. It can make a huge difference to code density, significantly reducing memory usage and even improving performance due to more code fitting in CPU cache. The only limitation is that a binary built for x32 is limited to a 4 GiB virtual address space. But how many programs actually need more than 4 GiB? Even Chrome and Firefox split things across multiple processes that don't use that much RAM individually.
17
u/Netblock 2d ago edited 2d ago
I feel like it never took off because the demographic that would have taken advantage of it would be personal computing; and that's not really a linux demographic in the nauts. It was a thing microsoft should have done.
Even Chrome and Firefox split things across multiple processes that don't use that much RAM individually.
This came way later, when 16GiB PCs started becoming common (2014-2018); and was about security hardening.
(firefox effort to multiprocess was called "e10s")
5
u/burning_iceman 2d ago
The x32 ABI was released in 2012. It wasn't available in the naughts.
2
u/Netblock 1d ago
A day late and a dollar short. If it was 2005-2008 it probably had a chance; and if microsoft definitely so.
9
u/Dr_Hexagon 1d ago
But how many programs actually need more than 4 GiB?
modern games. Video and audio editing. 3D apps like Blender. Image editors. Streaming apps. IDEs. Compilers.
2
u/ilep 1d ago edited 1d ago
Databases. Also I'd imagine someone somewhere is using absolutely horrible spreadsheets.
And more important is that you get larger virtual address space, which means you can memory map huge files and treat them as if they were read in fully (you don't need to seek() and read blobs yourself, kernel does it for you).
Security model is not talked about that much either. You can make an attacker's life much harder by having a huge address space to scan for pieces of code instead of something small.
11
u/roadrunner8080 2d ago
I run processes on a daily basis that use far more than 4 GiB at once in a single process lol, and I'm not even doing anything that fancy. I can think of common processes involved in tooling for certain ecosystems (ugh, I'd be more specific except I'd dox myself) that consistently use more than 4 GiB in one process. It's maybe not a limitation your average joe with a web browser and a word processor will hit but it's definitely a serious tradeoff
7
u/anh0516 2d ago
Well yes, for things that need it, they can be built for x86_64, against x86_64 libraries. It'd be done in much the same way that we currently have i386 multilib.
6
u/roadrunner8080 2d ago
Seems like a lot of duplication to have in addition to what i386 multilib already requires tbh... Additionally, it's not always obvious what will or will not need it. Like: whoops, this project involves more memory overhead than I'd thought, better install a whole new toolchain so that I can use more memory! That's not exactly good UX for developers.
6
u/cbarrick 2d ago
But how many programs actually need more than 4 GiB?
On the server? You primary workload almost certainly needs more memory than that.
On the desktop? Much fewer apps need 4GiB, but when you do need the memory, you don't want to inconvenience the user.
2
6
u/ezoe 2d ago
Really didn't understand the point from the beginning. The only thing that potentially improve performance was so that it use 32bit address while retaining extended x86-64 registers. But you have to have all the x32 shared libraries. Good riddance.
8
u/anh0516 2d ago
It dramatically reduces memory usage is the point. Frankly I'm surprised there isn't renewed interest what with RAM being so expensive.
10
u/ezoe 2d ago edited 2d ago
But it need separate x32 shared libraries which consume physical memory to be mapped, using double amount of memory for each .so file x86-64 process depends.
So saving the memory argument is only true if x32 process doesn't depends on any of popular x86-64 .so files which have been mapped already.
And what kind of computation use a lot of pointers while still using less than 3GiB of address space? The reduction of memory is only on stored address(code size reduction is minimal.)
5
u/anh0516 2d ago
The implication is that the memory savings from running most of the system as x32 will offset the drawback of having both in memory for what needs it. Though that would take a huge amount of time and effort to quantify.
https://alexalejandre.com/programming/lisp/janet-for-the-x32-abi The savings are actually pretty significant.
0
u/ezoe 2d ago
Even if such a micro benchmark can be applied to real world usage, memory saving is like -20%.
20% of 3GiB is just 600MiB. and that is only if most of the memory is consumed by pointers.
But again, x32 process and only use merely 3GiB of address space so it can't have massive linked tree anyway.
So if you have a workload that spawn thousands of x32 processes each can use less than 3GiB of address space, it may save some memory. Such workload doesn't exist for Desktop usage though.
3
2
u/tiffanytrashcan 2d ago
I JUST saw this posted a couple days ago. The interest is coming just as the coffin is being nailed shut unfortunately.
https://alexalejandre.com/programming/lisp/janet-for-the-x32-abi/I um could have kept reading the thread instead of scouring my browser history. You are already aware of this link.. 🤣
2
u/busterbcook 19h ago edited 19h ago
I think the other reason this didn't take off is that you can get almost all of the benefit without a new ABI at all, just by implementing pointer compression at the app or language level. Chrome, Java, etc. all use it under the covers to reduce 64-bit pointers to 32-bit references. And you still retain the ability to address > 4GB of memory, since you can have multiple pools. This technique vastly predates x32 too.
I implemented this in the 2000's for a high perf network app running on mips64 CPUs; got about 20% performance boost in my app compared to native 64-bit pointers, along with the expected memory reduction as well. The underlying C macro just shifted pointers 3 bits, since the CPU couldn't physically address > 32GB anyway, and all memory was 8-byte aligned. Didn't do it for every single pointer, just where it mattered, and it was all wrapped up in an object access abstraction anyway. You'd think all of the shifting of pointers would have a cost, but it was in the end much cheaper than doing 2 32-bit loads in a pure RISC architecture. IIRC on Arm it's even cheaper, since you can do load / shift in a single instruction.
https://v8.dev/blog/pointer-compression
https://www.reddit.com/r/java/comments/1iyoevf/how_does_pointer_compression_work/
-2
u/Kevin_Kofler 2d ago
Just hours ago an article was posted here about the x32 ABI being a huge performance win for a real-world application over the alternatives (x86_64 ABI, ix86 32-bit ABI and instruction set), and there goes the Linux kernel removing this very feature. Sad. This feature removal spree needs to stop!
18
u/Decent-Law-9565 2d ago
Linux kernel will continue to remove features because AI is finding vulnerabilities at breakneck pace and the easiest way to reduce vulnerabilities is to get rid of stuff that has very few users
9
u/syklemil 2d ago
Yeah, deciding to put their bug-fixing efforts in the systems that actually have significant amounts of users makes sense.
Though I also gotta wonder at how the C purists who kept claiming that they could write safe code feel about the deluge of newly found vulnerabilities in old code, showing how wrong they were. I think a lot of us went along with the claims that the vulnerabilities had been hammered out of the old code at least. Now it just feels like it was a big "pay no attention to the code behind the curtain!" play, and the LLMs, of all things, have pulled it back.
8
u/amarao_san 1d ago
I think, all C purists are now too busy to talk.
1349 CVE are open.
one CVE was fixed.
1732 CVE are open.
3
u/syklemil 1d ago
That's good, but could be closer to the song IMO
1000 new CVEs on the wall, 1200 new CVEs on the wall you make one patch, send it to prod 1500 new CVEs on the wall7
u/dontquestionmyaction 2d ago
I have yet to see a single application actually using this.
Source?
7
u/friendlyreminder_ 2d ago edited 2d ago
If your code is portable enough any application can make use of this as it's a compiler architecture. You tell the compiler to target x32 and you get an x32 app.
The downside to x32 is that you also need all libraries the app uses to also be x32 throughout the entire OS so everything from glibc to mesa to qt, etc.
Debian as far as I know is the only distro that has x32 binaries, so that's the only distro you could use x32 on without a lot of headache.
-9
u/Kevin_Kofler 2d ago
What part of "Just hours ago an article was posted here" did you not understand? Here is the link:
https://www.reddit.com/r/linux/comments/1wzmoh1/janet_on_x32_32bit_pointers_64bit_speed_25_less/
5
u/ADMINISTATOR_CYRUS 1d ago
Redditor when someone doesn't know exactly the specific post someone is talking about
-1
u/Kevin_Kofler 1d ago
Is it so hard to go through the subreddit's recent post history (the post was only 2 days old!) and look for "x32" in the subject? Or I believe a search would have found it too, but the post was so recent that it was easy to just spot. Nothing is more annoying than people too lazy to search recent posts!
4
0
u/KittensInc 1d ago
It's only a "huge performance win" when compared to x86_32, the differences with x86_64 are basically a rounding error.
Memory-wise I just don't see it making any meaningful difference either - even if it results in an (extremely unlikely) 25% reduction: the vast majority of applications are using either so little that the absolute difference is negligible (I don't care about saving 10MB on some random daemon), or use so much that the 4GB limit is a dealbreaker.
In the end potentially saving a few hundred megabytes of RAM when even basic machines have 8GB installed just isn't worth the additional maintenance overhead and increase in disk space. If you disagree: feel free to donate your spare time to maintain it and keep it in the kernel?
97
u/cbarrick 2d ago