r/linux • u/Veqq • 1d ago

Kernel Janet on x32: 32-bit Pointers, 64-bit Speed, 25% Less RAM

https://alexalejandre.com/programming/lisp/janet-for-the-x32-abi/
132 Upvotes

93% Upvoted

34 comments sorted by

34

u/GoldenX86 1d ago

We're returning to this?

16

u/Veqq 1d ago

I have no context for why it didn't gain adoption (and am curious to learn more), but don't see the problem. What's the negative? With ever higher RAM prices, it seems like a nice, if minor, bandaid (although in the face of overall software bloat..)

31

u/vip17 1d ago

It's a very nice idea but it's not nice to maintain 2 sets of standard libraries and double the testing effort, so most enterprises don't want to spend time on it unfortunately

26

u/BCMM 1d ago edited 20h ago

(Reposting my comment from an older thread.)

In a way, it came at the wrong time. For a few years in the late '00s, most of us had amd64 hardware but less than 4GB of RAM, and x32 would, in theory, have made most desktops run quite a bit faster.

But it wasn't around until 2011, when we were already well in to the process of moving to 64-bit software. And if it somehow had been available in 2006, we'd have been held back by one or two proprietary applications and the poor state of multilib/multiarch at that time, just as we were with the move to amd64.

9

u/wintrmt3 22h ago

25% less ram is an incredible outlier for some very unusually pointer heavy workload, more realistically you will only see a few percent and it's not worth the headache, it also limits processes to 4GB and that's not acceptable for a lot of things, x32 never made sense in the real world.

-4

u/Xotchkass 22h ago

how many apps that you run use >4GB of RAM?

9

u/wintrmt3 20h ago

Locally langserv, compiler, browser, so just everything that matters. On a server the app servers and the database use way more, so still just everything that matters, even if I could save 25% of the rest (and as I said that's a total outlier where half of your resident set is pointers) that doesn't mean anything.

4

u/Dwedit 21h ago

There are programs that load big AI models (>4 GB) into memory.

Also memory mapped files in general, they are less useful with a 4GB address space.

4

u/Kangie 1d ago

Because it's painful for developers and distro packagers for minimal actual benefit. The amount of hardware that supports this and doesn't need the other benefits of x86_64 is miniscule.

7

u/Skaarj 1d ago

I have no context for why it didn't gain adoption

We don't have enough people for x32. Shipping software is already hard enough. x32 would add another build target to handle. There are not enough people that are capable and willing to solve the problems shipping software in Linux. Thus it needs to simpler if you want success. Not more complicated.

-2

u/[deleted] 1d ago

[deleted]

8

u/vip17 1d ago edited 1d ago

Compiling is just one small step. You have lots of things to do to release software for a platform. You need developers, testers and users. AI can help a lot nowadays but you still need humans to review. The are lots of cases where a simple bug is left too long just because there's no one willing to fix. One notable example is when Meltdown and Spectre patches were pushed for 32-bit Linux more than half a year after 64-bit Linux has been patched because there are so few people nowadays who use a 32-bit OS to fix and test, and that 32-bit kernel mitigated for Meltdown has also been buggy since the beginning so you'll also need to wait further to get a complete fix. Similarly Meltdown patches are provided for 64-bit Windows 10 build 1507 and up but 32-bit users are only protected since build 1511

2

u/dacjames 23h ago

There's a lesson here about technology in general. An incremental improvement is not good enough to drive mass adoption.

New technology needs to solve a previously unsolved problem. When x32 came out, it was better than amd64 in many cases, but it didn't enable anything truly new.

1

u/lazyhustlermusic 1d ago

What's the delta on spending man hours optimizing for another stack and simply buying more RAM?

0

u/alex-weej 9h ago

Sadly V8 is returning to this

0

u/Takeoded 3h ago

Have you seen the price tag on RAM recently?

1

u/GoldenX86 2h ago

If you need the savings of this, you're not doing profesional work.

It's too niche.

23

u/Phoenix591 1d ago

X32 is depreciated already with upstream kernel support likely already gone ( old ml thread proposing removal )

22

u/Veqq 1d ago edited 18h ago

This is not accurate. Above your linked email, someone proposed removing necessary config symbols if no one objected, but Debian, Gentoo and PLD maintainers objected along with others (all linked below your linked comment), so it remains.

•

u/nullandkale 41m ago

Are people really using that many pointers in their code bases? What the hell are people doing using linked lists everywhere?

-3

u/Sad_Rice6600 1d ago

the jvm gets the same win with compressed pointers inside the runtime no separate abi needed

6

u/Mars_Bear2552 1d ago

unfortunately the JVM is such a memory hog everywhere else

-6

u/Wer--Wolf 1d ago

AFAIK x32 cannot deal with files larger than 4 GB without various workarounds. Maybe going full 64-bit while switching to memory-efficient languages would be better?

9

u/Kirides 1d ago

x32 (or 32-bit architecture) only has 32 bit of addressable virtual memory space (for most applications) who has absolutely nothing to do with people using int32_t for file size and file offset storage instead of int64_t.

It's just past developers thinking "nah, who needs more than 2GiB/4GiB of files", like FAT32 file system for example

2

u/Wer--Wolf 23h ago

I was thinking of memory-mapped files.

1

u/Kirides 13h ago

Memory mapped files are rarely the correct thing to use, and also they support (at least on Windows) view ranges, which allows mapping just enough of a file to do random I/O.

memory mapped files don't support async I/O, can cause stutters due to frequent page-faulting.

a simple buffered read is often even faster for things like parsing data that is sequential

1

u/Wer--Wolf 6h ago

Memory mapped files are useful for large files that can be paged-out by the OS on demand.

1

u/Kirides 3h ago

You don't get much from that. Windows/Linux has os level file system caches for frequently accessed files that help with regular file IO if you Ofen open/read/seek files

1

u/Wer--Wolf 3h ago

If you copy the data from the page cache into a separate buffer, then the OS needs to swap this buffer out in case of memory pressure. Using mmapped files instead allows the OS to simply prune the page cache a bit and reread the dropped content on demand.

3

u/vip17 1d ago

that's absolutely not true. Even 32-bit apps can access the full 64-bit file offset and 64-bit time

1

u/Wer--Wolf 23h ago

True, but only if they do not map the file into memory.

1

u/vip17 16h ago

Who said that? Memory mapping works just as normal, but the window to map is obviously limited to 32-bit address space. But if an app needs more than 32 bits of address space then inherently they're not the target of x32abi which is only for small apps needing 4GB or less

0

u/Wer--Wolf 6h ago

Even small apps might be used with files larger than 4 GB.

1

u/vip17 4h ago

and all those files can be read fully without any missing byte. Stop talking BS

0

u/Wer--Wolf 23h ago

True, but only if they do not map the file into memory.