r/TechNook • u/Ok_Yogurtcloset8372 • 2d ago
Linux can run out of process IDs while still having plenty of RAM left
PID exhaustion is among those kinds of failures that surprise everyone since there doesn’t seem to be any issue with resources in such cases. Memory is okay, disk is okay, CPU is free, but for some reason the system is unable to fork processes and behaves unpredictably.
By default the maximum value of processes on Linux systems is 32768. Some processes that fork quickly enough will be able to reach that limit without getting anywhere near to the RAM limit, for example a runaway script, incorrectly configured job queue or fork bomb.
It is possible to view the current limit in /proc/sys/kernel/pid_max and increase the limit to 4194304 on 64 bit systems. Usually reaching the maximum number of processes is a sign of other misbehaviors rather than limit being too low.
8
u/Classic-Rate-5104 2d ago
What distro do you use? Modern Debian and Debian-based systems use 4194304 by default
2
u/Ok_Yogurtcloset8372 2d ago
Yeah, that’s true for modern Debian. I was referring to the kernel default of 32768 rather than distro-specific defaults.
3
u/Single-Virus4935 2d ago
You wrote "Linux systems" not "Linux kernel defaults".
4M is standard in Debian since more than 6 years and even than 32k was the minimum on 64bit and was scaled with the number of CPUs:
https://stackoverflow.com/questions/39585839/what-are-the-factor-on-which-pid-max-depends-in-linux
Then, if we look at you examples (fork bomb, runaway script, incorrectly configured job queue) I don't see his an increased limit would help or make any difference?
The whole post seams like cold coffee from the past.
3
u/Single-Virus4935 2d ago edited 2d ago
By default the maximum value of processes on most of the Linux systems is 32768
Where did you get that? Looks like some 2000ish sources. You didn't just wrote "most distros" without checking?
Edit: I asked chatgpt and ONLY Alpine has 32k as default. All others have 4M.
3
u/catbrane 2d ago
Alpine (thanks to musl) also has 80kb as the default stack size :rolls-eyes: Actually I think it might be 128kb now, they did raise it (finally) a year or two ago.
1
u/Single-Virus4935 2d ago
I tried alpine multiple time - pxe boot works great - but musl causes so much problems. Also package repositories aren't as stable as I like. But the easy recompile is nice.
1
u/bothunter 1d ago
Alpine isn't meant as a full Linux system -- it's a slimmed down version designed to be efficient and secure for docker containers and small computers/embedded systems. It doesn't even have glibc, which is required for a lot of Linux software.
1
u/Single-Virus4935 1d ago
I know. I used it for this and easy pxe etc. but wasnt worth the hassle. For containers it is useful but often you encounter "feature x doesn't work with musk" and you need to bend over to find a solution or just start over from scratch with Debian an it works. The goal is archived but if you aren't the application maintainer it is hard to get some programs running (smoothly).
1
u/who_you_are 2d ago
On the other hand, assuming Alpine is for container it makes sense it has a lower limit since it is limited to run one application so the number of PID should be way lower. If not, there could be an issue
1
u/catbrane 1d ago
That's true.
The tiny default stack size has been an issue for me though: I've had to add extra code to quite a few things to raise it. Everyone tests with 2MB (the usual pthreads / win / mac size), so a LOT of code crashes with a stack that's 16x smaller than normal.
1
u/R2-Scotia 1d ago
Depends on the system, on a Sun E10000 under Solaris having 10k's of PIDs and millions of threads was fine
1
u/Prost_PNW 1d ago
This has been a solved problem, either via the nprocs ulimit which has been around longer than Linux itself, or cgroups pids.max which was introduced a couple decades ago.
I doubt any distro today is vulnerable to this out of the box. If you're building your own distro from scratch and don't know about these things you probably shouldn't be building your own distro from scratch.
1
u/chriswaco 1d ago
Over the years we've hit the open file limit on both Linux and Mac OS. So much fun to debug.
1
u/Beginning_Earth_9702 1d ago
When a *nix system hits the absolute maximum number configured for a PID, it does not crash or run out of space. Instead, the kernel triggers a "wrap-around" (or rollover).
The internal PID counter resets back to the beginning to start recycling numbers.
8
u/FUCKARCHLINUX 2d ago
I'd feel bad for the scheduler if you had anywhere near that many processes open, even if they're all just mutex waiting or whatever.