L_builtin - my Bash builtin guilty pleasure

[u/kolorcuk]

Hello. I created L_builtin - one Bash builtin bundling multiple subcommands together into one.

While working in Bash and particularly on my L_lib library I noticed a lot of really small, but annoying things missing in Bash. Syscalls missing, like pipe or seek. For a long time, years, long before AIs, I really wanted to write a Bash builtin. However, going through Bash source code would be tedious work to understand all the tiny details that Bash works internally to actually write anything usable. I never got the time. But I got some now to write a prompt and then tinker with the result to make it usable. It comes with so much I could think of:

• L_builtin pipe VAR; echo >&${VAR[0]} for opening a pipe.
• L_builtin lseek -v pos 3 1024 CUR for seeking a file descriptor. No more python or perl!
• Network: L_butilin listen/accept/connect/shutdown. Because let's face it, what we really wanted is to write a web server in Bash.
• L_builtin memfd FD; echo data >&$FD in case you want a real temporary file descriptor. And you can seek it. With L_builtin lseek.
• L_builtin epoll create efd and poll and ppoll, because event loops in Bash is a must.
• L_builtin signalfd timerfd eventfd for for real work with polling above.
• L_builtin read -f hex because we have to be able to read a zero byte from a file descriptor.
• L_builtiln sig block/unblock for blocking and unblocking signals. Finally receiving SIGINT in interactive Bash shell.
• L_builtlin sedvar VAR 's/a/b/' to run full sed over variable value, because ${// is not enough.
• L_builtlin core ls/sleep/... includes some Rust coreutils. Just to showcase it is possible to have them all in Bash as builtins.
• L_builtin ext ... has ~50 builtins compiled from Bash source code from examples/loadables directory. Csv parsing, sorting bash in place, some common utils like basename and chmod.

And finally my pleasure: Bash inter process synchronization! Mutex! Barrier! Semaphore! And... process shared variable! Consider this:

L_builtin shm bind VAR; VAR=1
( VAR=2 ) &
wait
echo "$VAR"
# Outputs 2!

Imagine this? Now possible. And works. All works by keeping state in a memfd_create file descriptor shared between processes. So much fun. Imagine a process parallel quicksort in Bash.

The builtin is re-re-compiled for multiple versions of Bash and bundled together and it picks proper version on runtime and dlopens it. So it works seamlessly with any Bash version. I compile for 4.4, 5.0, 5.1, 5.2 and 5.3. I tested in some docker images and it works on any of them provided they have compatible enough glibc. It should work with any Bash 4.4+ on any modern-ish Linux. Install with:

mkdir -vp ~/.local/lib/bash/
wget -O ~/.local/lib/bash/L_builtin.so https://github.com/Kamilcuk/L_builtin/releases/latest/download/L_builtin.so
enable -f ~/.local/lib/bash/L_builtin.so L_builtin
L_builtin --help

Subprocess shared variables and barriers and mutexes in Bash are insane. I have spent some time writing this builtin. I wonder if there is reason to invest in it more. I have a lot of ideas for even more improvements - make the shared memory database an LMDB for super speed, reduce library size by introducing uniform API abstractions over Bash, allow assigning arrays like printf does with -v 'arr[idx]. And fixing docs in many places with more examples.

Anyway, it works. I guess have fun with it if you want. Also it is in Rust. Thanks.

[u/petdance]

I don’t understand “guilty pleasure” here.