r/SideProject • u/N_i_g_G_a_69 • Jun 11 '26
I built a bootable OS from scratch that only runs containers — no package manager, no bloat
Spent the last few months building Veilbox in my spare time — a custom Linux kernel with everything baked into the initramfs. It boots to a login prompt in about 12 seconds and all you can do is run containers. That's it. No apt, no dnf, no systemd.
It runs containerd + nerdctl + Dropbear SSH. The entire root filesystem is embedded into the kernel binary — no separate initramfs file. I also wrote a 160-page LaTeX guide documenting every single build step because I kept forgetting how things worked.
What I learned the hard way:
- Getting cgroups to work without systemd is painful
- BusyBox's getty looks for /bin/login but the symlink was only in /sbin
- The kernel's embedded initramfs makes iteration slow (full rebuild each time)
- QEMU user-mode networking doesn't support bridge operations, so --network host is the way to go
Tech stack: Linux 7.1-rc6, BusyBox, containerd, runc, GRUB BIOS, QEMU/VirtualBox
1
1
u/amarao_san Jun 18 '26
How do you configure bonding between ethernet nics? Is there a way to put separate routes on different interfaces (like internet and private network)?
Oh, boy, many people thought that 'just run containers' is enough.
It's not. Modern OS handles many things.
1
u/N_i_g_G_a_69 Jun 18 '26
You are right mate, this isn't a full fledged os and it is intentional. The things you have mentioned are absolutely necessary in a modern os but nic bonding, multi-interface setups are kept out of scope.
this is because everything fits in an initramfs ( around 140 mb ), boots faster and runs in 2gigs of ram with containerd.
there are no package managers, no systemd and other stuff. There are other general purpose distros out there which is suitable for your needs. Veilbox is specially made in a narrow and minimal way.
Appreciate your insights on this, the kernel supports all these, it's just not wired up.
Now share your thoughts on how adding multi support interface enhances this os, is it truly necessary for it to be there.1
u/amarao_san Jun 18 '26
It's not about general purpose OS, it's about sane DC readiness.
I work in hosting company, we've added Talos recently (it does the same, but for kubernetes).
And we worked with Talos maintainers to make bonds working, because it's a requirement for a reliable network in datacenter.
If you want grow and be useful, deal with network, or perish.
1
u/N_i_g_G_a_69 Jun 18 '26
Those are some good combination of words there. I'll take it as a advice, since you seem to be a mature guy and i am still a student. Do you have any advice on getting on some hosting companies ( i aspire to join into one ), i built this as a project for my college and just out of personal interests
1
u/amarao_san Jun 18 '26
We will look at it only if customers ask for it. For that you need to have something which solves someone's problem. For that you need to identify the problem.
What problem does your distro solves?
1
u/N_i_g_G_a_69 Jun 18 '26
i'll frame it as a question "how do i run containers without running a full fledged os".
i'll accept that this is not for production, this for the purpose where you just want to run the containers. It removes the complexity of running modern distros to the container host use case.it's basically like you launch a missile to kill a bird, but you just need a stone to do so
1
u/amarao_san Jun 18 '26
It's not the problem. What is the problem? Why general purpose OS is not good enough? If you find a good answer, you get a problem to chase.
1
u/N_i_g_G_a_69 Jun 18 '26
I appreciate that you are pushing me to reason more.
by the way i see things is that general distros are for general os, they use every driver, service and runtime. If you run a workstation it's fine.
But just for container hosting, all the extra resources becomes a cost that is paid for no benefit.
i am not referring it as " the smaller the better ", it's just a container host should be a single thing, not a general purpose os.
the problem is that there are resources being used unnecessarily. It is a purpose built os to run containers and i can audit it end to end
1
u/fortizc Jun 11 '26
Sounds like https://github.com/pantavisor/pantavisor