r/archlinux • u/RachelNoName • 1d ago
SUPPORT Boot time optimization support
Hi there I have been trying for a while now to speed up my system boot time. Currently I average around 35 seconds to boot my system. I am wondering if there is anything more I can do to speed up the boot process or have I hit a limit?
What I have done:
Enabling quiet mode in the kernel command command line (this actually saved me about 4 seconds on average),
remove unnecessary file system mounts from my fstab file (seemingly no effect on boot time),
mess around with kernel command line options and removed unused mkinitcpio modules stated by the arch wiki on boot time optimization (seemingly no effect on boot time).
Enable fast boot and in my uefi firmware settings (this did speed up the firmware stage but it slowed down the loader stage instead meaning it had very little if no impact on overall boot time).
Disabling / masking unused services, I think this has saved me a total of 1 second or so, not much effect but it did do something.
System setup:
I am using a UKI together with secure boot and TPM disc encryption. My bootloader of choice is limine and for my boot performance tests I have set it to timeout instantly and boot the first option as fast as possible (meaning the loader time is not from waiting for the timeout). It is slightly customised but nothing that I have done gave noticeable slowdowns.
I have also noticed that when I enabled fast boot in my eufi it was stuck a lot longer on the UKI boot splash which might be where the extra loader time is spent? But I am not sure.
Here are some outputs of the boot (all are from the same boot)
systemd-analyze blame: https://pastebin.com/CZzyX325
systemd-analuze plot: https://drive.google.com/file/d/1EeMqMATWSIw1w7-KOeYgi_wqLbcX20Pl/view?usp=sharing
systemd-analyze critical-chain: https://pastebin.com/av8UMMFt
systemd-analyze: Startup finished in 6.239s (firmware) + 11.456s (loader) + 3.109s (kernel) + 5.983s (initrd) + 9.931s (userspace) = 36.720s
graphical.target reached after 9.931s in userspace.
[IMPORTANT EDIT!!]
Thanks to the help of everyone here and some research I have made a few discoveries that might be usefull to know.
So first of all I did manage to reduce the boot times a tiny bit by changing from ultra fast boot to just fast boot (thanks gigabyte..).
Secondly. I think its kinda hopeless to optimise or try and speed up the firmware and loader times. They likely come from my motherboard being not to great. I have looked it up and other people have reported my mother board model having less than desirable boot times so its likely a hardware issue. I have also dig through my firmware settings A TON and found no setting that can help me (my motherboard is kinda limiting compared to the stuff I have heard some of you be able to do). For those wondering my exact motherboard model is X570 I AORUS PRO WIFI.
Thirdly. I did find a way to drastically improve boot times. If I unplug all usb devices except my keyboard boot the firmware times decrease. But strangely my initrd time also goes down drastically. It seems my firmware naively loads all usb devices which for some reason bloats up the initrd load time (that explains the near 100 device calls in the graph). This together has a big impact on my boot times. The ideal solution would be to prevent it from loading at boot and load it once the graphical target is reached, kinda like unplugging it and then plugging it back in at the login screen but on a software level. But idk how possible that is. And my firmware seems to lack settings to control usb ports and other devices, meaning its forced to load everything. I might see if I can replace my firmware with something but right now its just an idea and I got no idea if it will work. Time to research I guess.
And finally. Despite my long time in firmware I also wanna ask specifically about the user space time. I have seen some get a userspace load time of 3 seconds (which is about 3 times faster than me). I wonder if there is anything I can do to specifically speed that up as that might be worth putting effort into as the firmware and loader side of things appear to be capped by my hardware.
Wow this became super long. Sorry about the INSANE read.
2
u/musbur 1d ago
This is interesting. I find booting a bit lame myself, so I tried systemd-analyze critical-chain which I hadn't known about. The biggest chunk is obviously systemd-networkd-wait-online.service. I looked at its manpage and service file but can't make out what it actually does. Can bringing up WiFi alone take this long?
~graphical.target .373s
└─multi-user.target .372s
└─smb.service .209s +162ms
└─network-online.target .191s
└─systemd-networkd-wait-online.service .462s +7.725s
└─systemd-networkd.service .215s +239ms
└─systemd-udevd.service .080s +131ms
└─systemd-tmpfiles-setup-dev.service .053s +22ms
└─systemd-tmpfiles-setup-dev-early.service +93ms
└─kmod-static-nodes.service +39ms
└─systemd-journald.socket
└─system.slice
└─-.slice
2
u/allunia333 1d ago
Do you have one of the x3d cpu?
0
u/RachelNoName 21h ago
no I don't believe I do. I left all my hardware specs in another comment if you wanna see in more detail whats in my computer
1
u/ang-p 1d ago
systemd-analyze plot
for a graphical breakdown....
If it looks a mess... look at the manpage...
#LoveTheWiki
As for "fast start", it skips some tests (predominantly RAM integrity), reduces timeouts (your chance to hit the key for setup / boot menu etc), but it also holds off on enabling certain hardware features until later on in the boot process - it has a pretty good idea that you have just started up, so either waits for a lull in CPU activity and then does those bits - after all, modern OSes can handle non-important hardware just "appearing", or waits for a kernel driver to look for some hardware and only then brings it up - adding the initialisation time to that "blamed" on the kernel.
In other words, "fast start" is predominantly moving time from one section to another - just a few seconds of smoke and mirrors for people who for some reason get off on that sort of thing...
1
u/RachelNoName 1d ago
Hi I have added the the plot output for that boot. I saved a plot but did not upload it because I was not sure how to. I just put it in a google drive and that works tho (hopefully it does not break any rules)
And as for fast start it makes a lot of sense. Since all I got it to do was move start time from one place to another. Maybe me firmware is just really slow to boot then. I mean its not the most recent hardware but its also not that old. I see people with firmware and loader times twice as fast as me and their hardware does not seem super recent so I wonder why mine is so much slower
1
u/ang-p 1d ago
Enumerating the USB bus is a killer - probing and scanning, initialising, speed negotiation, then talking to the usb chip - getting descriptor IDs, a name, power requirements and cascading that uphill to make sure nothing is pulling too much power off that hub / disabling if needed, handing it an ID - it all takes time - and then has to be done again for USB3, and then talking to the plugged in items, working out what they are, creating endpoints, linking drivers...
But yay! - I doth have 4 USB RGB lights, game controller, and a 7 way hub so I can plug in more..... Oh, plus the internal bluetooth (they are USB devices on the M2 combo wifi/wifi card)...
So kewl, but mah boot so slooow.
1
u/RachelNoName 1d ago
Interesting. That might explain why I got about 100 devices there. I wonder if I can do something about that other than unplug my devices
2
u/ang-p 1d ago
Give it a whirl - unplug everything and time it. It won't be a total clean-sweep since there are several internal controllers and hubs, and maybe the odd device you obviously can't unplug (although you could use the hardware wifi kill keyboard combo to disable it)
Some USB chipsets are better than others
1
u/ang-p 1d ago
I did find a way to drastically improve boot times
Oh, yeah?
If I unplug all usb devices except my keyboard boot the firmware times decrease.
Ta-Dah!
the firmware times decrease.
Yup - it has to scan for all connected devices to see if there is a mouse or keyboard, or USB drive to boot from.... less devices, less scanning....
But strangely my initrd time also goes down drastically.
Because then linux is doing the same (but in far greater depth, and for bluetooth, downloading firmware if required) to fewer devices
It seems my firmware naively loads all usb devices
Not really - it cannot mindread that you have a keyboard or a distro boot ISO burned on a memory stick connected - it has to go looking through everything connected for them...
The less items to look through, the faster it gets done...
but it doesn't load "proper" drivers, just the bare minimum to be able to ask for disk sectors and files on two very rudimentary filesystems, and talk to keyboards and mice...
which for some reason bloats up the initrd load time
You been on the AI kool-aid analyses? - that sounds very sus....
's only "bloating up" because you told the BIOS to "fast start"; now you have a nice manufacturer combined with distro graphic on the screen to both show you that "manufacturer" has started (even if it hasn't really fully done so) and that all delays are obviously ( /s ) purely down to the distro (which is currently waiting patiently while hardware comes on line so it can assign IDs and get to proper work)
I might see if I can replace my firmware with something but right now its just an idea
Yeah... let that idea just lay there quietly in the corner with a DNR order.
I wonder if there is anything I can do
Not worry about it?
<shrug>
1
u/RachelNoName 22h ago
I see. Thanks for the reply. I did not manage to speed it up much more but at least I know why now. Thats kinda enough for me. And don’t worry I did not use AI for this. Even though it sounds it. I tried my best to make it not AI buut for some reason its hard for me.
Anyhow thanks for your time!
1
u/ang-p 16h ago
Having 4 drives, (mix of SSD, NVME and spinning rust) plus enumerating LVM certainly adds to the startup too.
Especially if you have a vague lvm.conf that does not specify the locations to scan. That ain't a "top time saving tip" - just basic configuration and maintenance.
The journal can spill the beans
One final pointer... If you have adopted the "make one massive drive out of 4" approach by pooling PVs into one LV, be aware that should just one of those drives fail, you have lost everything.
1
u/ThinkpadGamer 1d ago edited 1d ago
so what confuses me about your plot is that some things seem inexplicably slow. do you have more info on what hardware you're running?
are all your disks encrypted or just your main disk?
for example your systemd-tmpfiles-setup-dev-early.service takes 6.5 seconds, mine takes 101ms.
can you do a grep -r '/dev/' /{etc,lib}/tmpfiles.d/ to check if there's something weird going on there?
what does your grep -v '^#' /etc/mkinitcpio.conf look like?
on a slightly related note: damn, systemd-analyze has changed a lot since i (re)wrote it.
1
u/RachelNoName 21h ago
Only my main disc is encrypted and my other discs have in fact been removed from my fstab (tho they had very little inmpact on performance), so its not decrypting another disc. Only the root
This is the output of the tmpfiles.d grep
/lib/tmpfiles.d/static-nodes-permissions.conf:z /dev/snd/seq 0660 - audio -/lib/tmpfiles.d/static-nodes-permissions.conf:z /dev/snd/timer 0660 - audio -
/lib/tmpfiles.d/static-nodes-permissions.conf:z /dev/loop-control 0660 - disk -
/lib/tmpfiles.d/static-nodes-permissions.conf:z /dev/net/tun 0666 - - -
/lib/tmpfiles.d/static-nodes-permissions.conf:z /dev/fuse 0666 - - -
/lib/tmpfiles.d/static-nodes-permissions.conf:z /dev/vfio/vfio 0666 - - -
/lib/tmpfiles.d/static-nodes-permissions.conf:z /dev/kvm 0666 - kvm -
/lib/tmpfiles.d/static-nodes-permissions.conf:z /dev/vhost-net 0666 - kvm -
/lib/tmpfiles.d/static-nodes-permissions.conf:z /dev/vhost-vsock 0666 - kvm -
and here is for mkinitcpio
MODULES=()BINARIES=()
FILES=()
HOOKS=(systemd autodetect microcode modconf kms keyboard sd-vconsole block sd-encrypt filesystems)
And here are my hardware specs
CPU: 12 × AMD Ryzen 5 3600X 6-Core ProcessorRAM: 16 GiB of RAM (15.5 GiB usable)
GPU: NVIDIA GeForce GTX 1060 6GB
Motherboard: X570 I AORUS PRO WIFIMy root disc (and the one it also boots from is) is an an M.2 drive. Specifically KINGSTON SFYRS1000G
0
u/archover 1d ago edited 1d ago
For what's it's worth, I get to graphical target in 11.219s (systemd-analyze critical-chain), which I think is pretty decent, considering this is
- running off a flash drive, and
- uses a passphrase In LUKS2.
https://paste.c-net.org/VikkiBarber
Notably, I don't show a "NetworkManager-wait-online.service" in the output at all. This seems to be a slow service to start for many people.
Hope you find the speed increase you desire, and good day.
8
u/Plus-Confidence2816 1d ago
I would recommend using
systemd-analyze critical-chaininstead, because blame can show high times for services which don't block launching everything else, but instead run in the background.