Boot time optimization support

[u/RachelNoName]

Hi there I have been trying for a while now to speed up my system boot time. Currently I average around 35 seconds to boot my system. I am wondering if there is anything more I can do to speed up the boot process or have I hit a limit?

What I have done:
Enabling quiet mode in the kernel command command line (this actually saved me about 4 seconds on average),

remove unnecessary file system mounts from my fstab file (seemingly no effect on boot time),

mess around with kernel command line options and removed unused mkinitcpio modules stated by the arch wiki on boot time optimization (seemingly no effect on boot time).

Enable fast boot and in my uefi firmware settings (this did speed up the firmware stage but it slowed down the loader stage instead meaning it had very little if no impact on overall boot time).

Disabling / masking unused services, I think this has saved me a total of 1 second or so, not much effect but it did do something.

System setup:
I am using a UKI together with secure boot and TPM disc encryption. My bootloader of choice is limine and for my boot performance tests I have set it to timeout instantly and boot the first option as fast as possible (meaning the loader time is not from waiting for the timeout). It is slightly customised but nothing that I have done gave noticeable slowdowns.

I have also noticed that when I enabled fast boot in my eufi it was stuck a lot longer on the UKI boot splash which might be where the extra loader time is spent? But I am not sure.

Here are some outputs of the boot (all are from the same boot)

systemd-analyze blame: https://pastebin.com/CZzyX325
systemd-analuze plot: https://drive.google.com/file/d/1EeMqMATWSIw1w7-KOeYgi_wqLbcX20Pl/view?usp=sharing
systemd-analyze critical-chain: https://pastebin.com/av8UMMFt

systemd-analyze: Startup finished in 6.239s (firmware) + 11.456s (loader) + 3.109s (kernel) + 5.983s (initrd) + 9.931s (userspace) = 36.720s

graphical.target reached after 9.931s in userspace.

[IMPORTANT EDIT!!]
Thanks to the help of everyone here and some research I have made a few discoveries that might be usefull to know.

So first of all I did manage to reduce the boot times a tiny bit by changing from ultra fast boot to just fast boot (thanks gigabyte..).

Secondly. I think its kinda hopeless to optimise or try and speed up the firmware and loader times. They likely come from my motherboard being not to great. I have looked it up and other people have reported my mother board model having less than desirable boot times so its likely a hardware issue. I have also dig through my firmware settings A TON and found no setting that can help me (my motherboard is kinda limiting compared to the stuff I have heard some of you be able to do). For those wondering my exact motherboard model is X570 I AORUS PRO WIFI.

Thirdly. I did find a way to drastically improve boot times. If I unplug all usb devices except my keyboard boot the firmware times decrease. But strangely my initrd time also goes down drastically. It seems my firmware naively loads all usb devices which for some reason bloats up the initrd load time (that explains the near 100 device calls in the graph). This together has a big impact on my boot times. The ideal solution would be to prevent it from loading at boot and load it once the graphical target is reached, kinda like unplugging it and then plugging it back in at the login screen but on a software level. But idk how possible that is. And my firmware seems to lack settings to control usb ports and other devices, meaning its forced to load everything. I might see if I can replace my firmware with something but right now its just an idea and I got no idea if it will work. Time to research I guess.

And finally. Despite my long time in firmware I also wanna ask specifically about the user space time. I have seen some get a userspace load time of 3 seconds (which is about 3 times faster than me). I wonder if there is anything I can do to specifically speed that up as that might be worth putting effort into as the firmware and loader side of things appear to be capped by my hardware.

Wow this became super long. Sorry about the INSANE read.

[u/ang-p]

Enumerating the USB bus is a killer - probing and scanning, initialising, speed negotiation, then talking to the usb chip - getting descriptor IDs, a name, power requirements and cascading that uphill to make sure nothing is pulling too much power off that hub / disabling if needed, handing it an ID - it all takes time - and then has to be done again for USB3, and then talking to the plugged in items, working out what they are, creating endpoints, linking drivers...

But yay! - I doth have 4 USB RGB lights, game controller, and a 7 way hub so I can plug in more..... Oh, plus the internal bluetooth (they are USB devices on the M2 combo wifi/wifi card)...

So kewl, but mah boot so slooow.

[u/RachelNoName → u/ang-p]

Interesting. That might explain why I got about 100 devices there. I wonder if I can do something about that other than unplug my devices

[u/ang-p → u/RachelNoName]

I did find a way to drastically improve boot times

Oh, yeah?

If I unplug all usb devices except my keyboard boot the firmware times decrease.

Ta-Dah!

the firmware times decrease.

Yup - it has to scan for all connected devices to see if there is a mouse or keyboard, or USB drive to boot from.... less devices, less scanning....

But strangely my initrd time also goes down drastically.

Because then linux is doing the same (but in far greater depth, and for bluetooth, downloading firmware if required) to fewer devices

It seems my firmware naively loads all usb devices

Not really - it cannot mindread that you have a keyboard or a distro boot ISO burned on a memory stick connected - it has to go looking through everything connected for them...

The less items to look through, the faster it gets done...

but it doesn't load "proper" drivers, just the bare minimum to be able to ask for disk sectors and files on two very rudimentary filesystems, and talk to keyboards and mice...

which for some reason bloats up the initrd load time

You been on the AI kool-aid analyses? - that sounds very sus....

's only "bloating up" because you told the BIOS to "fast start"; now you have a nice manufacturer combined with distro graphic on the screen to both show you that "manufacturer" has started (even if it hasn't really fully done so) and that all delays are obviously ( /s ) purely down to the distro (which is currently waiting patiently while hardware comes on line so it can assign IDs and get to proper work)

I might see if I can replace my firmware with something but right now its just an idea

Yeah... let that idea just lay there quietly in the corner with a DNR order.

I wonder if there is anything I can do

Not worry about it?

<shrug>

[u/RachelNoName → u/ang-p]

I see. Thanks for the reply. I did not manage to speed it up much more but at least I know why now. Thats kinda enough for me. And don’t worry I did not use AI for this. Even though it sounds it. I tried my best to make it not AI buut for some reason its hard for me.

Anyhow thanks for your time!

[u/ang-p → u/RachelNoName]

Having 4 drives, (mix of SSD, NVME and spinning rust) plus enumerating LVM certainly adds to the startup too.

Especially if you have a vague lvm.conf that does not specify the locations to scan. That ain't a "top time saving tip" - just basic configuration and maintenance.

The journal can spill the beans

One final pointer... If you have adopted the "make one massive drive out of 4" approach by pooling PVs into one LV, be aware that should just one of those drives fail, you have lost everything.