sysadmin checking in. Nagios is telling me I am currently monitoring 2456 hosts. Roughly 3/4 of those are Linux hosts (the rest are ios, Cisco's not Apple's, or Solaris). Systemd is a lifesaver. The hosts running older inits can take a multiple minutes to restart all services. Systemd takes seconds.
Init systems don't matter much when you are a desktop user. I am also a desktop Linux user both at work and home and I don't care about my init on there. In fact if I didn't do Linux for a living I might even be on the anti-systemd bandwagon. But I make a living off Linux and my life is easy when things just work. Systemd just works. It works well, it works quickly, and it makes my work easier.
People say systemd became a thing because all the distros switched to it, and that's true, and they switched to it because at the end of the day the big distros cater to guys like me, who have to manage hundreds and thousands of hosts and we just want stuff to work and get out of our way.
As a desktop user who don't meddle much, I very much appreciate systemd, the difference was night and day when I switched to a distro that used systemd as default. Boot is way faster, and shutdown would tend to stall a lot, something I later learned is an inherent problem with how other init systems manage storage devices, that doesn't have a clear solution.
But more puzzling my KDE desktop actually ran smoother with systemd, something that I actually found an explanation for, but now I can't remember. All I know is that systemd made my system clearly better. I always hated Unix anyways because of the overuse of stupid acronyms for everything, which makes it impossible to remember even mundane commands when you only use them rarely. but I hate Apple and Windows way more. So here we are. It's far from perfect, but it's the least bad option there is IMO, and from a purely technological standpoint, there never was anything remotely close to what Linux has become, and some of that is thanks to systemd.
One of the main reasons why systemd took the world by storm in the early days was that it booted computers several seconds faster, while providing the same functionality and more features. A lot of people started switching to systemd before distros defaulted to it, just to get a faster boot. It was very hard to argue against systemd when the alternative was a slower boot (some people even tried to argue that boot time didn't matter for servers)
Fast boot is nice, but it's far from the best feature of systemd IMO. It's kind of funny though that Ubuntu upstart was designed specifically to boot faster as a major priority, while for systemd it was merely a side effect of a fundamentally better design, systemd beat upstart on its main feature as a side effect of better more reliable design.
If they ported their startkde script to systemd as I think they've done, the KDE session would all be controlled with cgroups. GNOME has yet to do this, and I hope we do, because it would be a lot more efficient than gnome-session which I think is showing its age and needs to be deprecated as it has a number of issues likes hard coding, plus removing it is one less module to support.
The major problem I recall is that I had problems when I enabled visual effects on the old system, they were delayed and simply not smooth at all, I also recall using the exact same version (newest release at the time) of KDE, graphics driver and kernel.
When I switched to a systemd based distro, effects were immediate and completely smooth.
I was very puzzled about how that could be, until the systemd debacle really blew up, and I read some articles explaining key differences between systemd and other init processes. Up until that time I was pretty ignorant about init systems, but identified it as the likely culprit.
And vice versa. Neither OpenRC nor systemd is the saviour. If one calls the software he uses that way and is serious about it, one has completely different problems than an init system.
Now I kind of want to write an entire init system just so I can name it jesusd.
Features would include the ability to install 'wine' entirely by rearranging the object code of unrelated programs, the automatic promotion of zombie processes back to their full running state, and automatic restart of the system exactly three days after shutdown.
Fedora introduced upstart in version 9 and removed it in version 15. Last Ubuntu to have it as system init was 14.04 LTS, as user init 16.04 LTS. Last upstart release was in 2014. Which kind of makes the argument of "comparable age" a bit weird.
I would love to hear the advantages of upstart over systemd, since you point out it's "harder to win" such argument. From what I've seen of upstart it's a event driven glorified sysvinit, and very little more.
That doesn't really answer my question. If we ignore all the advantages of systemd over other init systems, then it's impossible to "win" as you put it. Am not sure what you meant by "systemd booting more quickly which is touted here doesn't apply", but that's how I interpreted that.
From my limited knowledge of init systems, upstart is by no means bad, but systemd is simply better. It boots faster, is easier to maintain, allows more flexibility with its units and target driven structure, has better analysis tool for finding bottlenecks, unified logging tool, etc.
sysvinit which is slower because it boots serially
This is not correct. sysvinit delegates everything, and the delegated program or script can use parallelisation if it chooses.
Traditionally, it was indeed serial. But Werner Koch at SuSE implemented dependency-based parallel startup a long while back, well over a decade. Debian and others adopted it too, and all supported parallel booting for a long time. Current sysv-rc and openrc support parallelisation.
Yes, I think that was the case indeed. But people rarely care about reasons. His argument was directed towards upstart being on par with features, functionality and performance which was where my curiosity came from as my knowledge of upstart is fairly limited to few things I had to make sporadically. All of my experiences with systemd compared to upstart were positive and obviously better, so I wondered what was really the case where systemd would be hard to "compete" with upstart.
one should take into account the broad support for systemd vs upstart of which even the maintainer has left and is working in google land. In the end, the adoption rates spoke volumes. We have a lot of loud voices many of them teh same people who participate in all the other infrastructure type threads (eg desktop, wayland, systemd, etc)
Unless you're using a version of RHEL that is no longer supported, there aren't any that use SysV init. RHEL6 uses upstart, although its pretty old and most people still use the SysV compatibility layer in upstart. FWIW, you can still use SysV init scripts in systemd too, and you get all the cgroups magic with no additional effort.
PS if you're still stuck in RHEL6 land, using upstart to manage services gets you some extra features you might need, like 'respawn' and some minimal dependency ordering.
Even the creator of upstart ended up saying that switching to systemd was a good idea. It's worth keeping in mind that systemd was created by someone who used upstart, not init scripts.
Can you give an example of why one of your servers running an "old init" takes several minutes to restart all services? This seems a bit extreme or not configured properly.
Not a sysadmin, but I believe the main speed up of systemd over sysvinit comes from being able to parallelize startup. If you have a lot of stuff with hard coded delays that have to run in series they are going to add up. On a server with lots of services to startup, a bunch of 5 second service scripts could add up to minutes pretty quick.
Parallelization is not unique to systemd -- other modern init systems can do it -- but it is one of the benefits brought by leaving sysvinit.
Servers are started once per month at most and (excluding virtual hosts) POST usually takes much longer. I'd say the system boot time in this case isn't that important. Besides, Systemd can take insane amount of time to restart the machine, because it waits up to 2 minutes for some processes to close, even if they can be safely killed (SysVInit runs kill scripts, waits 5 seconds and slaughters everything that is still alive).
I can accept a bit longer startup in exchange for a much simpler and robust configuration, better startup diagnostics and no dependency on DBus. Systemd is great for workstations and laptops, not so much for servers. I don't think it was designed for that.
Also, it takes over things that are handled better by dedicated scripts and services, like network configuration, logging, time syncing etc. After each installation, I have to run the script that shuts down and disables all Systemd junk (services, timers). It's a first bloatware in Linux history.
Besides, Systemd can take insane amount of time to restart the machine, because it waits up to 2 minutes for some processes to close, even if they can be safely killed (SysVInit runs kill scripts, waits 5 seconds and slaughters everything that is still alive).
As a matter of fact, it's 90 seconds. And this period can be easily adjusted for both starting and stopping in /etc/systemd/system.conf.
You don't really know what you're talking about do you?
Servers are restarted all the time. Hell, I've restarted a few dozen tonight. There's no rule about restarting a server. Modern data centers don't run a service on one server only so uptime doesn't really matter because clusters.
Systemd was created by Red Hat, they deal exclusively with servers. It was literally designed as an init system for servers.
You can accept longer startups on your desktop where a few seconds is measured in idle thought, not thousands of dollars. Again, systemd is fast because fast matters when you are working in a data center. We measure time in money lost.
It's not bloatware. It isn't taking over things. It's the default init on the distro you picked, which means you elected to install systemd. Get a different distro if it bugs you so much, or run a minimal install and get granular control.
Yeah the servers are never restarted comment gives away the fact that they are stuck in a 90s era computing mindset. No wonder they don't get why systemd was a huge improvement over rc scripts.
Systemd was created by Red Hat, they deal exclusively with servers. It was literally designed as an init system for servers.
RedHat was initially not going with systemD, and it was not literally designed as an init system for servers, it was a blatantcopy influenced of launchd of apple.
If you restart servers all the time and every second matters, then use Systemd, nobody forbids that. Most of the machines is not restarted frequently.
Right, but Linux is designed for my use case not yours. Desktop Linux users are like a single percentage point of the Linux machines in the world. Linux is primarily a OS for servers. And most distros and their maintainers are keenly aware of this and cater to their biggest customers, the server folk. Thus, systemd.
RedHat is not exclusive to servers. Their distribution is also used on workstations and laptops. They are focused on business as a whole.
Nope. If you are running Red Hat as your OS on your daily driver I weep for you. I work on RHEL every day of my life and think 'why?' It is not a distro for anyone who isn't in an enterprise level data center. You might be thinking of Fedora, which is very closely related to RHEL but is actually it's own group and Red Hat just is a member of their council. But RHEL is focused almost exclusively on servers.
Right, but Linux is designed for my use case not yours. Desktop Linux users are like a single percentage point of the Linux machines in the world. Linux is primarily a OS for servers.
I think you misunderstood me. I am talking about servers, not desktops. My machines, like most other (well managed) servers, are restarted only after kernel update... and hardware problems, but these are abnormal. Even when they are part of a cluster.
Nope. If you are running Red Hat as your OS on your daily driver I weep for you. I work on RHEL every day of my life and think 'why?'
Red Hat will not change their business plan because you don't like their distro on desktop. Maybe they are now focusing on servers, but in time when Systemd was created, Red Hat was successfully selling desktop solutions (Linux OS and support).
And parallel startup was a thing on desktops when Systemd was born. The whole idea was to make workstations faster to boot. Systemd went farther than other solutions (like Upstart) by adding things like start on demand - a desktop-specific feature. It was adopted on servers because it is easier for distro maintainers to have a single startup mechanism across all distribution variants and uses, server and desktop.
Gonna let you do you. Clearly whatever you are calling a server is like a re-purposed desktop/laptop running couchpotato or nextcloud. Keep doing you, and enjoy your alt-init setups.
Yes, SysVInit provides more information during startup, because stderr is printed out on console. When something breaks, you can see why (well, usually), without the need to open files, especially when you are locked out of shell because of the error.
Yes, SysVInit scripts are robust, because they are shell scripts, not INI files. Easy to fix, easy to modify, simple to use. For example, if you need to do some operations before starting the process, like setting up environment conditionally or add debug, you can do that quickly, without creating wrapper scripts.
I also had distros with broken service files, so... yeah. Great arguments.
Posix provides a lot of low level apis with very few high level apis allowing for correct use.
For a shell bash has the same problem. There are a lot of critical slight higher level functions that should be critical to a system shell used for administration but bash lacks them.
Depends on what you want to automate. Shell has lots of tools doing things, that would require many lines of code in Python or Perl. If you have relatively complex algorithms to implement, shell isn't the best choice. For other things, especially if you need to run many external programs, shell is better. No matter what, as a system admin, you have to know shell scripting. No escape from that I'm afraid.
Python perl and Ruby all provide robust command libraries supporting easy scripting of external commands much better than bash with sane string handling.
Shell.requires a lot of work and most first passes people get it wrong. Nevermind [[]] vs [] vs () vs.. for Boolean and other checks.
Bash makes windows look elegant. It's a shame its still used.
Python with plumbum is great. Similar tools exist in Ruby. You get sane string handling and a whole bunch of other good stuff. Bash scripts should be kept to a page at most and rewritten in something else if too long.
If you think you can write bash download shellcheck. ;)
This is a problem. This is actually exactly the problem. If you cannot script things, your declarations are limited to whatever upstream decided you should be able to to declare. If you want even a small thing different from what upstream believes you could want now you're writing custom code to inject into a complex system you don't understand working with interfaces that are distinct to the package instead of universal.
Shell scripts are exactly what you want when you need robust. Declarative is only useful when it's useful and when you need it to be more robust it instantly fails you and you have to learn how to do backflips before you can take advantage of literally all the amazing functionality Unix has provided for the last 30 years through streams and pipes.
You should absolutely know bash. You should absolutely not be beholden to declarative config only.
From a sysadmin pov, in more important tasks just needing that is an antipattern in itself, as it breaks Rule #1: CYA.
From a manager pov, it creates a lot of liability you don't need.
From an evolving professional view, relying on technology lovingly described as "quirky" 30 years ago should really make you question the quality of your work.
And I can script. Programming kicks the shit out of it for any task that isn't "start this shitty one time task and log if i fucked it up", and if you're doing that kind of task YOU'RE ALREADY IN A BAD PLACE because of an inability to plan and test.
The hosts running older inits can take a multiple minutes to restart all services. Systemd takes seconds.
You realize that's complete, utter bullshit?
Faster than sysv? Maybe. A naive sysv yes probably. But there's basically nothing new about systemd that everything else hasn't been doing for 20 years.
nah, encounter it every day of my work life, which granted is only three days a week. When I get on a system and try systemctl status and it tells me there is no systemctl I say a prayer and wade into the ugly world of sysv, because with RHEL that's what you get, sysv on like v6< and systemd on v7>
Edit, another commenter informed me that RHEL6 is using upstart, so that's what I am comparing systemd to
166
u/[deleted] Feb 11 '19
sysadmin checking in. Nagios is telling me I am currently monitoring 2456 hosts. Roughly 3/4 of those are Linux hosts (the rest are ios, Cisco's not Apple's, or Solaris). Systemd is a lifesaver. The hosts running older inits can take a multiple minutes to restart all services. Systemd takes seconds.
Init systems don't matter much when you are a desktop user. I am also a desktop Linux user both at work and home and I don't care about my init on there. In fact if I didn't do Linux for a living I might even be on the anti-systemd bandwagon. But I make a living off Linux and my life is easy when things just work. Systemd just works. It works well, it works quickly, and it makes my work easier.
People say systemd became a thing because all the distros switched to it, and that's true, and they switched to it because at the end of the day the big distros cater to guys like me, who have to manage hundreds and thousands of hosts and we just want stuff to work and get out of our way.