He's hit the nail on the head - systemd's fundamental design is not appropriate for the server environment:
"I have to provide a system that runs reliably and can easily be reasoned about and yet I have to build it on distributions created by people who consider how long it takes to get to the fucking GDM login screen and if shutting the laptop lid will cause the system to hibernate properly or not."
I'm in the same boat - I wind up in a situation where I find myself missing the features, stability, coherance, and integration of systemd at least once a week in my day job, where we currently use Ubuntu with upstart.
For me, it's a combination of things:
1. Proper dependency management, where I specify dependencies as they are, rather than flattening the dependency graph.
2. Proper service supervision. I've had upstart lose track of running processes, which is silly - you had one job!
3. Ability to decouple the init system's view of "process is running" from "process is ready/live". This is nice if you want to await liveness by letting the system supervisor tell you if the service is live or not, rather than writing a bunch of external scripts or checks.
4. A single place to modify the way system services are run, and unified config for e.g. resource limits, and tools for working with them.
5. Simple exploration and interleaving of logs from various communicating services via journalctl, which is insanely helpful for debugging things in a distributed system. You can see the calls come in from the network and bounce between services, and see exactly where something goes wrong. You could do this with other solutions, but I get this out of the box with journalctl, and I can look in more detail at any of the services, or when I'm checking a service's status.
It's true that many of these things could perhaps have been done in other ways, but the fact is that the systemd developers did the work to make it happen and now I can focus on my product, rather than on learning a bazillion pieces of plumbing to enable me to ship product. I'm fine adapting a few things to run via unit files instead of shell scripts if it means I can ship more correct and more reliable product in less time.
systemd is less stable than upstart (and sysvinit, obviusly), "it lacks maturity" says the author. You've had several problems with upstart, but count yourself lucky, certain problems with systemd will take down the whole system, and make it unbootable too!
systemd is not reliable, unlike eg daemontools, the developers are busy adding new features and don't care at all about old bugs. They say if you're not using one of the latest kernels, tough luck. So you're continuously debugging systemd "modern innovations" instead of focusing on your product.
If you're using Ubuntu, let's hope that by the time it gets adopted, systemd has evolved into a functional, dependable component. I wouldn't bet my product on that, though..
I've been also using systemd since Arch switched over, I've discovered bugs and vulnerabilities. But the thing is, if you're happy with systemd and works well for you, great! I've seen this very prevalent attitude in systemd users, the less they know, more fervently defend systemd, even if they think it sucks, they tell you it's the best thing ever because they're using it, and they always have to use the best.
Arch is bleeding edge, if you've never seen a bug, you haven't been paying attention. So, again, you've never suffered a problem with systemd. Now do a web search, and in less time it takes to write "[citation needed]", you'll see that this is real. Sorry, but systemd has bugs, just like every other program, but by being so intermingled with the kernel, the consequences are much worse.
If you know of some old neglected bugs like you keep saying then point some out. I would love to see an actual source on something other than the constant made up retorts of "PID 1 now has an HTTP server" or "Now kernel panics display a QR code".
I definitely don't want to hide problems with my init systems behind a supervisor that has its own host of problems. The union of those two products does not make a more stable, easy-to-reason-about system.
Granted I could choose something better than monit, but then why add another layer to hide brokenness, when we have the capability of doing it correctly in the first place.
Being able to see what started what, cgroups, which binary produced what output, automatic process restarting is what I like for using on a server. I also like the nice timers output and that I don't need an additional cron or network configuration utility.
The problem is that in many server environments, particularly at scale, the speed of the bootstrap processes of the OS doesn't matter. One of the goals of systemd is to make the bootstrap process faster, which is irrelevant for those (non-desktop) use cases. So you see, systemd is solving problems, and making trade-offs to do it, that just don't matter for the non-desktop use case. I can of course choose not to install gdm, but that doesn't change the fact that systemd was designed (you might say over-designed) with the desktop use case in mind. Because of that, I have to live with those trade offs, even though my use case does not benefit. Since all major Linux distros are deciding to adopt systemd, it will be very difficult to "roll your own" and use something else, especially since the rest of the userland will assume that everyone is doing things the systemd way, this making it even harder to go against the flow.
If you're spinning up a lot of VMs, or paying by the hour or watt, you might care about how fast a server boots. Anyway lots of other init systems do parallel boot, not just systemd. So you don't have to use systemd just to get a fast boot time.
> Since all major Linux distros are deciding to adopt systemd...
Slackware is a major distro. So I expect everyone who opposes systemd to migrate to Slackware, and I'll get even more slackbuilds to browse through on SBO, right?
The idea that you should artificially separate OS into server and desktop, solely so you can charge more for the license for server OS, is a false dichotomy that needs to go away and only exists in proprietary commercial OS. It exists solely in the marketing sphere, and only intersects with the technical sphere WRT including artificial limits to ruin performance on a system that has a workload marketing doesn't approve of.
I do not want parallel non-deterministic booting on a server. If it works I don't care because I don't reboot daily/hourly/for fun. Its a linux box not Windows. If it doesn't work then non-deterministic behavior makes race conditions and error messages very confusing.
There is an inherent architectural / philosophical violation where flexibility is considered "unix"ish yet it is forbidden under systemd domination. If you disagree with the above paragraph for init system on a server, thats OK, I think someone with that has the wrong opinion but I respect an honest disagreement. Under a unix-ish philosophy there have been about ten alt-inits over the past couple decades that'll parallel / non-deterministic init, so go ahead friend, more power to you, give one a try until your fingers get burnt. Sysvinit (or BSDs init) will be waiting for you when you return. This flexibility, this compatibility, is philosophically forbidden under systemd.
You will do it the systemd way, because we won the political battle, not for technical reasons (god knows systemd is not the first attempt at this architecture). Or you will be forced to leave. Well OK then. The future is looking very freebsd to me. So long and thanks for all the fish!
Was there even an argument presented in the section titled 'The invasion of “desktop linux”'? I couldn't find it. Where's the evidence that systemd is making design choices that are driven by desktop linux and detrimental to usage on servers?
"I’m going to state up front (and people are free to disagree with me) that I believe you cannot provide a distribution of Linux that is both designed for the 'server' and the 'desktop' and provide a product that is worth using on either."
I, uh, what? Someone let everybody know that, as I'm pretty sure all the popular server distros (I guess not CoreOS, but that's pretty new) work just fine as desktops. Heck, Ubuntu is a very popular server distro[1], and it's certainly a desktop distro. And it's not like sysvinit wasn't used on both the server and the desktop.
"I have to provide a system that runs reliably and can easily be reasoned about and yet I have to build it on distributions created by people who consider how long it takes to get to the fucking GDM login screen and if shutting the laptop lid will cause the system to hibernate properly or not."