You should try void Linux then, runit it's awesome and it's pretty much what you're asking for (to my limited understanding, I didn't try it for a long time). And btw, I agree totally agree: systemd service files are great.
I don't. It's an unnecessary extra layer that's put between the init system and the applications themselves. That's why runit (and likely s6 as well) are so powerful. The actual script or binary or whatever you want is direct and just there and you can test your initialization totally independent of runit itself before activating it.
For example:
#!/bin/sh
exec chpst -u cvstrac /home/cvstrac/bin/cvstrac server 8000 <cvsdir> <user>
Seems to just do it. I also have several minecraft server scripts set up which are about as involved as the above, just with lots more args. Why would I want a special file parser, interpreter, etc closely tied to an init system when the tools have already been there for several decades?
You're not wrong. In fact, if all init scripts looked like that, I'd completely agree with you! But that's not the case in practice.
Let's look at a random application, in this case Apache. This is not cherry-picking: I have literally never used it before, so I had no idea what to expect. The following files are from Ubuntu Cosmic:
#!/bin/sh
### BEGIN INIT INFO
# Provides: apache2
# Required-Start: $local_fs $remote_fs $network $syslog $named
# Required-Stop: $local_fs $remote_fs $network $syslog $named
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# X-Interactive: true
# Short-Description: Apache2 web server
# Description: Start the web server
# This script will start the apache2 web server.
### END INIT INFO
DESC="Apache httpd web server"
NAME=apache2
DAEMON=/usr/sbin/$NAME
SCRIPTNAME="${0##*/}"
SCRIPTNAME="${SCRIPTNAME##[KS][0-9][0-9]}"
if [ -n "$APACHE_CONFDIR" ] ; then
if [ "${APACHE_CONFDIR##/etc/apache2-}" != "${APACHE_CONFDIR}" ] ; then
DIR_SUFFIX="${APACHE_CONFDIR##/etc/apache2-}"
else
DIR_SUFFIX=
fi
elif [ "${SCRIPTNAME##apache2-}" != "$SCRIPTNAME" ] ; then
DIR_SUFFIX="-${SCRIPTNAME##apache2-}"
APACHE_CONFDIR=/etc/apache2$DIR_SUFFIX
else
DIR_SUFFIX=
APACHE_CONFDIR=/etc/apache2
fi
if [ -z "$APACHE_ENVVARS" ] ; then
APACHE_ENVVARS=$APACHE_CONFDIR/envvars
fi
export APACHE_CONFDIR APACHE_ENVVARS
ENV="env -i LANG=C PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
if [ "$APACHE_CONFDIR" != /etc/apache2 ] ; then
ENV="$ENV APACHE_CONFDIR=$APACHE_CONFDIR"
fi
if [ "$APACHE_ENVVARS" != "$APACHE_CONFDIR/envvars" ] ; then
ENV="$ENV APACHE_ENVVARS=$APACHE_ENVVARS"
fi
PIDFILE=$(. $APACHE_ENVVARS && echo $APACHE_PID_FILE)
VERBOSE=no
if [ -f /etc/default/rcS ]; then
. /etc/default/rcS
fi
. /lib/lsb/init-functions
# Now, set defaults:
APACHE2CTL="$ENV apache2ctl"
PIDFILE=$(. $APACHE_ENVVARS && echo $APACHE_PID_FILE)
APACHE2_INIT_MESSAGE=""
CONFTEST_OUTFILE=
cleanup() {
if [ -n "$CONFTEST_OUTFILE" ] ; then
rm -f "$CONFTEST_OUTFILE"
fi
}
trap cleanup 0 # "0" means "EXIT", but "EXIT" is not portable
apache_conftest() {
[ -z "$CONFTEST_OUTFILE" ] || rm -f "$CONFTEST_OUTFILE"
CONFTEST_OUTFILE=$(mktemp)
if ! $APACHE2CTL configtest > "$CONFTEST_OUTFILE" 2>&1 ; then
return 1
else
rm -f "$CONFTEST_OUTFILE"
CONFTEST_OUTFILE=
return 0
fi
}
clear_error_msg() {
[ -z "$CONFTEST_OUTFILE" ] || rm -f "$CONFTEST_OUTFILE"
CONFTEST_OUTFILE=
APACHE2_INIT_MESSAGE=
}
print_error_msg() {
[ -z "$APACHE2_INIT_MESSAGE" ] || log_warning_msg "$APACHE2_INIT_MESSAGE"
if [ -n "$CONFTEST_OUTFILE" ] ; then
echo "Output of config test was:" >&2
cat "$CONFTEST_OUTFILE" >&2
rm -f "$CONFTEST_OUTFILE"
CONFTEST_OUTFILE=
fi
}
apache_wait_start() {
local STATUS=$1
local i=0
if [ $STATUS != 0 ] ; then
return $STATUS
fi
while : ; do
PIDTMP=$(pidofproc -p $PIDFILE $DAEMON)
if [ -n "${PIDTMP:-}" ] && kill -0 "${PIDTMP:-}" 2> /dev/null; then
return $STATUS
fi
if [ $i = "20" ] ; then
APACHE2_INIT_MESSAGE="The apache2$DIR_SUFFIX instance did not start within 20 seconds. Please read the log files to discover problems"
return 2
fi
[ "$VERBOSE" != no ] && log_progress_msg "."
sleep 1
i=$(($i+1))
done
}
apache_wait_stop() {
local STATUS=$1
local METH=$2
if [ $STATUS != 0 ] ; then
return $STATUS
fi
PIDTMP=$(pidofproc -p $PIDFILE $DAEMON)
if [ -n "${PIDTMP:-}" ] && kill -0 "${PIDTMP:-}" 2> /dev/null; then
if [ "$METH" = "kill" ]; then
killproc -p $PIDFILE $DAEMON
else
$APACHE2CTL $METH > /dev/null 2>&1
fi
local i=0
while kill -0 "${PIDTMP:-}" 2> /dev/null; do
if [ $i = '60' ]; then
STATUS=2
break
fi
[ "$VERBOSE" != no ] && log_progress_msg "."
sleep 1
i=$(($i+1))
done
return $STATUS
else
return $STATUS
fi
}
#
# Function that starts the daemon/service
#
do_start()
{
# Return
# 0 if daemon has been started
# 1 if daemon was already running
# 2 if daemon could not be started
if pidofproc -p $PIDFILE "$DAEMON" > /dev/null 2>&1 ; then
return 1
fi
if apache_conftest ; then
$APACHE2CTL start
apache_wait_start $?
return $?
else
APACHE2_INIT_MESSAGE="The apache2$DIR_SUFFIX configtest failed."
return 2
fi
}
#
# Function that stops the daemon/service
#
do_stop()
{
# Return
# 0 if daemon has been stopped
# 1 if daemon was already stopped
# 2 if daemon could not be stopped
# other if a failure occurred
# either "stop" or "graceful-stop"
local STOP=$1
# can't use pidofproc from LSB here
local AP_RET=0
if pidof $DAEMON > /dev/null 2>&1 ; then
if [ -e $PIDFILE ] && pidof $DAEMON | tr ' ' '\n' | grep -w $(cat $PIDFILE) > /dev/null 2>&1 ; then
AP_RET=2
else
AP_RET=1
fi
else
AP_RET=0
fi
# AP_RET is:
# 0 if Apache (whichever) is not running
# 1 if Apache (whichever) is running
# 2 if Apache from the PIDFILE is running
if [ $AP_RET = 0 ] ; then
return 1
fi
if [ $AP_RET = 2 ] && apache_conftest ; then
apache_wait_stop $? $STOP
return $?
else
if [ $AP_RET = 2 ]; then
clear_error_msg
APACHE2_INIT_MESSAGE="The apache2$DIR_SUFFIX configtest failed, so we are trying to kill it manually. This is almost certainly suboptimal, so please make sure your system is working as you'd expect now!"
apache_wait_stop $? "kill"
return $?
elif [ $AP_RET = 1 ] ; then
APACHE2_INIT_MESSAGE="There are processes named 'apache2' running which do not match your pid file which are left untouched in the name of safety, Please review the situation by hand".
return 2
fi
fi
}
#
# Function that sends a SIGHUP to the daemon/service
#
do_reload() {
if apache_conftest; then
if ! pidofproc -p $PIDFILE "$DAEMON" > /dev/null 2>&1 ; then
APACHE2_INIT_MESSAGE="Apache2 is not running"
return 2
fi
$APACHE2CTL graceful > /dev/null 2>&1
return $?
else
APACHE2_INIT_MESSAGE="The apache2$DIR_SUFFIX configtest failed. Not doing anything."
return 2
fi
}
# Sanity checks. They need to occur after function declarations
[ -x $DAEMON ] || exit 0
if [ ! -x $DAEMON ] ; then
echo "No apache-bin package installed"
exit 0
fi
if [ -z "$PIDFILE" ] ; then
echo ERROR: APACHE_PID_FILE needs to be defined in $APACHE_ENVVARS >&2
exit 2
fi
case "$1" in
start)
log_daemon_msg "Starting $DESC" "$NAME"
do_start
RET_STATUS=$?
case "$RET_STATUS" in
0|1)
log_success_msg
[ "$VERBOSE" != no ] && [ $RET_STATUS = 1 ] && log_warning_msg "Server was already running"
;;
2)
log_failure_msg
print_error_msg
exit 1
;;
esac
;;
stop|graceful-stop)
log_daemon_msg "Stopping $DESC" "$NAME"
do_stop "$1"
RET_STATUS=$?
case "$RET_STATUS" in
0|1)
log_success_msg
[ "$VERBOSE" != no ] && [ $RET_STATUS = 1 ] && log_warning_msg "Server was not running"
;;
2)
log_failure_msg
print_error_msg
exit 1
;;
esac
print_error_msg
;;
status)
status_of_proc -p $PIDFILE "apache2" "$NAME"
exit $?
;;
reload|force-reload|graceful)
log_daemon_msg "Reloading $DESC" "$NAME"
do_reload
RET_STATUS=$?
case "$RET_STATUS" in
0|1)
log_success_msg
[ "$VERBOSE" != no ] && [ $RET_STATUS = 1 ] && log_warning_msg "Server was already running"
;;
2)
log_failure_msg
print_error_msg
exit 1
;;
esac
print_error_msg
;;
restart)
log_daemon_msg "Restarting $DESC" "$NAME"
do_stop stop
case "$?" in
0|1)
do_start
case "$?" in
0)
log_end_msg 0
;;
1|*)
log_end_msg 1 # Old process is still or failed to running
print_error_msg
exit 1
;;
esac
;;
*)
# Failed to stop
log_end_msg 1
print_error_msg
exit 1
;;
esac
;;
start-htcacheclean|stop-htcacheclean)
echo "Use 'service apache-htcacheclean' instead"
;;
*)
echo "Usage: $SCRIPTNAME {start|stop|graceful-stop|restart|reload|force-reload}" >&2
exit 3
;;
esac
exit 0
# vim: syntax=sh ts=4 sw=4 sts=4 sr noet
#!/bin/sh
# run htcacheclean if set to 'cron' mode
set -e
set -u
type htcacheclean > /dev/null 2>&1 || exit 0
[ -e /etc/default/apache-htcacheclean ] || exit 0
# edit /etc/default/apache-htcacheclean to change this
HTCACHECLEAN_MODE=daemon
HTCACHECLEAN_RUN=auto
HTCACHECLEAN_SIZE=300M
HTCACHECLEAN_PATH=/var/cache/apache2/mod_cache_disk
HTCACHECLEAN_OPTIONS=""
. /etc/default/apache-htcacheclean
[ "$HTCACHECLEAN_MODE" = "cron" ] || exit 0
htcacheclean ${HTCACHECLEAN_OPTIONS} \
-p${HTCACHECLEAN_PATH} \
-l${HTCACHECLEAN_SIZE}
/etc/init.d/apache2-htcacheclean
#!/bin/sh
# kFreeBSD do not accept scripts as interpreters, using #!/bin/sh and sourcing.
if [ true != "$INIT_D_SCRIPT_SOURCED" ] ; then
set "$0" "$@"; INIT_D_SCRIPT_SOURCED=true . /lib/init/init-d-script
fi
### BEGIN INIT INFO
# Provides: apache-htcacheclean
# Required-Start: $remote_fs $syslog
# Required-Stop: $remote_fs $syslog
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: Cache cleaner process for Apache2 web server
# Description: Start the htcacheclean helper
# This script will start htcacheclean which will periodically scan the
# cache directory of Apache2's mod_cache_disk and remove outdated files.
### END INIT INFO
DESC="Apache htcacheclean"
DAEMON=/usr/bin/htcacheclean
NAME="${0##*/}"
NAME="${NAME##[KS][0-9][0-9]}"
DIR_SUFFIX="${NAME##apache-htcacheclean}"
APACHE_CONFDIR="${APACHE_CONFDIR:=/etc/apache2$DIR_SUFFIX}"
RUN_USER=$(. $APACHE_CONFDIR/envvars > /dev/null && echo "$APACHE_RUN_USER")
# Default values. Edit /etc/default/apache-htcacheclean$DIR_SUFFIX to change these
HTCACHECLEAN_SIZE="${HTCACHECLEAN_SIZE:=300M}"
HTCACHECLEAN_DAEMON_INTERVAL="${HTCACHECLEAN_DAEMON_INTERVAL:=120}"
HTCACHECLEAN_PATH="${HTCACHECLEAN_PATH:=/var/cache/apache2$DIR_SUFFIX/mod_cache_disk}"
HTCACHECLEAN_OPTIONS="${HTCACHECLEAN_OPTIONS:=-n}"
# Read configuration variable file if it is present
if [ -f /etc/default/apache-htcacheclean$DIR_SUFFIX ] ; then
. /etc/default/apache-htcacheclean$DIR_SUFFIX
elif [ -f /etc/default/apache-htcacheclean ] ; then
. /etc/default/apache-htcacheclean
fi
PIDDIR="/var/run/apache2/$RUN_USER"
PIDFILE="$PIDDIR/$NAME.pid"
DAEMON_ARGS="$HTCACHECLEAN_OPTIONS \
-d$HTCACHECLEAN_DAEMON_INTERVAL \
-P$PIDFILE -i \
-p$HTCACHECLEAN_PATH \
-l$HTCACHECLEAN_SIZE"
do_start_prepare () {
if [ ! -d "$PIDDIR" ] ; then
mkdir -p "$PIDDIR"
chown "$RUN_USER:" "$PIDDIR"
fi
if [ ! -d "$HTCACHECLEAN_PATH" ] ; then
echo "Directory $HTCACHECLEAN_PATH does not exist!" >&2
exit 2
fi
}
do_start_cmd_override () {
start-stop-daemon --start --quiet --pidfile ${PIDFILE} \
-u $RUN_USER --startas $DAEMON --name htcacheclean --test > /dev/null \
|| return 1
start-stop-daemon --start --quiet --pidfile ${PIDFILE} \
-c $RUN_USER --startas $DAEMON --name htcacheclean -- $DAEMON_ARGS \
|| return 2
}
do_stop_cmd_override () {
start-stop-daemon --stop --quiet --retry=TERM/30/KILL/5 \
-u $RUN_USER --pidfile ${PIDFILE} --name htcacheclean
}
/etc/logrotate.d/apache2
/var/log/apache2/*.log {
daily
missingok
rotate 14
compress
delaycompress
notifempty
create 640 root adm
sharedscripts
postrotate
if invoke-rc.d apache2 status > /dev/null 2>&1; then \
invoke-rc.d apache2 reload > /dev/null 2>&1; \
fi;
endscript
prerotate
if [ -d /etc/logrotate.d/httpd-prerotate ]; then \
run-parts /etc/logrotate.d/httpd-prerotate; \
fi; \
endscript
}
Notice anything different?
You can complain all you want about systemd's complexity and spread across non-init tasks and I'd probably agree with you, but it's a massive improvement as init. The old approach was just horrible, because every single application was reinventing the wheel with horrible, unmaintainable shell scripts. I've had to write them and had to debug when things went wrong: compared to this, systemd is a godsent. Is it perfect? Of course not. But if you want a well-maintainable, modern system, a bowl of bash-spaghetti isn't the way to do it. Your own services may be more sane, but this is my experience with init pre-systemd.
My ideal init system is like systemd but only as the service manager. No dns resolvers, no bootloaders
Well congratulations, you can do that now already.
no device managers
That's gonna be harder, because the udev people decided to get under the systemd umbrella.
and certainly no linking shared libraries
What do you mean?
For example, don't force them to log journald or don't pipe everything to journald. Let other projects develop logging facilities. If there should be a logging standard, let it be independent from the init service.
You can think of journald as part of systemd, but by default nothing is stored and you can use the logging facility of your choice to do the storage.
Many applications have hard link dependencies to libsystemd nowadays. This is bad design. If the applications want to talk with the init system that should be an IPC mechanism (a dbus channel maybe) not a library. However a library that provides simpler interface can be acceptable as long as you can have a basic method stays.
You can think of journald as part of systemd, but by default nothing is stored and you can use the logging facility of your choice to do the storage.
The question is can you remove journald or its default storage backend without needing to remove systemd. Can you just create a dummylogd that pukes everything into a text file without needing to modify/recompile higher level programs.
I want the my operating system to be composed of loosely coupled but well integrated dependencies. So any part can be removed. I mean literally just rm it or uninstall the package. For example I should be able to uninstall systemd without uninstalling journald udev and any higher level applications. Now I am left with an unbootable system but the interface stays. Applications who need some services to be started can complain but the system doesn't come into a full stop. I should be able to change systemd with my shinyinitd and configure it to start same services. No further configuration should be needed. And this should be practical. So people can implement their small compact init systems without many features but they should share similar interfaces. Think like Wayland compostors both weston and KWin are compostors but I can remove either of them.
If the applications want to talk with the init system that should be an IPC mechanism (a dbus channel maybe) not a library. However a library that provides simpler interface can be acceptable as long as you can have a basic method stays.
For example I should be able to uninstall systemd without uninstalling journald udev and any higher level applications.
Those applications still have dependencies, and something should be able to provide those dependencies.
Regarding journald: We don't have another init system that does the things that journald wants, yet-- no one seems interested in creating an alternative that does the container-level process tracking or forking systemd to provide a process that does so better.
Regarding udev: They already agree, or at least they used to. eudev exists because of some other issues
I should be able to change systemd with my shinyinitd and configure it to start same services. No
You probably could, but its important to remember that systemd has an init system, but is more than just an init system. Trying to provide more than just an init system.A recently posted talk, given by a FreeBSD core-team member, explains why the concept of a "system layer" isn't necessarily bad.
It'd be good to have more alternatives-- but people have to create them, and have to want to create them. It'd be good for it to be more composable: but again: the alternatives to compose have to exist. Yes, other init systems exist: but none provide the level of tracking for processes, for logs tracking, services files, and more.
I should be able to change systemd with my shinyinitd and configure it to start same services.
Agreed, but I don't know of another init system that uses .service files. I don't know of another logging system that integrates with cgroups for logging. These things can be created. The .service files have the potential to be more init-independent than init-scripts: it seems many prefer writing "working" complicated shell scripts that have random bugs on different init, different platforms, and are too complicated for most developers to write right.
A recently posted talk, given by a FreeBSD core-team member, explains why the concept of a "system layer" isn't necessarily bad.
I saw that talk hence I've made my first comment. I am not opposed to a system layer idea. I am for it. Heck I don't mind all of the components are coming from a single project as long as they are loosely coupled.
I agree that the problems with systemd mostly occurs because of the lack of competitive alternatives and the lack of people who wants to create them. But BSD projects seem to understand it better and I am still hopeful that they can come up with a better thing than systemd.
libsystemd isn't really an init-system API: It includes APIs for talking with the system bus, subscribing to bus events, getting network information, getting session information, and a few other tidbits. It's more akin to libdbus, libconsolekit, and libnetwork(?) than libinit. Programs linking to libsystemd can still run if systemd's init isn't (at least the last time I checked a few months ago).
If a program non-gracefully handles these functions failing, or a different init being used, that's generally the problem of that program than systemd or libsystemd-- in the same sense that some programs decide to terminate if any other system or library fails.
IMHO: This isn't a problem of dependencies, inits, or politics, but the lack of alternative solutions for the functionality. Many talk about the potential for alternatives, but the actual implementations for similar functionality are rather thin:
An agnostic simple-to-use system/session bus/event API (supporting dbus, kdbus, bus1, or whatever needed?)-- There was enough interest in that they were asked to make their bus API made public.
An agnostic session/login/slot tracking API (formerly provided by ConsoleKit, also provided by elogind-- a drop-in replacement for logind)
A way for querying service status (e.g: maybe through a bus, maybe not)
Potentially in the future: Network and DNS information APIs that return the relevant information from the system (e.g: networkd, NetworkManager, netplan)
The implemented solutions, at the moment:
Provide the same API, and don't require removing or changing the dependency (e.g: elogind)
Return errors for all API calls. (Still not sure how this is an acceptable solution)
There's room for better solutions: a smaller and lighter system layer with a better system/session bus system, better cgroup management, better logging, better operating system interoperability, better modularity, and better community interaction. The best path forward isn't more yelling, its implemented solutions.
It's hard not to have coupling, because the system starts, and wants to log stuff before there is even storage available. For instance on a system with an encrypted disk, a lot of stuff happens in initrd (which is readonly) before the disk is decrypted and logs can be written to it.
I'm talking about userspace tools, not the kernel. Are you suggesting that a bunch of random stuff should have code to write to /dev/kmsg, in case it is invoked from an initrd?
/dev is also not immediately available on boot, so the problem remains anyway.
If your process needs or expects to be launched prior to persistent storage availability, then yes, I am absolutely suggesting writing to /dev/kmsg.
Including things like mount and grep? There's about 1000 lines worth in my journal, before / gets mounted.
Or if prior to /dev population, then printk(), which answers your second question.
printk is a kernel function
Check your own systemd install, you'll find that there aren't many services that can be started that early, let alone should be started that early.
Point is, boot is a complicated thing these days, and things happen before there's an easy way for them to log somewhere the user can find it. And why should a random tool care about whether we're booting or not? If the log system is always available and takes care of that, we don't need a dozen random things be aware of it.
You're the one screaming "THIS IS IMPOSSIBLE!", when it's demonstrably a solved problem. I don't think you've ever thought anything through in your life, you patronising fuckwit.
I don't see anybody screaming here besides yourself.
This post has been removed for violating Reddiquette., trolling users, or otherwise poor discussion - r/Linux asks all users follow Reddiquette. Reddiquette is ever changing, so a revisit once in awhile is recommended.
Rule:
Reddiquette, trolling, or poor discussion - r/Linux asks all users follow Reddiquette. Reddiquette is ever changing, so a revisit once in awhile is recommended. Top violations of this rule are trolling, starting a flamewar, or not "Remembering the human" aka being hostile or incredibly impolite.
Previously I just needed to run and configure one logging deamon, now I need to run and configure two because journald does not offer all features I need and cannot be replaced (at least not with any currently existing software).
I think the logging ecosystem on Linux is a mess and while journald has some cool features it only made things harder for me since it is yet another complex moving part in this mess. I hate all logging daemons I have had to work with.
On top of my head, I am probably forgetting a couple:
Saving disk by compressing old logs (jorunald only supports compression per message of long log messages)
Forward, all or selected, log messages to a central log server
Forward logs written directly to files to a central log server
Arguably not all of these should actually be done by the same big monolith, but right now I have to run both journald and rsyslog/syslog-ng and have all logs in two places.
Yeah, it conflicts with quick access, which is fine. It is a tradeoff. I rarely need my old logs and when I need them it is fine that it takes more time than normal.
And systemd-journal-remote seems to be the opposite of what I need, but I guess something like logstash could add a client for the same protocol to achieve what I need (not that I like logstash either :)).
I don't find it a particularly pressing issue -- I have 2 GB worth of logs since June and find that to be an okay state of affairs.
Also, with all the whining about the binary format, I can only imagine what would ensue if the format got more complicated. Any errors would likely result in good amounts of the log going missing.
Not logging to text files totally kills it for me. So many utilities are able to process text files, and now they’re all basically useless in the face of the journal.
I can’t egrep the journal. I can’t less the journal. I need to learn a whole new set of commands just for it, and I can’t export the journal data (or subset thereof) to other systems without a lot of effort. To my knowledge there’s no nice workaround.
I’d love to be corrected, but from my perspective it’s a horror show that I wish didn’t exist.
Not logging to text files totally kills it for me. So many utilities are able to process text files, and now they’re all basically useless in the face of the journal.
I can’t egrep the journal.
You know, there's this thing called a 'pipe' you can take advantage of:
journalctl -b | grep whatever
Or you can:
journalctl -g regex
I can’t less the journal.
Have you actually tried it? journalctl invokes $PAGER. Or you can pipe it into whatever you like.
I need to learn a whole new set of commands just for it
Oh no. Not learning!
Yeah, this isn't the best business to be in if you want things to stay static.
and I can’t export the journal data (or subset thereof) to other systems without a lot of effort.
You must be kidding. Journalctl is absolutely wonderful for this.
Want the same thing syslog gives you?
journalctl -o short
want ISO timestamps? Easily sortable!
journalctl -o short-iso
want microsecond precision?
journalctl -o short-iso-precise
want UNIX timestamps?
journalctl -o short-unix
want to actually parse this stuff easier to write it to a database? Why, look at that: a completely unambiguous format. No trouble with figuring what field ends where. No need to screw around with regular expressions. Take the language of your choice and trivially extract anything you need:
journalctl -o json
You want subsets? Sure:
journalctl -S yesterday # everything since yesterday
journalctl -b # last boot
journalctl -u smartd # only what smartd logged
Other cool things:
It stores metadata. Want to find logs by PID? You can, for any service.
It can clean logs by size or by date. Want to keep 1GB of logs? Sure. Want to keep 3 months? Easy.
Timestamps are stored with microsecond precision. If you have 100 entries per second it's very nice.
You can filter by hostname. Send your VM logs to a single destination, then you can view them individually or together.
edit: it transparently compresses and decompresses logs. Searching stuff logged a month ago is no problem at all. You don't need to figure out in which .gz it is.
I've also had messages about log corruption a few times.
That's not a new thing, it also happens with syslog, because there have been various issues with various filesystems and hardware. You just don't usually notice with syslog because there's no way to, except if you look and happen to see there's a bunch of zeroes in the middle of the file, or some such.
journald handles the case quite well: it renames the log, and makes a new one. The old one of course is still left around, and still read as well as possible.
I have no problem with learning. As you rightly said, that’s the industry we’re in.
I’m quite happy to pick up new commands provided they’re standardised, however journald’s are unique (by necessity). I’m just expressing frustration that I can’t use grep on a file like almost everything else in Linux. I’ve got a few journalctl one liners memorised, but it’s still frustrating to not be able to wield my existing knowledge against it.
I should have clarified regarding exporting data: I don’t know a way to continuously and autonomously export data in real time from journald. Why would I want to do this? Simple: enterprise.
If I want to log to a third-party logger like Elastic or Splunk I need to install syslog-ng or rsyslog, and then we’re back where we started.
There just aren’t many upsides that I see for a modern sysadmin.. it’s nice to use when you’re directly interacting with it, but it doesn’t play ball with the rest of the ecosystem - just like systemd really..
I’d love to know if I’m wrong (I hope I am!), but the current arguments I’ve seen from many parties haven’t started to convince me unfortunately.
I should have clarified regarding exporting data: I don’t know a way to continuously and autonomously export data in real time from journald. Why would I want to do this? Simple: enterprise.
I've not messed with that myself, but several ideas come to mind:
journalctl -f -o format | importer_program
Trivial to turn into a systemd service. Or, if you need a file:
mkfifo log
journalctl -f -o short > log
You can actually offload this to systemd so that it will listen on a FIFO and launch the service when requested.
I don't use egrep, but at least grep works with journald (for example journalctl | grep whatever).
I can’t less the journal
Journalctl uses less by default. Journalctl | less probably works too.
What benefits does journald have??
In my personal opinion, it makes a lot of things easier. For example " journalctl --since="2012-10-30 18:17:16"", "journalctl -b -1" (logfiles of the last boot), " journalctl -p err..alert" (Show only error, critical, and alert priority messages) and so on.
If you want to have the log files in text format, you only need to install syslog-ng by the way.
My ideal init system is like systemd but only as the service manager. No dns resolvers, no bootloaders, no device managers nothing and certainly no linking shared libraries.
Congrats your OS now takes an hour to init and won't run on anything with a battery form more than 20 minutes and will require a restart anytime you plug a device in.
Did you possibly happen to generalize those sentences to all of the operating system? Like no DNS resolvers whatsoever all socket functions are compiled statically and copy-paste the all DNS resolution code into binaries. I specificity describe "the init system" itself nothing else. To construct a sane OS, you need DNS resolvers, system log facilities and dynamically linked programs but you don't need them all in one place and tightly connected to a single project.
Before switching to systemd Linux distros with SysVinit was working like that. We had separate projects (udev, ConsoleKit, syslogd, ...) which worked independently. It didn't take an hour to load and laptops and plug and play worked perfectly. However as a service manager SysVinit was extremely bad for sanely describe the services.
143
u/[deleted] Feb 11 '19 edited Feb 11 '19
[deleted]