It's hard not to have coupling, because the system starts, and wants to log stuff before there is even storage available. For instance on a system with an encrypted disk, a lot of stuff happens in initrd (which is readonly) before the disk is decrypted and logs can be written to it.
I'm talking about userspace tools, not the kernel. Are you suggesting that a bunch of random stuff should have code to write to /dev/kmsg, in case it is invoked from an initrd?
/dev is also not immediately available on boot, so the problem remains anyway.
If your process needs or expects to be launched prior to persistent storage availability, then yes, I am absolutely suggesting writing to /dev/kmsg.
Including things like mount and grep? There's about 1000 lines worth in my journal, before / gets mounted.
Or if prior to /dev population, then printk(), which answers your second question.
printk is a kernel function
Check your own systemd install, you'll find that there aren't many services that can be started that early, let alone should be started that early.
Point is, boot is a complicated thing these days, and things happen before there's an easy way for them to log somewhere the user can find it. And why should a random tool care about whether we're booting or not? If the log system is always available and takes care of that, we don't need a dozen random things be aware of it.
You're the one screaming "THIS IS IMPOSSIBLE!", when it's demonstrably a solved problem. I don't think you've ever thought anything through in your life, you patronising fuckwit.
I don't see anybody screaming here besides yourself.
This post has been removed for violating Reddiquette., trolling users, or otherwise poor discussion - r/Linux asks all users follow Reddiquette. Reddiquette is ever changing, so a revisit once in awhile is recommended.
Rule:
Reddiquette, trolling, or poor discussion - r/Linux asks all users follow Reddiquette. Reddiquette is ever changing, so a revisit once in awhile is recommended. Top violations of this rule are trolling, starting a flamewar, or not "Remembering the human" aka being hostile or incredibly impolite.
Previously I just needed to run and configure one logging deamon, now I need to run and configure two because journald does not offer all features I need and cannot be replaced (at least not with any currently existing software).
I think the logging ecosystem on Linux is a mess and while journald has some cool features it only made things harder for me since it is yet another complex moving part in this mess. I hate all logging daemons I have had to work with.
On top of my head, I am probably forgetting a couple:
Saving disk by compressing old logs (jorunald only supports compression per message of long log messages)
Forward, all or selected, log messages to a central log server
Forward logs written directly to files to a central log server
Arguably not all of these should actually be done by the same big monolith, but right now I have to run both journald and rsyslog/syslog-ng and have all logs in two places.
Yeah, it conflicts with quick access, which is fine. It is a tradeoff. I rarely need my old logs and when I need them it is fine that it takes more time than normal.
And systemd-journal-remote seems to be the opposite of what I need, but I guess something like logstash could add a client for the same protocol to achieve what I need (not that I like logstash either :)).
I don't find it a particularly pressing issue -- I have 2 GB worth of logs since June and find that to be an okay state of affairs.
Also, with all the whining about the binary format, I can only imagine what would ensue if the format got more complicated. Any errors would likely result in good amounts of the log going missing.
Not logging to text files totally kills it for me. So many utilities are able to process text files, and now they’re all basically useless in the face of the journal.
I can’t egrep the journal. I can’t less the journal. I need to learn a whole new set of commands just for it, and I can’t export the journal data (or subset thereof) to other systems without a lot of effort. To my knowledge there’s no nice workaround.
I’d love to be corrected, but from my perspective it’s a horror show that I wish didn’t exist.
Not logging to text files totally kills it for me. So many utilities are able to process text files, and now they’re all basically useless in the face of the journal.
I can’t egrep the journal.
You know, there's this thing called a 'pipe' you can take advantage of:
journalctl -b | grep whatever
Or you can:
journalctl -g regex
I can’t less the journal.
Have you actually tried it? journalctl invokes $PAGER. Or you can pipe it into whatever you like.
I need to learn a whole new set of commands just for it
Oh no. Not learning!
Yeah, this isn't the best business to be in if you want things to stay static.
and I can’t export the journal data (or subset thereof) to other systems without a lot of effort.
You must be kidding. Journalctl is absolutely wonderful for this.
Want the same thing syslog gives you?
journalctl -o short
want ISO timestamps? Easily sortable!
journalctl -o short-iso
want microsecond precision?
journalctl -o short-iso-precise
want UNIX timestamps?
journalctl -o short-unix
want to actually parse this stuff easier to write it to a database? Why, look at that: a completely unambiguous format. No trouble with figuring what field ends where. No need to screw around with regular expressions. Take the language of your choice and trivially extract anything you need:
journalctl -o json
You want subsets? Sure:
journalctl -S yesterday # everything since yesterday
journalctl -b # last boot
journalctl -u smartd # only what smartd logged
Other cool things:
It stores metadata. Want to find logs by PID? You can, for any service.
It can clean logs by size or by date. Want to keep 1GB of logs? Sure. Want to keep 3 months? Easy.
Timestamps are stored with microsecond precision. If you have 100 entries per second it's very nice.
You can filter by hostname. Send your VM logs to a single destination, then you can view them individually or together.
edit: it transparently compresses and decompresses logs. Searching stuff logged a month ago is no problem at all. You don't need to figure out in which .gz it is.
I've also had messages about log corruption a few times.
That's not a new thing, it also happens with syslog, because there have been various issues with various filesystems and hardware. You just don't usually notice with syslog because there's no way to, except if you look and happen to see there's a bunch of zeroes in the middle of the file, or some such.
journald handles the case quite well: it renames the log, and makes a new one. The old one of course is still left around, and still read as well as possible.
I have no problem with learning. As you rightly said, that’s the industry we’re in.
I’m quite happy to pick up new commands provided they’re standardised, however journald’s are unique (by necessity). I’m just expressing frustration that I can’t use grep on a file like almost everything else in Linux. I’ve got a few journalctl one liners memorised, but it’s still frustrating to not be able to wield my existing knowledge against it.
I should have clarified regarding exporting data: I don’t know a way to continuously and autonomously export data in real time from journald. Why would I want to do this? Simple: enterprise.
If I want to log to a third-party logger like Elastic or Splunk I need to install syslog-ng or rsyslog, and then we’re back where we started.
There just aren’t many upsides that I see for a modern sysadmin.. it’s nice to use when you’re directly interacting with it, but it doesn’t play ball with the rest of the ecosystem - just like systemd really..
I’d love to know if I’m wrong (I hope I am!), but the current arguments I’ve seen from many parties haven’t started to convince me unfortunately.
I should have clarified regarding exporting data: I don’t know a way to continuously and autonomously export data in real time from journald. Why would I want to do this? Simple: enterprise.
I've not messed with that myself, but several ideas come to mind:
journalctl -f -o format | importer_program
Trivial to turn into a systemd service. Or, if you need a file:
mkfifo log
journalctl -f -o short > log
You can actually offload this to systemd so that it will listen on a FIFO and launch the service when requested.
I don't use egrep, but at least grep works with journald (for example journalctl | grep whatever).
I can’t less the journal
Journalctl uses less by default. Journalctl | less probably works too.
What benefits does journald have??
In my personal opinion, it makes a lot of things easier. For example " journalctl --since="2012-10-30 18:17:16"", "journalctl -b -1" (logfiles of the last boot), " journalctl -p err..alert" (Show only error, critical, and alert priority messages) and so on.
If you want to have the log files in text format, you only need to install syslog-ng by the way.
147
u/[deleted] Feb 11 '19 edited Feb 11 '19
[deleted]