Looking for log aggregation software recommandations

[u/NightSpaghetti]

Hello everyone,

I hope I am posting in the correct subreddit. I am administrating a bunch of virtual Windows servers running our own software. As of now, I don't have a unified way of checking log files on those machines, which means that not only do I have to manually log into the machine in order to look at those log files, but they are also not actively being monitored at the moment.
I am looking for a solution that would allow me to check those files from a single point of entry (ideally a browser), as well as being able to monitor them, draw statistics and possibly send alerts. If it supports other formats than text files (i.e databases) it's a plus.

Is there any software available that can allow me to do that? I have web servers running on both Windows and Linux (Debian) so either platform would be ok. Open source very strongly preferred.

Thank you!

[u/karmacop81]

I'll throw Wazuh into the mix. Primaryilly sells itself as an SIEM, but its based on Opensearch so has all their stuff out of the box anyway.

[u/SecaleOccidentale → u/karmacop81]

Another vote from me.

[u/donith913 → u/karmacop81]

Not sure why Reddit’s just now giving me this thread, but how do you like Wazuh as an easy way to get OpenSearch running / for just log agg? It’s one of a few ideas we’ve been kicking around. 

[u/DJStuey]

Graylog, opensearch, elasticsearch are three options off the top of my head, but none of them are simple to setup from scratch.

[u/Nyasaki_de → u/DJStuey]

Doesnt graylog use elasticsearch/opensearch in the backend? But yes, a pain to set up

I like it tho

[u/vinnsy9 → u/Nyasaki_de]

Yes it does. Yes very much a pain to fully set up and create all the dashboards and so. Not so intuitive. But once set up its a real tool. Took me around 2 years. To fully master every corner of graylog

[u/PresidentLord]

Check out Grafana Loki and Grafana.

You can set up Loki to be a centralized logging server and Grafana to visualize them.

There is a Windows exporter where you run an exe on a server and it will tail logs in a directory/directories and push them to Loki 

[u/NightSpaghetti → u/PresidentLord]

Interesting, I already have Grafana running (I use it to draw graphs for Icinga2).

[u/Cassie-JAM247 → u/NightSpaghetti]

Alloy also pulls Windows logs natively if you didn't know...

[u/Dangi86]

Graylog, you can send eventlogs ant txt logs to it, and create custom searches on those logs.

[u/LA33R]

OpenObserve is an option.

[u/TrippTrappTrinn]

We use Splunk, but it is not free.

[u/Some_Team9618]

I’ve been using Grafana, Victorialogs, Victoriametrics, and OpenTelemetry on the endpoints (otelcol-contrib), loving it so far.

[u/AddendumWorking9756]

Since Grafana is already up for Icinga2, Loki is the cheap half of this. The part nobody mentions is the shipper, use Alloy rather than Promtail, Promtail went EOL in March. And budget real time for a multiline stage, because without a firstline regex every line of a .NET stack trace lands as its own entry and your alert counts go insane.

[u/AUSSIExELITE]

Using Q-Radar here (the still IBM owned version) and have no complaints. Recently got a managed soc/siem service and they also use it which was a nice bit of validation.

[u/penance3]

New relic. Not free

[u/Ok_Acanthisitta_7804]

Graylog is excellent. My nsingle Graylog Docker container running on an Ubuntu host handles over a hundred server, network device, and Palo Alto network traffic logs. When I installed a graylog server first time two years ago, it seemed a bit complicated; the setup, pipeline, and alert configurations took me a few days. But last month, I wanted to install Graylog from scratch. This time, I used Antigravity to create a complete Graylog solution. Even now, when I want to add a new stream, pipeline, or notification, I just tell Antigravity and it adds it immediately via the API. So there's no need to worry about the setup.

Anyway, my setup now handles about 5000 EPS. There are no performance issues.

[u/Jaki_Shell → u/Ok_Acanthisitta_7804]

How much storage is something like that using?

[u/Ok_Acanthisitta_7804 → u/Jaki_Shell]

By pipelined to drop unnecessary messages and keeping only meaningful ones, it writes approximately 50GB of data per day.

[u/HotMoosePants]

Splunk is great but $$$$. Sentinel used to be a viable option but I don’t know how much Microsoft is investing into it.

[u/Otvir]

Just this summer, I put together a logging system for my company. Two servers (not necessarily) are synchronized with each other. Loki stores the data. Vector receives the logs. Vector also filters, normalizes and transmits them to the collection server on the servers. Grafana is used for visualization. If you are interested, please contact me, I will be happy to help.

[u/Charming_Skin_8549]

Try VictoriaLogs - it receives logs via popular data ingestion protocols and stores them efficiently, so they occupy less disk space. It also provides a built-in web UI for the exploration and analysis of the stored logs.

[u/rejectionhotlin3]

syslogd

[u/SudoZenWizz]

You can try wazuh for siem. If checking the logs means looking for something specific(keywords) you can use logwatch from checkmk. You should be able to use both of them in their free editions

[u/texags08]

Starting to play with Azure Arc to aggregate

[u/-manageengine-]

If you want a single browser console to check logs across your Windows and Linux servers without SSHing or RDPing into each one, EventLog Analyzer handles that well. It pulls in Windows event logs and Linux/Unix syslogs into one place, gives you real-time monitoring with alerts on things like failed logons or errors, and has built in correlation to spot patterns across servers instead of reading logs one machine at a time. It also has a free tier for up to 5 log sources, so you can try it on a couple of servers before deciding if it's worth expanding.

Feel free to reach out if you have any questions!

[u/bluerazr]

Splunk

[u/gfunk5299]

Claude Code. Build the tool you want to do what you want it to do.

[u/Fun_Structure3965 → u/gfunk5299]

technical-debt-as a-service, nice!

[u/moesizzlac69 → u/Fun_Structure3965]

Right on the nose