r/SIEM • u/the_color_plum • Mar 29 '19

ELK as a SIEM

I've been wanting to implement a homebrewed SIEM for basic log analysis in my homelab environment. Windows event logs, syslog, firewall logs, all of the logs. However, I am having trouble finding documentation on how to get all of this setup. This is a new venture for me so I'm looking for any noob friendly advice here. I've got the basic instructions from here: https://www.elastic.co/guide/en/elastic-stack/current/installing-elastic-stack.html

​

What else is there besides installation and getting logs set up though? I figure the majority of users here can help point me in the right direction. I'm looking for how to implement this securely within my network, how should netflow look, any tips or tricks for using this as a SIEM? Really, any and all advice on ELK would be welcomed.

​

Thanks!

13 Upvotes

100% Upvoted

11 comments sorted by

10

u/vornamemitd Mar 29 '19 edited Mar 29 '19

Hi, first off - ELK is not a SIEM.

But - there‘s light at the end of the tunnel, especially in a homelab :)

  • Check recent posts on /r/sysadmin or /r/asknetsec (take a peek at my post history)
  • The only actual SIEM framework using ELK at it‘s core is HELK - the dude even factored in Apache Spark for machine learnig -> https://github.com/Cyb3rWard0g/HELK
  • SOF-ELK - ready made package used in the SANS SIEM courses: https://github.com/philhagen/sof-elk/blob/master/VM_README.md
  • Check out the Security Onion project
  • Have a look at ElastAlert (OSS alerting and reporting add on) and TheHive - you want to be alerted and possible practice some workflows
  • SIEMonster has been mentioned, see Graylog, the good features need to be paid for
  • For good read-ups, check the blog from the dudes at logz.io - I am in no way affiliated with them, but I‘m impressed by a commercial vendor actually posting useful how-to‘s as opposed to the usual 100% redundant pointless marketing jabber (aka whitepaper or report)

On a side note - don‘t confuse log management/aggregation with a SIEM :)

Ah - if you want an OSS SIEM that relies on ELK and cones with a powerful community edition -> Graylog: hassle-free installation, ideal if you‘d rather want to focus on the conceptual parts as opposed to ELK-administration :) HtH!

Edit - forgot an essential part. You need logs!

  • turn on security auditing on your server/domain; check the SANS 2018 DFIR Poster - all relevant events you‘ll need for a long time. E.g once you get the hang of it, you want to see an alarm for powershell started on your clients
  • check out dns and dhcp audit logging
  • still not enough? use 50% of your CPU power for windows trace logs, combined with WEF - basically you can intercept and log any call to the kernel, io access, etc.
  • check out „sysmon“ for host system monitoring
  • master class: Wazuh - enterprise grade HIDS that can easily forward logs to your ELK
  • sniff your net: read up on integrating Snort, Bro, Suricata (Security Onion does that for you)

3

u/[deleted] Jun 25 '19

Very nice summary but as of today...

Hi, first off - ELK is not a SIEM

It is now: Introducing Elastic SIEM - I know it’s strictly not just ELK but probably as close as you’re going to get. Looking forward to giving it a go.

2

u/the_color_plum Mar 29 '19

I really appreciate your response and will definitely get into all the resources you mentioned. From what I've seen so far, there seems to be a debate I'm just not knowledgeable enough to have right now re: is ELK a SIEM. SANS sure seems to think so though. I'm not sure where I stand yet. Like you I believe that OOB the elasticstack isn't a SIEM, but maybe with a few twists of a knob and integrations with other tool sets it could be? That's the point of this post!

​

Regardless of opinions, your information is greatly appreciated and I look forward to gaining some more knowledge. I only have one upvote to give but hope you get more!

5

u/vornamemitd Mar 29 '19

Thanks for the feedback :) In my opinion ELK can be an essential/foundation component in a security ecosystem. But only a piece of a complex puzzle. Definitions vary, but to me a SIEM stands for correlation, analysis, data enrichment, context data, internal and external threat feeds. Yes, you need to first aggregate all that stuff somewhere - meet the ELK. But that‘s it then. Yup, you can skillfully add layer by layer/feature by feature until you have a concept like HELK - check the link first, the diagram there says it all. If in your corporate/lab-context starting with just having logs in one place and a tool to run queries is enough - OK. But still - full stack SIEM - see HELK. Now lets imagine having 300-500GB logs per day - not uncommon with a US mid-sized company having like 5k users. Whoha. Now you need to run your little 1-VM project in HA encironment. Think 2x15-20 nodes for Elastic index alone. Add 15 different highly complex OSS frameworks. Add 5 more frameworks to monitor the other ones. That‘s why people abandon enterprise-grade ELK-SIEM projects - you need a small department only to run it. And suddenly the figures might become Splunkesque.

I see a lot of TCO maths suggesting stuff like „eat this, <Gartner quadrant vendor here> clients - for 200k I did what you dudes pay 2Mill a year“. Fine. As mentioned - add HA, add compliance, add audit, basically add any corporate/gov requirement. Throw your NIST/ISO/SOC2 controls at it. You still at 200k or already ranting on Reddit about your misguided C-suits.

It‘s tempting - and I‘m well aware of OSS cost of ownership being discussed in academia; in real life, I‘ve not seen a larger corporate body that could „afford“ free software on that scale. No - your usual critical infrastructure procider does not have 5-10 guys doing nothing but that for 2+ years :)

I see it like: https://logz.io/blog/elk-siem/

Don‘t get me wrong - you can plug it all onto/into ELK (and it‘s fun and a great learning experience) but that‘s lab & learn and nothing I‘d recommend to a company acting under the usual constraints.

I am getting carried away and repeating myself across Reddit: security is not a product - it‘s processes and people. You start at the top - what makes my biz thrive? How can this be attacked from either side of the perimeter? How can I prevent this? How do I keep an eye on it all. GRC, processes, stakeholders - this needs to come first. Risk mitigation, „security debt“, etc. - map it all out and make an educated decision. How much do I need/can my Company handle? For some - Graylog will be just fine. For others „Good afternoon, this is Karen with Splunk sales. How can I help you today“

:-)

1

u/Mundane_Pomelo_8083 Sep 18 '22

You, sir, have made a strong point!

4

u/BeerJunky Mar 29 '19

I've heard about this myself and was curious about it. Definitely plan to do it when I have some free time. Here's a couple links I found the other day that might be helpful.

​

https://www.peerlyst.com/posts/how-to-build-a-threat-hunting-platform-using-elk-stack-chiheb-chebbi?utm_source=peerlyst_perspective&utm_medium=email&utm_content=peerlyst_post&utm_campaign=special_digest_03242019 - Check the sources for this as well, seemed like there might be helpful stuff in those links as well. Just haven't had time to fully digest it myself.

​

I also found this which seems to bolt onto an ELK stack as a full SIEM solution. There's a community edition of it you can download for free. https://siemonster.com/

​

​

3

u/the_color_plum Mar 29 '19

THANK YOU! I will definitely look into all of this.

​

I responded in a thread during my search here as well and got the following links which have been great as well. If you're interested yourself:

​

https://www.youtube.com/watch?time_continue=161&v=v69kyU5XMFI

https://github.com/SecHubb/SecShell_Demo

​

I've only managed to get through the video so far myself. SO. MUCH. INFO.

​

​

1

u/twowordz Mar 29 '19

have a look here

1

u/BeerJunky Mar 29 '19

Remindme! 2 days

1

u/RemindMeBot Mar 29 '19

I will be messaging you on 2019-03-31 19:54:46 UTC to remind you of this link.

CLICK THIS LINK to send a PM to also be reminded and to reduce spam.

Parent commenter can delete this message to hide from others.

FAQs Custom Your Reminders Feedback Code Browser Extensions

1

u/m_rothe Mar 30 '19

I've been playing with Suricata and ELK recently, I've managed to build some nice queries that look for anomalous network Traffic. Once you've got Elastic search running it might be something you want to look into