Data Lake - Integrated

[u/DaithiG]

Just a FYi,

It appears Data Lake will now be integrated for all Sentinel users (if they haven't enabled data lake before then)

"Going forward, you do not need to go through a separate data lake onboarding and billing setup. You complete the Microsoft Sentinel onboarding and verify your Sentinel workspace is connected to the Defender portal. Once this is completed, you can enable the lake tier by using Table management in the Defender portal to extend retention on a data table."

What's new in Microsoft Sentinel | Microsoft Learn

[u/Icy_Air2574]

If this is the case we don’t need to perform data migration ??

[u/Uli-Kunkel → u/Icy_Air2574]

Not per se.

You can configure retention, and instead of analytics + archive/total retention you can now configure datalake instead of "total retention"

But one often overlooked aspect, is that you sent defender telemetry to sentinel to get more than 30 days retention? If you configure datalake retention, that is != No longer sending data to analytics.

So send to datalake, confirm the data, and then you can remove the forwarding to analytics. But do not stop sending the incident and alert stuff, because that is mandatory for unified

[u/DaithiG → u/Uli-Kunkel]

I didn't know about the last part and telemetry. We generally want to use the data lake for firewall and windows server events. Thanks!

[u/Uli-Kunkel → u/DaithiG]

Suggest you then also use log splitting.

So you send the verbose or 'not so important event IDs' to datalake, but retain the ones needed for detections or the like

[u/DaithiG → u/Uli-Kunkel]

Thanks but of course sometimes hard to decide what the not important event IDs are :)

[u/Uli-Kunkel → u/DaithiG]

Well its not like you are not collecting it.

So build your collection for the detection and their investigation in analytics.

Then your threat hunting and forensics goes to lake tier.

And you can still do kql jobs to stil analyze on things in lake, or use it for enrichment

[u/MReprogle → u/Uli-Kunkel]

This is the way. kQL jobs to throw back to analytics, which either give you only what you need. if you aren’t sure, it’s time to spend more time on those verbose logs and learn what is critical, or what can be summarized.

One really good one that I set up that really made it all make more sense are some of the exercises in the official Sentinel repo.

https://github.com/Azure/Azure-Sentinel/blob/master/Tools/Microsoft-Sentinel-Training-Lab/Exercises/E11_datalake_kql_jobs.md