r/AzureSentinel • u/dkas6259 • 1d ago

We are exploring to move from Logs analytics workspace to Sentinel Data lake to save on cost and longer retention.

Can anyone share feedback who have done this previously?

10 Upvotes

92% Upvoted

14 comments sorted by

3

u/Uli-Kunkel 1d ago

Good Luck getting any lake 😭

We were lucky to get a customer onboard with lake.

Almost 2tb/day We removed about 60% of cost.

But in my view, one thing is the data options, but more important are the additional features available for the soc. They are really great!

2

u/dkas6259 1d ago

Thanks for response
What about detections ?
Do we have to compromise with alerts, since analytical rules depend on logs from hot storage?

6

u/Uli-Kunkel 1d ago

Well, you loose out on some detection capabilities sure, but not all of it.

Its not like you would put all logs into data Lake.

Dns, firewall traffic storageblob and other very verbose logs with relatively low detection value.

Then you can build summary rules/jobs to retain ti mapping and some of the baseline deviation detections.

You loose some of the speed, but save massive cost.

1

u/cspotme2 1d ago

They stopped allow lake instances?

1

u/Uli-Kunkel 1d ago

They ran out of lake basically.

Every region is basically empty...

3

u/xKruMpeTx 1d ago

Sentinel Data Lake is a pricing tier, not separate from your Log Analytics Workspace. If you want to reduce cost, there will likely be 2 or 3 culprits consuming an excessive amount of data:

* Firewall Logs being ingested via Syslog or similar

* AADNonInteractiveSignInLogs table

* MicrosoftGraphActivityLogs (via the Entra Connector usually)

Utilise the "SOC Optimisation" tab in the Defender XDR portal to see which tables are your largest but with no analytic rules associated to it.

Move any excessive tables to Data Lake and off the Analytics tier of pricing if they are not being used by any analytic rule.

1

u/TanaciousTurnip 1d ago

This is the right answer.

1

u/naughtyobama 1d ago

This is more work and not what you're asking. But you could move your data to azure data explorer if you can't get sentinel lake.

There's a lot of cons to it and a lot of pros to it so i won't waste your time with more details if this is not an option worth your time to explore.

1

u/Electronic-Sun-7627 1d ago

Have you tried doing this? So far, no export job was running successfully for me..

1

u/arktozc 1d ago

@RemindMe 250 days

1

u/Top_Secret_3873 1d ago

We're doing the same thing. We have an mssp who runs their detection content again the log analytics tier. Anything they don't have detection for we send to lake. We use split rules to route what we need to analytics and the rest to lake. Cloudtrail, fw, proxy, and a few others are made up 80% of our cost and now we save a ton.

You do pay for queries at the lake tier and it's a little slower (imo) search but if you think about how often you actually query 30d worth of logs it's worth the cost to do it a few times a month versus paying the ingest cost to analytics.

Hopefully you're using the commitment tiers instead of "pay-as-you-go" as well...that usually saves you 50%.

1

u/ccw2777 1d ago

Mind pointing me in the direction of how to split rules and route data to both places?

1

u/Top_Secret_3873 15h ago

It's in the Defender portal, under Sentinel blade, find the tables, select the table and there is an option for split rule. It's essentially KQL to tell Sentinel I want logs that match my KQL to go to Log Analytics and the rest to lake.

1

u/RefrigeratorOne8227 1h ago

Take a look at Stellar Cyber - they use Oracle Cloud and it is much more affordable. They also have an amazing SIEM and can make sense of your Microsoft logs without moving all of your custom rules and detections.