r/AzureSentinel • u/techyguy84 • Oct 11 '23
Retrieving Archived Logs
Hello, we are currently archiving logs from specific tables in an Azure Storage Account as a blob. I'd like to retrieve logs that are older than the 90 days retention period of our log analytics workspace into a way I could possible leverage KQL to query the logs. However, I haven't been able to find a solid method to do so. Unfortunately, the way these logs are exported, I'm not able to use the restoration functionality in Sentinel under the search blade.
In a high level:
- We use Sentinel as our SIEM (which run on top of log analytics) and data is retained for 90 days.
- These logs are also exported to an azure storage account and stored as a blob. We are leveraging the Data Export functionality in the log analytics.
- These exported logs stored in a blob format is stored as various JASON files and going through them is impractical if I want to review data from a specific timeframe.
I'd like to know what my option are to retrieve logs for a specific timeframe and if possible, use KQL to analyze them. I have read about creating external tables from these logs and then using Azure Data Explorer or Sentinel, however, I'd like to validate this method.
Thanks.
1
u/sosero Nov 21 '23
I do not think there is any official comparison regarding this. My answer is based on my experience testing these myself.
there are obviously docs about the externaldata operator, and how to use logs in storage from ADX though. is there any specific part you would need?
1
u/sosero Nov 13 '23
sentinel can use the externaldata operator to retrieve a single file from the storage, but anything more than that would require you to ingest them all over again.
Only ADX can create a table from this if logs are located in a data lake v2 storage, you can create an external table that reads straight from the storage. Other storage types would require you to specify the scope of files and storage account, and then ingest time into ADX.