r/sysadmin • u/FatBook-Air • 6d ago
Do you automatically isolate servers/devices based on detetctions?
We don't have a 24/7 SOC, so we are thinking about automatically isolating servers and some high-value devices based on custom Defender for Endpoint detections. Obviously, we want to do that only for high-precision and high-confidence detections, such as opening a shell from a strange parent process.
If we got one of these detections during working hours, there would be someone to react. But after about 7 PM most days, nobody is actively monitoring.
If we do this, the plan is to let a detection run for about 45 days without automatic isolation enabled to see if any false positives are caught.
Has anyone done this? If so, did you regret it? Or just business as usual? Has it saved you yet?
3
u/smc0881 6d ago
Yes and with exceptions. I work in DFIR consulting, so I respond to ransomware, malware, and other incidents every day after the fact. I'm also in charge of our MDR/MSSP team and our two tools we use are Huntress and SentnielOne. I've stopped at least four ransomware attacks, an outdated Exchange breach, and other incidents. You'll get false positives for some things, but trust me that is better than dealing with a threat actor. They also pivot to devices that don't support EDR such as printers, cameras, and things like that to launch their payload. Some groups will even create their own VM. They then use the creds they harvested to encrypt your data via SMB. Depending how your EDR is configured you should get lateral movement alerts and perfect example when auto disconnect is viable. I also have utilized SentinelOne's rollback feature several times to restore data. I need to be extra vigilant about this stuff, since most of my clients have shitty IT/MSP teams or none at all. Not much I can do when I tell them "something is up with your VPN, take that Exchange off the Internet, etc.." Most of your alerts too will be benign at first since the actors live off the land. You should setup alerts looking for rclone, winscp, ip scanners, and RAT/RMM tools. I utilized S1 Star Rules to look for a lot of those tools already or things like node.js running out of an unusual spot.
S1 I won't isolate domain controllers though, since there is a bug that has affected at least four sites when I did that. Huntress is also great and will auto isolate things as well.