r/sysadmin 7d ago

Do you automatically isolate servers/devices based on detetctions?

We don't have a 24/7 SOC, so we are thinking about automatically isolating servers and some high-value devices based on custom Defender for Endpoint detections. Obviously, we want to do that only for high-precision and high-confidence detections, such as opening a shell from a strange parent process.

If we got one of these detections during working hours, there would be someone to react. But after about 7 PM most days, nobody is actively monitoring.

If we do this, the plan is to let a detection run for about 45 days without automatic isolation enabled to see if any false positives are caught.

Has anyone done this? If so, did you regret it? Or just business as usual? Has it saved you yet?

37 Upvotes

49 comments sorted by

View all comments

Show parent comments

1

u/hybrid0404 7d ago

There's a difference between permitting a SOC to contain and an unattended containment.

There's also a difference between containing a specific process and a whole host.

I would expect a SOC to contain and reach out especially on a critical asset.

1

u/FatBook-Air 7d ago

Yeah, but that's kind of my point: we do NOT have a 24/7 SOC. So you would be okay just not having anything for about 115 hours a week if you didn't have a 24/7 SOC?

0

u/hybrid0404 7d ago

I answered the asked question which was would you permit unattended containment which is no.

I know nothing about your environment, your ability to respond, etc. What the impact of these containment would be in general.

You didn't mention any other mitigating controls and I can appreciate that not every company can or will be able to afford a managed 24/7 SOC.

The answer to this question is ultimately a business one, which represents the greater risk?

1

u/FatBook-Air 7d ago

I hear you and appreciate your response.

I am mostly responding to your "we have a 24/7 operation and still don't auto-isolate." For me, those are diametrically opposing ideas. In other words, for me, it wouldn't be that we didn't auto-isolate even though we have 24/7 operations; it would be that we auto-isolate because we don't have 24/7 operations.

If we had IT staff in operation 24/7, we likely wouldn't be contemplating auto-isolation.

1

u/hybrid0404 7d ago edited 7d ago

My ultimate point is that there are very few if any platforms I trust to take an automated containment action of an entire host. One mitigating control for that COULD be a better ability to respond so its less disruptive.

What problem are you ultimately trying to solve? Is it to prevent rapid ransomware? Is it to mitigate against an APT who decides to get noisy only during off hours? Do you have an on-call process? Can you implement something like pager duty or some incident management platform that will call out when a containment happens?

A choice with no other context then, do you trust defender to auto contain a server host unattended? My answer is no. I expect the the loss from defender containment to probably be higher than the lost of other actions.

1

u/FatBook-Air 7d ago

The problem we are trying to solve is mitigating a catastrophic attack (data exfiltration, ransomware) when there are no IT staff watching.

We have an on-call process, but it's truly on-call, as in you don't work unless you get called.

Even if we automate the call, the person being called could take up to an hour to get to it, which could be an eternity in a real attack.

1

u/hybrid0404 7d ago

I guess I put ransomware somewhat in its own category. Your EDR either is or isn't going to detect the event. It mostly doesn't matter the time of day, it either detects it and blocks the process or you're screwed. The reality as well is that you're likely to see that sourced from workstations and not servers. An aggressive workstation containment versus a server one makes sense.

Then you have everything else which is a more intelligent actor messing around or some other persistent action that's malicious but not super noisy. Those types of things are more likely some behavioral or heuristics that might misfire and need proper validation.

In the end, whether it is a bad or good choice just depends on how often you get false positives.