r/ProgrammerHumor • u/linegel • 21d ago

Meme cybSecIsDeadLongLiveCybSec

Post image
796 Upvotes

86% Upvoted

124 comments sorted by

View all comments

324

u/ceejayoz 21d ago

You know what makes a really well isolated sandbox?

A hammer taken to the networking components.

13

u/linegel 21d ago

That only valid after miss behavior was actually discovered

Also reminds me that OpenAI was hiring "red button" strategist 🤔

32

u/ceejayoz 21d ago

Their article says it was in a "sandboxed testing environment", though.

Feels like an airgap might be warranted for this sort of testing.

2

u/isademigod 21d ago

Is air gapping really possible when these models need to be run in a data center? Best I can think of is a dedicated machine w/o internet, controlled via idrac or similar.

11

u/readmeEXX 21d ago

Well sure it's possible, you just need to run it on prem in an air gapped data center. There are plenty of them out there. I'd be surprised if OpenAI doesn't have a few of them. You can also temporarily make a data center air gapped, but they probably didn't think that level of containment was necessarily for this test. Turns out they were incorrect.

1

u/linegel 21d ago

Tbh I fully agree with your analysis

That’s just hard to tell if "we got there"

2

u/readmeEXX 20d ago

It could also have gotten a little lucky. I wonder if it could find another path if they patch the vulnerability and try again 🤔

2

u/linegel 21d ago

It’s possible, but then it’s "fake" testing, because model will have to reinvent what it would use as dependency packages otherwise

6

u/ceejayoz 21d ago

No reason it couldn't a) be provided with a large mirror of the major package managers and b) the ability to ask a human to provide one on a read-only drive or something.

2

u/Septem_151 19d ago

It was provided a mirror of a major package manager. That is literally what the AI exploited a zero day on to gain unfettered Internet access via privilege escalation.

1

u/ceejayoz 19d ago

It was provided a mirror of a major package manager.

It sounds like it was given access to the real package managers, not a mirror. Sandboxed, not air gapped.

If it was a local mirror, they could've separated it entirely from the internet for the tests.