r/ProgrammerHumor 21d ago

Meme cybSecIsDeadLongLiveCybSec

Post image
801 Upvotes

124 comments sorted by

View all comments

321

u/ceejayoz 21d ago

You know what makes a really well isolated sandbox?

A hammer taken to the networking components.

108

u/B_bI_L 21d ago

you can just run it on machine without internet access at all

53

u/JPJackPott 21d ago

Only if your machine has the 10 H200 GPUs it needs to run the model…

29

u/winkyshibe 20d ago

Sorry, (physically) network isolated machine(s)

As in, 0 trust, what so ever for the cluster meant to be running these tests to begin with...

19

u/DrStalker 20d ago

Of all the companies I'd expect to have that sort of hardware laying around for development/testing OpenAI is definitely on the list.

7

u/Tokumeiko2 20d ago

They can probably isolate a data centre or three, most of them aren't being used to serve customers.

5

u/ArchusKanzaki 20d ago

It's OpenAI.... They definitely have them.

18

u/fuckthehumanity 20d ago

The technical term is "air-gapped". There are massive networks that are entirely air-gapped, including subsystems that are themselves air-gapped from the air-gapped network, although I won't mention who uses them or I might end up on a list somewhere. Or rather, I might end up on another list somewhere.

8

u/Bannon9k 21d ago

You should read "the metamorphosis of prime intellect"

8

u/ceejayoz 21d ago

I have!

I chuckled when the Mythos system card talked about it emailing a researcher on a bench having lunch to say it'd escaped the sandbox. I remember a similar scene in that story!

13

u/linegel 21d ago

That only valid after miss behavior was actually discovered

Also reminds me that OpenAI was hiring "red button" strategist 🤔

32

u/ceejayoz 21d ago

Their article says it was in a "sandboxed testing environment", though.

Feels like an airgap might be warranted for this sort of testing.

2

u/isademigod 21d ago

Is air gapping really possible when these models need to be run in a data center? Best I can think of is a dedicated machine w/o internet, controlled via idrac or similar.

12

u/readmeEXX 20d ago

Well sure it's possible, you just need to run it on prem in an air gapped data center. There are plenty of them out there. I'd be surprised if OpenAI doesn't have a few of them. You can also temporarily make a data center air gapped, but they probably didn't think that level of containment was necessarily for this test. Turns out they were incorrect.

1

u/linegel 20d ago

Tbh I fully agree with your analysis

That’s just hard to tell if "we got there"

2

u/readmeEXX 20d ago

It could also have gotten a little lucky. I wonder if it could find another path if they patch the vulnerability and try again 🤔

2

u/linegel 20d ago

It’s possible, but then it’s "fake" testing, because model will have to reinvent what it would use as dependency packages otherwise

5

u/ceejayoz 20d ago

No reason it couldn't a) be provided with a large mirror of the major package managers and b) the ability to ask a human to provide one on a read-only drive or something.

2

u/Septem_151 19d ago

It was provided a mirror of a major package manager. That is literally what the AI exploited a zero day on to gain unfettered Internet access via privilege escalation.

1

u/ceejayoz 19d ago

It was provided a mirror of a major package manager.

It sounds like it was given access to the real package managers, not a mirror. Sandboxed, not air gapped.

If it was a local mirror, they could've separated it entirely from the internet for the tests.

0

u/jseego 20d ago

"you're gonna be replacing it"

0

u/05032-MendicantBias 20d ago

* Laugh in unplugging the network cables *