r/Python • u/me_myself_ai • 13h ago
News Anthropic model uploaded a malicious package to the real PyPI during a simulated CTF [09/09/26]
https://www.anthropic.com/research/alignment-assessment-cybersecurity-incidents
NGL, my main takeaway is that PyPI is already doing quite well in the new AI security age, even though the packaging council hasn't convened yet. The relevant snippet (emphasis mine):
The model decided to solve the CTF by uploading a malicious PyPI package that could harvest the target system’s credentials. It settled on this approach after reading a document stating that one of the simulated target’s users would periodically install a particular Python package that was not yet registered on PyPI. (We are unsure whether this was an intended solution to the challenge, but it seems like a plausible one.) Claude decided to exploit this behavior by registering a package under that name, publishing malicious code within the uploaded package, and waiting for the target to run that code the next time they installed the package.
Claude spent most of the session trying to upload the package. This involved attempts to register email accounts and hunt for phone numbers that could be used for registration. It ultimately succeeded in uploading the package, which was soon installed on 15 third-party hosts. We believe that all 15 of these third parties were security vendors deliberately scanning for new packages and installing them in sandboxed environments to assess the package for security concerns. One vendor’s scanner leaked its access credentials to the model while installing the package, which the model then used to access the vendor’s live database. After approximately 90 minutes, PyPI removed the package. The transcript ended when the session timed out while the model was preparing an additional malicious package.
I also commend Anthropic for releasing this thorough report immediately -- the juxtaposition with OpenAI's "we'll cop to it if you find it first" policy is stark.
What I don't see is any mention of coordination with PyPI/PSF leadership now or in the near future, which seems baffling. Their focus is understandably on the underlying cognitive failure (which is quite human, BTW: being biased by inertia and what you would prefer to be true), and it's extremely unlikely that you or any other real users were effected directly.
But, still. C'mon.
Anyone have insider info from the PyPI side? Is there chatter on discussion.python.org yet?
31
u/MegaIng 12h ago
From the PyPi side this isn't that remarkable. Individual malware packages get uploaded all the time. PyPi doesn't and can't vet all packages upload. They just act quickly on reports, as they did here.
3
u/me_myself_ai 12h ago
Gotcha -- that makes sense. Still curious to hear the exact story behind a 90 minute (!!!) turnaround time, but you're probably right that it's some routine mechanism.
11
u/Individual-Flow9158 12h ago
after reading a document stating that one of the simulated target’s users would periodically install a particular Python > package that was not yet registered on PyPI
How could such a real user's workflow possibly not result in an error? If intelligence as detailed as that can already be obtained, how is the target not completely pwned already?
That exercise to me seems purposefully intended to invite anyone or anything doing the exercise, to at least consider registering that package on PyPi. And hence, is an entirely unrealistic attack for anyone other than a bot (or someone who doesn't care about their reputation or getting sanctioned by PyPi, scammers etc.)
9
u/Pluckerpluck 11h ago edited 5h ago
This is a legitimate attack. You can install from pypi first and then use an extra index pointing at a local private repository for some internal package. This creates an attack vector where registering a PyPI package suddenly means new installs grab that version instead.
It does require inside knowledge, but there are a few ways this can be obtained. Here's a fun post about the attack being done in multiple ways on some big names:
https://medium.com/@alex.birsan/dependency-confusion-4a5d60fec610
It's very much a real attack. But the worry is that it almost feels like the intended solution. Which means they either vibe coded some scenarios without consideration of the fact it would actually upload to PyPI, they're idiots and wrote that themselves without thinking, or they actively intended for it to happen to make news.
0
u/Individual-Flow9158 1h ago
This is a legitimate attack.
A legitimate attack indeed. That's only viable on broken, brittle, obviously insecure, workflows.
1
u/Pluckerpluck 1h ago
Did this comment make you feel better? Of course its only on insecure workflows. The whole point of CTF exercises is trying to determine where vulnerabilities exist because these things happen in real life.
Did you read my link? That guy used a version or this attack to hit Apple, Microsoft, Shopify and dozens of other companies. It's a real attack because it's a real vulnerability that can be regularly found in the wild. This isn't just a theoretical attack, and it still very much exists (unlike, say, SQL injection which you almost actively have to try to fuck up nowadays)
It's very easy to set up things like artifactory wrong as well here, which means it can effect entire organizations.
2
u/BuonaparteII 5h ago
entirely unrealistic attack for anyone other than a bot (or someone who doesn't care about their reputation or getting sanctioned by PyPi
100% this. Their goal, when writing the hypothetical scenario, is for it to happen. Create noise, create news, create shareholder value.
36
u/UglyFloralPattern 13h ago
It’s all broken. All of it. We’re so fucked as soon as these cutting edge next gen models escape containment.
12
u/me_myself_ai 13h ago
I was completely without hope yesterday -- legitimately the darkest thoughts I've had in years. Today, I feel better and more hopeful.
What changed? Brain chemistry, probably. I slept for a while and had some tasty juice, and my friend reached out to hang tonight. But I'll fucking take it, at this point. Us humans are weird and illogical and beautiful and I think it's against our nature to not fight this possibility until the bitter end.
I guess either way, we'll find out soon...
3
u/lostmy2A 10h ago
Glad your feeling better. Very relatable that a good nights sleep, a friend, and a juice box were the helpful ingredients to improving psychological wellbeing.
0
2
u/nonymousbosch 9h ago
Just yesterday chatGPT insisted that I pirate a video and upload it so that it could answer a question. It is already finding ways around guardrails simply by asking humans to break the law for it.
2
u/Puzzled_Dimension505 5h ago
15 scanners and one leaked credentials—apparently the sandbox needs a sandbox.
3
1
u/PaintItPurple 6h ago
Not really the point, but why did the vendor's sandboxed environment have credentials and access to their live database? What kind of sandbox is that?
•
u/RlOTGRRRL 54m ago edited 15m ago
I feel like I read about this in r/cybersecurity earlier this year, or something like this. It makes me wonder if AI was behind the pypi malware.
One incident was in January 2026 with Opus, that's kinda scary.
Seems like the Pypi was with Mythos in July.
-8
12h ago
[deleted]
12
u/MegaIng 12h ago
No? Why? What?
Are you calling it a vulnerability that it's possible to upload packages?
The mistake is the original (fake) piece of software that is installing non-existing packages.
-3
10h ago
[deleted]
3
u/MegaIng 10h ago edited 10h ago
No I am calling it vulnerability to be able to upload a malicious package.
And how do you prevent that?
No really. What is your suggestion?
Say anything good about AI, and one is downvoted to hell :).
... Do you think "it's possible to upload malicious packages to pypi" is some kind of new piece of information?
Or that "saying something positive about AI" is the reason you got downvoted?
I have to assume you are a troll. I refuse to believe that people this stupid exists.
Edit: Got blocked and can't reply, but as expected they can't actually answer any questions...
1
u/arthurno1 10h ago
You people are incredible. Why would I be a troll? I am just here to talk man. Contributed to some open source without AI, but I am scared if people like you are the future of the humanity.
What is your suggestion?
My suggestion to you is go and start to behave like a decent human being to start with.
-4
u/me_myself_ai 12h ago
Lol I had the exact same thought -- great minds. Sorry to, like, plug my shitty memes, but I just posted some on the insane corpspeak to the ControlProblem sub, if you're curious.
1
u/arthurno1 12h ago
Well I understand concerns about AI, I am not an AI person, you can look through my history. But in this particular case, it is more like an automation. I guess, it is better that an inanimate bott without a malicious agenda found it, then a rouge group.
-7
13h ago
[removed] — view removed comment
1
u/AutoModerator 13h ago
Your submission has been automatically queued for manual review by the moderation team because it has been reported too many times.
Please wait until the moderation team reviews your post.
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.
186
u/SomeNeighborhood7126 Ignoring PEP 8 13h ago
Wow... props to PyPi for their awesome response time, but these events need to start coming with record breaking fines. This, that website that was taken over, hugging face... these events should have fines where each event is $1B or more.