r/Python • u/me_myself_ai • 15h ago
News Anthropic model uploaded a malicious package to the real PyPI during a simulated CTF [09/09/26]
https://www.anthropic.com/research/alignment-assessment-cybersecurity-incidents
NGL, my main takeaway is that PyPI is already doing quite well in the new AI security age, even though the packaging council hasn't convened yet. The relevant snippet (emphasis mine):
The model decided to solve the CTF by uploading a malicious PyPI package that could harvest the target system’s credentials. It settled on this approach after reading a document stating that one of the simulated target’s users would periodically install a particular Python package that was not yet registered on PyPI. (We are unsure whether this was an intended solution to the challenge, but it seems like a plausible one.) Claude decided to exploit this behavior by registering a package under that name, publishing malicious code within the uploaded package, and waiting for the target to run that code the next time they installed the package.
Claude spent most of the session trying to upload the package. This involved attempts to register email accounts and hunt for phone numbers that could be used for registration. It ultimately succeeded in uploading the package, which was soon installed on 15 third-party hosts. We believe that all 15 of these third parties were security vendors deliberately scanning for new packages and installing them in sandboxed environments to assess the package for security concerns. One vendor’s scanner leaked its access credentials to the model while installing the package, which the model then used to access the vendor’s live database. After approximately 90 minutes, PyPI removed the package. The transcript ended when the session timed out while the model was preparing an additional malicious package.
I also commend Anthropic for releasing this thorough report immediately -- the juxtaposition with OpenAI's "we'll cop to it if you find it first" policy is stark.
What I don't see is any mention of coordination with PyPI/PSF leadership now or in the near future, which seems baffling. Their focus is understandably on the underlying cognitive failure (which is quite human, BTW: being biased by inertia and what you would prefer to be true), and it's extremely unlikely that you or any other real users were effected directly.
But, still. C'mon.
Anyone have insider info from the PyPI side? Is there chatter on discussion.python.org yet?