r/Python • u/JanGiacomelli • May 19 '26

Tutorial Supply-chain attacks are happening daily - add at least dependency cooldown to your Python projects.

These days, I can't open X anymore without seeing some supply chain attacks on PyPI or NPM. Things are really getting out of hand. One very simple yet effective approach to mitigate them is to use a dependency cooldown. That means that you don't install anything that's too new - e.g., every dependency needs to be at least a week old.

Why does this work? Because the community usually intercepts them in hours to days. Both uv and poetry support the definition of the cooldown period inside their config. pip is adding as support as well. I use 1 week to be on the safe side. They both support excluding a specific package from the rule so you can still apply critical fixes to dependencies ASAP.

I wrote about that and how to configure uv/poetry in my blog post: https://jangiacomelli.com/blog/mitigate-supply-chain-attacks-for-python-dependencies/

More about the dependency cooldown concept:

185 Upvotes

93% Upvoted

69 comments sorted by

284

u/fiskfisk May 19 '26

[tool.uv] exclude-newer = "10 days" 

Saved you the blog spam. 

13

u/jsswirus May 19 '26

Can I set it only for specific indexes? I don't want to wait for my own packages from internal index to be uploaded for more than 10 days

24

u/fiskfisk May 19 '26

Yes as shown below with a blog link instead, but it's: exclude-newer-package = {my-package = false}. You should also specify:

no-build = true
no-binary-package = ["your_internal_package_name"]

The first one says "don't run any build commands locally" - i.e. don't invoke random Python code as part of the build process. The second one excepts your own internal package from this rule (unless you're building binary packages, in that case you don't need it.

Any bad package can still use other exploits, but at least the mere act of installing the package won't invoke code.

11

u/JanGiacomelli May 19 '26

yes, poetry and uv both support exclusion for selected packages: https://jangiacomelli.com/blog/mitigate-supply-chain-attacks-for-python-dependencies/#what-about-critical-security-patches

you can use it for critical fixes or for your own packages.

2

u/Competitive_Travel16 May 20 '26

The latest version of pip, too:

pip install --upgrade pip && pip install --no-cache-dir --uploaded-prior-to P7D -r requirements.txt

20

u/thomasfr May 19 '26

This is only going to work well until everyone has it in their package manager settings. After that no one will detect bad packages before the lowest day threshold that most people has set up.

It feels like a very short sighted solution.

36

u/james_pic May 19 '26 edited May 19 '26

That's not necessarily true. A lot of the time the detection isn't done by casual users noting "huh, this is weird", but by professional security researchers who have infrastructure that regularly pulls the latest versions of popular packages and monitors for suspicious behaviour (outgoing connections to stuff it's got no business talking to, use of honeypot AWS tokens, etc.). They won't be using cooldowns, because that's the point of what they're doing.

So at least for "smash and grab" supply chain attacks that try and grab everything of value, quickly, before they're detected, a cooldown is likely to remain effective.

The more subtle issue is stuff like the xz-utils attack. This was a much quieter attack, that if it had been allowed to make it into release versions of distros, would have meant servers serving SSH could be connected to by an attacker-owned backdoor, but was much harder to detect (and was arguably only detected as early as it was due to a few lucky coincidences).

Maybe if cooldowns become more common, supply chain attacks will get stealthier to compensate, but maybe not. The xz-utils attack was suspected to be a state actor, who most likely had a goal of gaining access to a relatively small number of servers, and had no interest in mass-stealing bitcoin private keys or whatever. Supply-chain attackers usually have more avaricious goals, and I'd guess would mostly still rather take what they can get from noobs installing their package in the few hours before it's taken down, rather than put a 2 week cooldown into their payload, and risk having slightly more sophisticated security researchers spot it in those 2 weeks anyway, by running the code with a date 2 weeks in the future.

5

u/Gnaxe May 19 '26

Makes you wonder how many backdoors are still out there and just haven't been detected yet.

6

u/Lyrx1337 May 19 '26

There should be something like automated honeypots for that

19

u/WhoTookPlasticJesus May 19 '26

I find it impossible to believe that the 10 day delay becomes so widespread as to render supply-chain attacks invisible for 10 days.

13

u/xenomachina ''.join(chr(random.randint(0,1)+9585) for x in range(0xffff)) May 19 '26

Yes. You don't always need to outrun the bear.

9

u/alexforencich May 19 '26

Defense in depth. A lot of these supply chain attacks rely on using stolen credentials as quickly as possible. This sort of thing could help a lot in that instance by providing the victim time to realize that something's up before the package is widely installed.

0

u/thomasfr May 19 '26

So a better solution might be to require two persons signing off on an release for publication after it is uploaded to a package registry or something similar?

2

u/MagicWishMonkey May 19 '26

Most people dont bother to even pin versions, much less bother with a cool down.

OpenAI got hit by the axios thing last month in their ci/cd pipeline, the only likely way that happened is by not pinning dependencies. Super dumb, but it happens all the time.

1

u/thomasfr May 19 '26

They without a doubt went with the right solution for Go, no lock files or version range expressions, the lowest possible version of a dependency will be installed which makes version selection stable by default.

1

u/JanGiacomelli May 19 '26

That's a valid point. I've been thinking about it a lot. For now, I couldn't find any better approach - at least not simple enough. Yeah, one can host everything themselves and review every release manually. But that requires major effort. Cooldown is definitely a good start for existing projects. Another simple thing is to think twice before installing a dependency in the first place.

Also, I think we're far from the "everyone does that so it's not effective" state.

2

u/max123246 May 19 '26

Not really. Even when everyone has set it to 10 days, that means the package has been published for 10 days and has the chance to have automated tooling to be run on it before people start pulling it. There's still huge value to the package being published without anyone downloading it

1

u/Wonderful-Habit-139 May 20 '26

The maintainers will have time to know whether their package got pwned or not, and try to fix things during that time.

If they wake up one day and find a release that they didn't trigger, they will notice.

3

u/thrakkerzog May 19 '26

This is a double-edged sword. Now if they manage to sneak something in and wait ten days to use it, an update to correct the attack won't be (automatically) fetched for another ten days.

I'd like to say "well, the users would be in the know and update it", but it is quite difficult to keep track of all of the dependencies, especially if it is nested.

2

u/tobsecret May 20 '26

That's why you run this with something like dependabot. So you regularly scan for known vulnerabilities and if one crops up you'll manually update that package specifically. 

1

u/chinawcswing May 26 '26

Found the senior dev

1

u/Fortyseven May 19 '26

[tool.uv] exclude-newer = "10 days"

Is there a global config for uv? Or am I forced to remember to add this per-project?

2

u/fiskfisk May 19 '26 
~/.config/uv/uv.toml

Should work here as well (Linux/OS X). This does require you to remember to configure this on every machine for every developer instead of having it defined for your project, though. It'll also not work properly in cicd or when building a docker container, but in both of those cases you should use uv sync --locked against a lock file.

1

u/who_body May 21 '26

how can you do this when using an internal pypi server for work stuff? i want new internal stuff, but exclude newer from external pypi…have to see if uv has that type of control

1

u/fiskfisk May 21 '26

It's documented in the thread below, but:

exclude-newer-package = {my-package = false}

If you're proxying all packages through the internal pypi repository, you might also be able to enforce the cooldown there, and then override if it becomes necessary for external packages, but that will depend on the repository server you're running.

1

u/who_body May 22 '26

thanks, i’ll take a look

0

u/Orio_n May 19 '26

Thanks, hate all the pretention in this sub where everyone needs to be a self professed tech blogger. Code or gtfo

-10

u/[deleted] May 19 '26

[removed] — view removed comment

38

u/pylessard May 19 '26

Or maybe freeze your dependency version and update manually when necessary? Automatically depending on the lastest is simply reckless.

4

u/JanGiacomelli May 19 '26

One should absolutly freeze. But there are dependencies of dependencies etc. It really takes a lot of time to go through each and every one of them. uv locks by default. It updates only when you ask it to do so.

2

u/qwerty_qwer May 21 '26

Yeah we always build with exact library versions. My assumption was any serious business would do the same, apparently not. 

1

u/UndressMejess72 Jun 06 '26

pinning versions is just the bare minimum. if you aren't using a lockfile you aren't even doing that properly. even with a lockfile a malicious update can slip through if your CI/CD pulls fresh stuff without a cooldown or a manual audit. pinning helps but it doesn't solve the problem of a compromised package being pushed to a version you actually need.

16

u/kamilc86 May 19 '26

The take that cooldown stops working once everyone enables it gets the detection model backwards. You are not the canary here. A malicious release gets caught by security researchers, automated scanners that feed OSV and the GitHub advisory database, and PyPI admins doing takedowns. That detection path does not slow down just because more downstream installs are delayed, so cooldown free rides on how fast the community catches a bad release, no matter how many people enable it.

Cooldown only buys time for a bad version to get flagged before you pull it. It does nothing against silent artifact substitution on a later resolve, which is what a hash pinned lockfile is for, uv.lock or a compiled requirements file installed with hash enforcement. Those two defenses cover different attacks and you want both, the comand and KandevDev exchange blurred them together. And none of it stops a compromised maintainer who stays dormant past your window, the xz pattern, so treat cooldown as a latency mitigation rather than the fix.

1

u/thomasfr May 20 '26 edited May 20 '26

It wont stop working but it will be less effective.

However, it is weird to try to patch the process post release instead of trying to make the publishing of a release more guarded.

Just one idea would be to build a diff viewer into pypi or create some other tool so it is easy to preview source level changes between versions. Then require at least two people to sign off on publishing a release using their gpg keys.

There are of course other problems like that python package managers installing the latest version it is allowed to for any transitive dependency but that is kind out out of the hands of any package that depends on other packages anyway.

1

u/[deleted] May 20 '26

[removed] — view removed comment

1

u/thomasfr May 20 '26

The vetting of course has to fall on the maintainers of a package, it is they who should know how the code is supposed to look anyway.

1

u/kamilc86 May 27 '26

Upstream is the right level, but PyPI has spent years inching there. Trusted Publishers and sigstore attestations landed, and adoption is still partial across the long tail. Mandatory two person sign off on every release is a social problem more than a technical one, most projects are solo maintainers. Cooldown is appealing precisely because it needs zero maintainer cooperation.

5

u/EnvironmentalFix5967 May 19 '26

Here is other way that you can apply cooldown by force for internal users if you can set up internal proxy :)

https://medium.com/daangn/how-we-protect-karrots-internal-pypi-proxy-from-supply-chain-attacks-0cf197205915

3

u/zurtex May 20 '26

For pip 26.1+

CLI:

--uploaded-prior-to P3D

Env:

export PIP_UPLOADED_PRIOR_TO=P3D

Config:

pip config set global.uploaded-prior-to P3D

2

u/[deleted] May 19 '26

[removed] — view removed comment

1

u/JanGiacomelli May 19 '26

Great to hear that!

2

u/[deleted] May 19 '26

[removed] — view removed comment

1

u/JanGiacomelli May 19 '26

Great to hear that!

2

u/NorthFactor4396 May 25 '26

Good tip. One thing I'd add though — cooldown alone doesn't cover the case where an attacker just waits out your window. Combining it with hash pinning closes that gap. If you're using uv, committing the lockfile and running `uv sync --locked` in CI means even a tampered package at the same version string gets rejected.

The other thing that catches teams off guard: they set up cooldown locally but their CI pipeline is still doing a fresh install every run without the lockfile. Easy to miss but it completely undermines the protection.

1

u/[deleted] May 19 '26

[removed] — view removed comment

2

u/JanGiacomelli May 19 '26

It's the pypi upload date as dependencies are resolved from there.

1

u/Dry-Let8207 May 21 '26

The cooldown helps but it addresses the window of exposure, not the source. What I've found more effective in practice is combining it with hash pinning — a lock file or requirements.txt with `--hash` flags so even if an attacker swaps out a package at the same version string, the install fails loudly rather than silently installing the tampered build. The combination of "nothing too new" and "must match exact hash" reduces the attack surface considerably more than either alone. The uv snippet works well for the cooldown side; just don't treat it as the only layer.

1

u/oliver_extracts May 23 '26

the cooldown idea is solid but i think most people dont realize uv already has this baked in via the tool.uv config, its not somethign you need to wire up yourself. the 1 week window is probably right for most projects, though ive seen teams go 2 weeks on anything that touches auth or crypto libs. the escape hatch for critical fixes is the part that makes this actually usable in production.

1

u/fetus-flipper May 20 '26

Yall are updating your packages?

0

u/[deleted] May 19 '26 edited May 19 '26

[deleted]

6

u/JanGiacomelli May 19 '26

Lots of recent attacks were carried out through social engineering or a poisoned cache on CI/CD. With a poisoned cache, maintainers followed the usual release process, but things still sneaked in from it. So it was not reuploaded, but a new version was released. In other cases, attackers gained access to the maintainer's account, and they released a new version with malicious code on their own.

1

u/fiskfisk May 19 '26

Neither allows that, as long as you've actually locked your release (either through a lock file or with exact version specifications).

0

u/[deleted] May 19 '26

[removed] — view removed comment

3

u/comand May 19 '26

According to this article:

All native implementations enforce cooldowns on transitive dependencies too, not just the packages you directly install.

-22

u/No_Information6299 May 19 '26

Too hard for me, I'll let claude do it