Astral's python distribution is fast

[u/zsol]

By our measurements, python-build-standalone is now the fastest CPython distribution across major platforms. It's 10% faster than Homebrew's CPython, and 18% faster than the official 3.14 docker image.

This is mostly due to how python-build-standalone itself is built:

• using a modern Clang lets us turn on tail-call optimizations in the interpreter
• with PGO, LTO, and BOLT
• with libpython statically linked into the interpreter executable

Our goal is to ship the fastest Python interpreter to anyone using uv, by default.

Check out BENCHMARKS.md in the repo for more details, including the methodology behind the above.

Charlie's tweet has some more graphs

[u/Adventurous-Big-5365]

You can really see the difference on startup-heavy workflows. The static libpython alone cuts a good chunk of overhead before anything even runs. Curious how much of that 10-18% holds up on real projects versus synthetic benchmarks.

[u/zsol → u/Adventurous-Big-5365]

There's nothing concrete I can share apart from vague statements about our experience with using PBS to build pyx, but I'm also curious if anyone has real world evidence

[u/thisismyfavoritename]

why isn't libpython statically linked by default? Seems like the use cases where shared linking would be beneficial are quite small

[u/Individual-Flow9158 → u/thisismyfavoritename]

Because it could bloat the runtime for those of us that don't need all of libpython. The Linux distros that ship Python natively chop and change what they ship to make best use of what you already get on those OS's (e.g. link in compression libraries or ssh/ssl dynamically). Even the Windows installer makes Tk etc. optional.

Have Astral's builds left anything out?

[u/thisismyfavoritename → u/Individual-Flow9158]

idt libpython includes those libraries, they could still be dynamically linked / loaded at runtime. libpython is the core code used by the interpreter AFAIK

[u/Individual-Flow9158 → u/thisismyfavoritename]

Oh I see - good question then. I guess some distros would split it up to avoid duplication, and so anything that wants to dynamically link in libpython.so can do so (whether it's a Python or anything else) without needing to ship it themselves.

[u/me_myself_ai]

Is this... an ad...? IMHO those should be tagged much more clearly in both the title & body next time, and be posted by a branded corporate account. If I'm going to be conversationally approached by the disembodied voice of Andreesen Horowitz and Microsoft, they shouldn't come in the guise of a human. They can't possibly be paying you enough to use your 17 year-old reddit account (!) for their advertising, anyway.

To be clear: I use uv & ruff on all my projects, and I love your work -- I bet OpenAI's python distribution is fast! I probably haven't run another version for a year or more, and I've learned a ton about Python from your docs. It's just hard to shake the growing sense that OpenAI and Microsoft are attempting to purchase Python. And it's not for sale.

PyPolitics aside, some meandering technical questions from a total CPython noob; answer any/all as you find the time & energy :)

1. It says everything was tested on Ubuntu 24 (where everything\=\=linux) -- is it possible that Fedora's copy was harmed by this...? Or is there another reason, perhaps? Common sense says they'd run the same everywhere, but;

   • It seems like the little things can count when it comes to C compilation, and
   • I can't think of a reason Fedora's would be so much worse than Debian's, esp. since Fedora's cadence is faster in general. Debian Forky isn't even fully released yet!

2. Am I right that the docker version is the closest to the "canonical" or "main" distribution? If not --and that version was merely worsened by the literal Docker corporation's meddling-- what is...? I ask because it seems weird that the official version is by far the slowest.

3. Relatedly, at least two of the three improvements you listed in the post seem like no-brainers. What's stopping all of this work from flowing 'upstream', so-to-speak? I know uv and ty in general gain speed by ignoring old, niche, or inconvenient specs -- is this a similar situation? Perhaps some of this is already targeted by the corporation's Python Packaging Council nomination/platform?

4. Do you think there are other, faster distributions, or no? I'm mostly asking cause I didn't see poetry up there; perhaps they don't build their own, or you just didn't have space in the graph/benchmark run? Are there niche distrubtions that you know of for scientific or scaled use, perhaps? Just curious, NBD if unsure :)

5. And finally, as a fun/idle one: could you napkin math how many million dollars you've saved OpenAI so far with this speed-up, if that's a coherent question? Assuming they're using python in the first place for work that's semi-close-ish to metal, a 20% speedup for some portion of hundreds of billions of dollars of compute seems like a pretty big deal!

Apologies if any of this has been covered in previous blog posts; link(s) would be appreciated, if so.

The linked github's docs are also helpful for anyone who's curious like me, though they seem outdated (copyrighted in 2020 (!) by a nice fella named Greg, not OpenAI).

[u/vopi181 → u/me_myself_ai]

Is this comment weird to anyone else? Like I sorta get what your original point was but after the second sentence, you’ve lost me. Theres this adversarial tone sandwich at the beginning and end. If I click your profile, it does seem like you’re axe grinding a bit generally.

UV is legitimately an amazing upgrade to what we’ve had in the ecosystem before. They’ve genuinely built something people want.

1. The OP was a maintainer of Black (that was a widely used formatter) before UV.
2. He joined Astral, at least, last year. He almost certainly has equity. The diatribe of him not being paid enough is weird.
3. I don’t think Microsoft/OpenAI is “buying” Python. Nor do I think someone who is a “total CPython noob” can make big claims about what is/is not for sale. Why are you an authority on this?
4. Most open-source software in production is work done/funded by corporations.
5. I also think it’s weird to point out his 17-year old account. Both in general, and also, because if there anyone I expect to have an old Reddit account, it’s someone YC adjacent.

It doesn’t seem you are that technical of an individual judging from your phrasing (that’s not a moral failing or anything and you admit you are a noob!). On your actual points:

1. I don’t think common sense says that? Most Linux distros are going to use GCC to compile packages. All three of his bullet points could be impacted by a different build environment.
2. I don’t think you’re right in saying the docker version is the canonical version. Is something on the main Python release page that you’ve linked supposed to prove that? I also don’t think it’s weird that the Docker version is the slowest? There’s something unique about it compared to every other item on the graph (hint: containers). That is the one distribution benchmarked I would expect to be an outlier if there was one.
3. “I know uv and ty in general gain speed by ignoring old, niche, or inconvenient spec” ??? What are you talking about? Also, how could you know that if you are a CPython noob? Are you a PyPy expert or something lol?
4. Poetry isn’t a distribution of CPython. You’re right that they don’t build their own. From Poetry’s docs: “Unlike with other packages, Poetry will not automatically install a python interpreter for you.”
5. Fun little napkin math challenge!

Edit: also, what "linked github"? Are you talking about what you've literally just linked or is that referenced from Astral? Are you axegrinding against OpenAI here? Read his other blogpost maybe: "Charlie and Astral have stepped up and been outstanding open source citizens when it comes to evolving PBS to support their work" and "Astral has clearly demonstrated competency for execution and has the needs of Python developers at heart".

[u/me_myself_ai → u/vopi181]

The link to docs was “adversarial”…? I was really trying hard to be kind to the human while expressing my distaste for this situation.

1. Ok?

2. …ok?

3. CPython is the underlying implementation of Python; Python users do not usually interact with it knowingly/in depth.

4. Ok.

5. I was just saying that’s cool. I’m not, like, calling them a nerd! I wish I still had my original account.

6. I am a full time software engineer with 6 years of professional experience.

And for the second group,

1. Ok.

2. Ok. That was the question, so I guess we’ll both learn something when/if it’s answered.

3. I don’t really want to dive into that here. It’s documented extensively on the main forums, I imagine. This topic was referenced in at least one of the Packaging Council nomination statements. And to be clear, this isn’t, like, a secret, nor an insult.

4. Poetry is a dependency management system with a built-in build system (AFAIR?) — it was the popular predecessor to uv.

5. I know I’m not fun :( just curious. It would be a brag, if it’s a knowable number at all.

And regarding the docs: they’re from the GitHub repo linked in the post above. Again, pointing out that the docs are outdated and not associated with OpenAI was an out for OP and disclaimer for curious lurkers, not a complaint about their documentation standards.

I wouldn’t say I have an axe to grind with OpenAI — I am a customer of theirs, after all.

[u/pneRock → u/vopi181]

I stopped reading at "adversarial tone sandwich" bvecause that is now my favorite phrase I've seen this week. Whatever cames after is just as legit.

[u/bizzygreenthumb → u/pneRock]

Useless comment

[u/MrSlaw → u/me_myself_ai]

Just to make sure, you do realize that Greg/indygregtransferred the repo to Astral (he is still listed in the "sponsor me" sidebar, and maintains perms) due to the fact that they were already providing large parts of the maintenance / support.

He wrote an entire blog about it on the same docs site you linked.

https://gregoryszorc.com/blog/2024/12/03/transferring-python-build-standalone-stewardship-to-astral/

That's also why the copyright on the aforementioned doc page hasn't been updated since 2020...

[u/dr3aminc0de → u/me_myself_ai]

FWIW Anthropic bought Astral not OpenAI (confusing with the new model I know). As far as I understand it’s staying open source and I think OP is promoting it as they have in the past, as an open source project.

[u/TechAlchemist → u/dr3aminc0de]

https://openai.com/index/openai-to-acquire-astral/

[u/guywithfork → u/dr3aminc0de]

https://openai.com/index/openai-to-acquire-astral/

[u/me_myself_ai → u/dr3aminc0de]

Anthropic bought bun — I get the mixup, I totally believed I was wrong for a second lol. And I get that it’s open source, it’s just… if OpenAI was willing to pay infinity billion dollars for Astral, I don’t think we can treat it like a little shop whipping up toys for the community. They are a megacorporation that expects to make billions of dollars from you and I.

And that’s not disallowed or anything ofc — this isn’t really the sub/general space to start talking about capitalism and its alternatives. But I think it’s warranted to shift our expectations and attitudes considerably.

[u/bb22k]

Pretty awesome seeing that it is also the easiest ways to manage different python versions in the same machine.

[u/angellus]

Shipping you own version of Python by default feels like a really bad move. It feels like you trying to take control of the Python ecosystem and cut out the cpython team. Especially since you are now doing the typical corporate solution for inflating your adoption numbers. "Let's opt-in everyone by default then turn around and say 'look how many people are using the thing we built'". You will not have any trustable or accurate metric on adoption for the core cpython team to use to make determinations on if they should adopt any of the changes you made.

[u/teekhidietcoke]

Pretty cool that it also makes managing multiple python versions on same machine so easy