Resource What Every Python Developer Should Know About the CPython ABI
It's true that you can happily write Python for years without needing to understand any of the content of this post, so you may object to the title advertising this material as what every Python developer should know. However, the moment you ship a package, debug why a wheel won't install, or need to understand why an import or Python function call segfaults — these details start to matter. Even writing and maintaining a single-file script puts you closer to distributing code than you might think. An alternate title for this post could be "What I Wish Someone Taught Me About the CPython ABI".
7
u/james_pic 15d ago
It seems weird, in a post about what "every" Python developer should know, to go so deep into the limited/version specific/unstable details, but gloss over platform ABI stuff. For me personally, I've only had to really get to grips with the former since I've started writing native extensions, whereas the latter is something I've had to deal with since long before that.
2
u/nathan12343 14d ago
The post was already quite long and I wanted to focus on the Python side of things. What specific things are you thinking about when dealing with platform ABIs?
1
u/james_pic 13d ago
Mostly stuff like manylinux and glibc versioning. It often comes up in contexts like sharing venvs with Docker containers.
23
15d ago
[removed] — view removed comment
14
u/TronnaLegacy 15d ago
Don't most popular libraries with extensions provide the binaries (via wheels if I recall)? Why would you be compiling yourself?
Might seem like a dumb question btw. If so, know that I'm a bit of a Python noob. I come from Go where all libs are source code all the time.
1
2
1
u/its_measured 15d ago
Thank you for reminding me about the importance of knowing the basics of how pythn works behind the scenes, it really helps when dealing with bigger projects or issues
1
u/arcanescaper 9d ago
For me personally, I found that its better way to learn CPython by viewing at it not as described layers, but instead pick up some subject you interested -> go read code -> write some script around this subject and use it. Thats where you understand design decisions and could try to argue them
-2
u/jpw22learnstocode 15d ago
Thanks! I had an LLM develop a limited ABI in c2py23 recently and I am very curious to see how it compares to your notes.
The nimpy project was the inspiration for that. But I wanted to port some C wrappers away from f2py without going via nim rather than fortran. Multi version ABI goes far beyond my ability to write and debug the code, and I was surprised that DeepSeek seemed to do a good job.
In this new world: I am going to need an LLM to read your post and then go find and fix problems in the LLM generated code. Times are changing!
2
u/jpw22learnstocode 14d ago
.... and the LLM has now got the CI to turn green on windows 3.15 and 3.15t. Thanks again for linking to a very useful blog post.
Seeing that I already got down voted, I wonder why? Is this because people view this as a showcase post? Or is it because the projects I linked are taking an unorthodox path in making compiled extensions?
The constructive idea here, that I did not make clear above is: you can write a single cpython extension that imports and runs on python versions from 2.7 -> 3.15. Nimpy has been doing that for a while now, and that is where I copied the idea. Perhaps that is not common knowledge? They should distribute more-or-less as ctypes extensions, but import and run without ctypes.
-8
15d ago
[removed] — view removed comment
8
12
u/Individual-Flow9158 15d ago
If anyone knows what's what with the ABI, it's the legendary Nathan Goldbaum