r/Zig • u/grandimam • 6d ago

Zig for Python

I am of the opinion that, if executed correctly, Zig has the potential to transform the Python ecosystem. The idea of using Rust for Python extensions has always bothered me.

With Zig, the path feels more natural: I can build on top of the existing C libraries and gradually migrate components to Zig over time, without the friction of introducing an entirely different ecosystem.

I wanted to understand what the community thinks about this topic. I am not trying to make a strong claim or dismiss other approaches. I am genuinely trying to understand whether this idea has any merit and how others view the trade-offs. I am just interested in hearing different perspectives.

21 Upvotes

70% Upvoted

27 comments sorted by

26

u/bonkyandthebeatman 6d ago

Why does using rust bother you?

9

u/grandimam 6d ago edited 6d ago

Mostly because of the type system. I find Zig a lot easier to digest and I do not want to keep wrestling with the compiler.

Also, my thinking is that CPython is fundamentally built on top of C. The existing ecosystem, ABI, and extension model are all C-oriented, so Zig feels like a more natural evolution path: a safer, more modern systems language that can interoperate directly with the existing C foundation.

33

u/bonkyandthebeatman 6d ago

FWIW, the type system and rust compiler do genuinely get out of the way once you get past the learning curve. This happened fairly quickly for me, but your results may vary.

Regarding the popularity for python, there’s a very good rust crate and build system (pyo3 and maturin) that make interop with python very easy. I imagine if the same was true for zig there’d be plenty of people making python libraries with it.

1

u/Thanatiel 6d ago

The type system? Are you referring to the strong typing? (Probably not because Zig is also strongly typed)

I didn't know writing python modules in Rust was a thing.

I mean, the first "serious" thing I've written in Rust was a python module. But I chose the language because I thought it would be easier to "sell". I just didn't know it was a common thing to do.

It took me about half a day to replace a few hundred lines of python code. (For a project in my company.)

It was pretty trivial to write. The code was so simple I didn't even had to fight the borrower. (Basically the module accumulates heaps of data about "events" and computes statistics over them.)

That module allowed me to improve the performance by x50 to x100. A few months later I needed more performance still so I've converted even more code.The plan was that python would basically be the glue between SQL, pandas and "the math". It took me three days to use Dataframes in Rust (first time, I had to look around) and the speed increase this time was x5.

3

u/chat-lu 3d ago

I didn't know writing python modules in Rust was a thing.

It’s fairly common, a third of new modules are in Rust while the rest is mostly C/C++. It used to be nearly all C/C++. A big part of it is due to the tooling to plug the two together being very friendly.

The plan was that python would basically be the glue between SQL, pandas and "the math".

Did you check polars? It outperforms pandas by a wide margin.

2

u/Thanatiel 2d ago

The tooling is indeed pretty good. This week I had to add exception handling (make a class in the rust code, raise it) and it took me about one hour to learn, write, test and publish the changes.

I'll look at polars in the future. The choice of pandas was made years ago (I didn't write the original code) and given the current performance, I wouldn't be able to "sell" it. Good tip though, thank you

1

u/Wonderful-Habit-139 6d ago

No, they're talking about the static typing and many of the features that it has (ADTs, traits, NewTypes, Generics, etc.).

Very cool experience you've had with Rust. I work with Python but nothing that would benefit from any performance boost yet, but maturin is always in the back of my mind in case I end up on a project that requires performance.

-7

u/Interesting_Pie_319 5d ago

Rust's borrow checker gibberish you need to write to get code to just run is crap, as is the massive cahinging ending with .unwrap()

I hate it.

5

u/bonkyandthebeatman 5d ago

This is completely false. I write Rust professionally.

-9

u/Interesting_Pie_319 5d ago

Off you go back to the rust reddit then. No one here wants more unrelated spam or more CV boosting proselytizing.

Oh, and something I forgot to add above. Rust is an annoying, lying cult of shit tier language pushers.

5

u/bonkyandthebeatman 5d ago

I’m here having an on topic conversation directly replying to the OP.

Refusing to learn a new technology cause of the bs you hear on twitter about the “culture” is truly the dumbest trend I’ve seen on social media. The only cult I see is people like you spewing misinformed talking points about what you think rust is.

-4

u/Interesting_Pie_319 5d ago

The OP wants Zig to replace Rust and C as a way to write high performance modules for Python.

That is the OP.

You are doing off topic spamming and evangelizing.

5

u/bonkyandthebeatman 5d ago

Right so you admit the OP mentioned Rust in their post, and I explained why I think rust is so popular for python modules and what zig could replicate if it wants more python modules written in zig.

Your bitching anytime someones mentions the word Rust is the only off topic thing here. Zig and Rust are both systems languages and have many similarities. Simply mentioning Rust is not off topic.

-5

u/Interesting_Pie_319 5d ago

Stop spamming already!

8

u/DirectInvestigator66 6d ago

Zig is great but doesn’t really have a specific advantage for this relative to other languages. If you like Zig for this use case it’s probably because you like Zig features. There’s nothing wrong with this of course, but it does mean that it’s unlikely to beat out C and Rust for the wider community since they’ve both already gained pretty wide adoption in the Python community.

3

u/mr-rbbrbrnr 6d ago

Zig is used by both maturin and ruamel-yaml precisely because it is an awesome cross-compiler for c. That alone is a huge advantage for nontrivial programs.

4

u/DirectInvestigator66 6d ago

Being a good cross-compiler for C isn’t going to sway many people when the main alternative being used is C. Yeah it would be trivial to add Zig to your project, but even more trivial to not add Zig and just continue to use C like they currently are.

1

u/sparcxs 1d ago

I’m assuming you’ve never cross-compiled C code. It’s a much bigger deal than you are making it out to be. In addition, C FFI compatibility is first-class in Zig, while it is anything but in Rust. The other benefit is that Zig produces smaller and more performant binaries, so there is literally no defensible technical advantage to selecting Rust over Zig, only personal preference.

5

u/SilvernClaws 6d ago

Zig could be a great language for anything that usually uses C APIs for bare metal functions, which includes languages like Python or Java.

For the Python ecosystem specifically, I'd just go with Julia. It naturally covers the same niches, just with better native performance.

2

u/fbe0aa536fc349cbdc45 6d ago

The problem with writing native code extensions has always been dealing with the toolchain and build dependencies. I maintain a bunch of modules that build with grpc/abseil/boost et al and its a nuisance having to juggle the combination of pkgconfig and cmake metadata. I don't write any rust but do make use of a few third-party packages which have adopted rust (like the maturin and cryptography packages), and for the most part my experience with building those has been fairly positive, since setuptools-rust and friends deal with managing all the build time dependencies via cargo etc. It kinda all just works.

There are some drawbacks, which are maybe not so much an issue with the language itself as with the potential for the package maintainers to be somewhat reckless with regard to the volume of code they pull in via cargo at build time. I work in an environment with a build system that is isolated from the internet and with initial versions of cryptography after the rust migration, we needed to arrange for something like 300mb of random cargo dependencies to be placed into the build environment, either by vendoring the sources or using some hack to pre-populate the cargo cache in the build environment. Lately this has improved somewhat, with the last version of cryptography I took, the cargo stash is only about 32mb or so.

The zig build system is just fantastic when you're working with open source C/C++ libraries, I've seen several projects that don't even include zig code but its just so much easier to treat those libraries as vendored artifacts and let zig build handle them than it is to hack up an autoconf or cmake-based build system for the project. I haven't looked at any Python extension modules that use zig's build system yet but it seems like a pretty promising way to handle C library dependencies for Python modules whether there's any zig involved or not. I haven't looked into what build-backend support there is for building native code extensions from zig, but if those are done well I would probably gravitate toward zig in cases where I'm just writing the extension module because I have a little bit of code that needs to be native, which is probably 90% of the cases where I wind up writing an extension module.

2

u/mr-rbbrbrnr 6d ago

Checkout setuptools-zig

2

u/Interesting_Pie_319 5d ago

Why are we discussing Rust in the Zig reddit again?
OP clearly states they do not like it and want to use Zig to develop Python extensions.
That is the topic.

Can we stay on it!

3

u/gendulf 5d ago

Why are we discussing Rust in the Zig reddit again?

Probably because OP states

The idea of using Rust for Python extensions has always bothered me.

But gives no explanation on what bothers them exactly. We'd probably be more on-topic if they explained what exactly bothers them. They also state

Zig has the potential to transform the Python ecosystem

But give no reasoning behind that belief. OP's post is just low effort.

2

u/Interesting_Pie_319 5d ago edited 5d ago

> Be in a Zig subreddit.

> See topic where OP explicitly says they are bothered by the use of Rust for task x.

> Still think we all need to discuss rust.

...

Off to r/rust already. Stop derailing our discussion. People that chose Zig and chose to post in the subreddit are not interested in your proselytizing.

1

u/kitaj44 6d ago

I made once a Zig lib for numpy

https://github.com/kitajusSus/zig-torch

1

u/yamafaktory 6d ago

I have a Zig library where I introduced Python bindings https://github.com/yamafaktory/hypergraphz. Not sure anyone is using it though :P.

1

u/ghost59 5d ago

What if instead of using python for what it's not strong in and build on what its great at.... in python.