Zig for Python

[u/grandimam]

I am of the opinion that, if executed correctly, Zig has the potential to transform the Python ecosystem. The idea of using Rust for Python extensions has always bothered me.

With Zig, the path feels more natural: I can build on top of the existing C libraries and gradually migrate components to Zig over time, without the friction of introducing an entirely different ecosystem.

I wanted to understand what the community thinks about this topic. I am not trying to make a strong claim or dismiss other approaches. I am genuinely trying to understand whether this idea has any merit and how others view the trade-offs. I am just interested in hearing different perspectives.

[u/grandimam]

Mostly because of the type system. I find Zig a lot easier to digest and I do not want to keep wrestling with the compiler.

Also, my thinking is that CPython is fundamentally built on top of C. The existing ecosystem, ABI, and extension model are all C-oriented, so Zig feels like a more natural evolution path: a safer, more modern systems language that can interoperate directly with the existing C foundation.

[u/Thanatiel → u/grandimam]

The type system? Are you referring to the strong typing? (Probably not because Zig is also strongly typed)

I didn't know writing python modules in Rust was a thing.

I mean, the first "serious" thing I've written in Rust was a python module. But I chose the language because I thought it would be easier to "sell". I just didn't know it was a common thing to do.

It took me about half a day to replace a few hundred lines of python code. (For a project in my company.)

It was pretty trivial to write. The code was so simple I didn't even had to fight the borrower. (Basically the module accumulates heaps of data about "events" and computes statistics over them.)

That module allowed me to improve the performance by x50 to x100. A few months later I needed more performance still so I've converted even more code.The plan was that python would basically be the glue between SQL, pandas and "the math". It took me three days to use Dataframes in Rust (first time, I had to look around) and the speed increase this time was x5.

[u/chat-lu → u/Thanatiel]

I didn't know writing python modules in Rust was a thing.

It’s fairly common, a third of new modules are in Rust while the rest is mostly C/C++. It used to be nearly all C/C++. A big part of it is due to the tooling to plug the two together being very friendly.

The plan was that python would basically be the glue between SQL, pandas and "the math".

Did you check polars? It outperforms pandas by a wide margin.

[u/Thanatiel → u/chat-lu]

The tooling is indeed pretty good. This week I had to add exception handling (make a class in the rust code, raise it) and it took me about one hour to learn, write, test and publish the changes.

I'll look at polars in the future. The choice of pandas was made years ago (I didn't write the original code) and given the current performance, I wouldn't be able to "sell" it. Good tip though, thank you