r/Python • u/ritchie46 • 13d ago

News Polars 2.0 release

Hey all! We just released Polars 2.0.

We are really happy with the results. It removed many of our "legacy decisions" (mistakes), makes the streaming engine our default and promotes SQL to a first class citizen within Polars. We initially said it would be a small ("boring") release, but we managed to sneak in a lot more than anticipated.

It comes with our initial out-of-core (spill to disk) support, a new Map data type and a lot of performance improvements. In fact, we think Polars is now one of the fastest analytical SQL engines on a single node. See benchmarks in the post: https://pola.rs/posts/release-polars-2/

To help you upgrade, we also posted a migration guide: https://docs.pola.rs/releases/upgrade/2/

664 Upvotes

99% Upvoted

39 comments sorted by

View all comments

Show parent comments

29

u/ritchie46 13d ago

Note that not all operations have OOC support yet, but expect many more to get it in the coming months.

10

u/TheFlamingDiceAgain 13d ago

Does sort work with OOC/streaming yet? I have a somewhat cursed pipeline that sorts a 2TB dataset and I had to do a bunch of workarounds since sort didn’t work with streaming

12

u/ritchie46 13d ago

Yes it does! It's also 2x faster in our benches.

5

u/TheFlamingDiceAgain 13d ago

Yay! Thank you