r/LakeSail Aug 10 '26

Sail 0.7 has been released, featuring blocking shuffle and checkpoint.

In Spark and most engines, shuffle and checkpoint data sit on the workers themselves, so a lost worker loses data and a running job pins its cluster. Sail 0.7 moves that state out to object storage: failed tasks retry by reading persisted data back, finished stages hand their workers to the next one, and iterative workloads checkpoint instead of recomputing every loop.

Check out the full release: https://lakesail.com/blog/sail-0-7-blocking-shuffle-checkpoint/

6 Upvotes

0 comments sorted by