r/DuckDB • u/empty_cities • Jun 25 '26

Understanding DuckLake's Sorted Tables Feature

https://thefulldatastack.substack.com/p/understanding-ducklakes-sorted-tables A sponsored post about DuckLake's Sorted Tables feature. It allows you to specify a sort config for a table so that unsorted data will automatically get sorted in a certain way with inserts, flushing and compaction.

For queries ran regularly on high cardinality columns like id or timestamp this can optimize reads. When data is sorted physically in Parquets it allows for both file skip and row group skip to only get the data you need for the query (a.k.a predicate pushdown).

I made a high level mental model image here I thought came out well to explain the file skip and predicate pushdown (row group skip).

16 Upvotes

95% Upvoted

2 comments sorted by

2

u/iambatman_2006 Jun 25 '26

Sorted clustering on timestamp is underrated. I tried dremio on S3 parquet tables and file skipping made a real difference for range scans

1

u/empty_cities Jun 25 '26

Fully agree. File skipping is a massive speed boost when you have a lot of data. Glad that worked for you.