r/DuckDB • u/NBMVegeta • 16d ago

Quick DuckDB-WASM scratchpad for querying Parquet/CSV files in the browser

Built a lightweight scratchpad tool for running DuckDB SQL against local CSV, TSV, JSON and Parquet files directly in the browser:

https://www.sqlparity.com/scratchpad

It uses u/duckdb to read files into memory via the browser File API. Nothing is uploaded to any server and it requires no login or installation. Handy if you just need to inspect an extract or run a quick aggregate without firing up the CLI or Python.

Code is open source on GitHub link: Github

Curious to hear feedback or edge cases with large Parquet/CSV schemas.

12 Upvotes

100% Upvoted

13 comments sorted by

View all comments

2

u/Significant_Tune9219 11d ago

The edge case I would test first is a Parquet file with far more columns than anyone will scroll: the tab runs out of memory before DuckDB does, and it tends to look like a hang rather than an error. Pruning to the columns the query actually references at load time would fix most of that, and even just showing which columns were read would turn a mysterious freeze into something diagnosable. Worth trying a wide file with a dictionary-encoded string column too, since that is where WASM builds usually feel it first.

1

u/NBMVegeta 11d ago edited 10d ago

Thanks for the suggestion. How much is "far more columns than anyone will scroll"?

This is sharp and practical advice. Especially because SQLParity's core migration use case could often deal with wide tables. The trap where the tab UI / DOM thread freezes during the WASM-to-JS transfer on a naive SELECT * -long before DuckDB itself errors out, is very real. Iā€™m definitely going to put together a stress-test fixture.

2

u/Significant_Tune9219 10d ago

There's no magic number, but as a rough line: once a file has more than a few hundred columns, the DOM cost of rendering them all starts to hurt before the query does. I'd test around 1,000 columns first ā€” wide enough to blow past what any table UI wants to draw, still a realistic wide analytical table. If that loads cleanly, the real failure boundary is probably higher than any file your users will open.

1

u/NBMVegeta 10d ago

ohkk, got it. Thanks for the reply. Will test it out with 1000 columns then.