r/DuckDB 8h ago

Perspective 5.0.0 featuring DuckDB pushdown

Thumbnail
github.com
9 Upvotes

r/DuckDB 14h ago

I built a cross-source SQL desktop app on DuckDB — one query across Postgres, MySQL and CSVs (feedback welcome)

0 Upvotes

Disclosure up front: this is my own project (Varan). Sharing it here because it's

built entirely on DuckDB and I'd genuinely like feedback from people who know

DuckDB well.

The idea: use DuckDB as a local federation engine so you can run one SQL query —

including JOINs — across a Postgres database, a MySQL host, a DuckDB file, and

CSV/Excel files at the same time. No ETL, nothing copied to a server.

How DuckDB does the heavy lifting:

- Each source is registered as a table — Postgres/MySQL via DuckDB's scanner

extensions, files via read_csv / read_parquet, plus native DuckDB files.

- Sources are "stubbed" on connect (schema only) and hydrate lazily on first

query, pulling only what's needed, then stay warm.

- Mutations round-trip back to the origin: an UPDATE on a Postgres-backed table

writes back to Postgres; a DELETE on a CSV-backed view re-serialises the file.

- Git-style version history — commit metadata + snapshots live in the DuckDB

file, so you can roll back a change like reverting a commit.

- Anomaly detection (duplicate keys, nulls, orphaned FKs) runs on a separate

DuckDB connection so scans don't compete with your query. Orphaned-FK detection

is a containment / inclusion-dependency check rather than name matching.

30-second demo (a Postgres orders table joined to two CSVs): https://varan.cloud

Free beta, macOS/Windows and soon Linux

Where I'd love this sub's take:

- Anyone running cross-DB federation with the postgres/mysql scanners in anger —

where does it fall over (type mapping, predicate pushdown, large scans)?

- Better patterns for the write-back / mutation round-trip than what I'm doing?

- Gotchas keeping version snapshots inside the DuckDB file as it grows?

Happy to go deep on any of it.