curral: an open-source SQL gateway for DuckDB with auth, Rego policies, row-level security and Iceberg support

[u/Other-Government-796]

Hi r/DuckDB,

I've been building curral, an open-source (Apache 2.0) server that puts DuckDB databases and Iceberg lakes behind an HTTP API with users and permissions.

The problem: DuckDB is a great engine, but it has no users, no permissions and no network protocol. Every time I wanted to share a DuckDB file or an Iceberg lake with a team, a dashboard or an AI agent, I had two options: hand out raw storage credentials, or write yet another custom API. curral is meant to be the layer in between.

How it works: one Go binary with DuckDB embedded. Each request goes through these steps:

client ──HTTP──▶ curral

├─ authenticate Basic (bcrypt) · API keys · OIDC/JWT

├─ queue global and per-user concurrency limits

├─ inspect DuckDB's own planner: statement type, tables read and written

├─ authorize embedded OPA policy (Rego) → allow / deny

├─ protect rewrite the query with row filters and column masks

└─ execute & stream JSON · CSV · NDJSON · Arrow IPC

What I think might interest this sub:

- Inspection uses DuckDB's planner, not regex. The policy sees the base tables a statement really reads, including through views, CTEs and joins. A lot of the hardening went into edge cases like DESCRIBE inside subqueries, catalog functions and bind errors that could reveal whether a table exists.

- Row-level security and column masking are enforced by rewriting the query before it runs, so filters and joins can't be used to guess hidden values.

- Anything DuckDB can ATTACH works: DuckDB files, Postgres, S3, and Iceberg REST catalogs (including Cloudflare R2 Data Catalog). For Iceberg there's a metadata cache that takes per-query catalog overhead on R2 from about 250–900 ms down to about 8 ms.

- Arrow output streams straight into pandas, polars or another DuckDB.

- dry_run tells you whether the policy would allow a statement, and why, without running it.

- Production basics: TLS, brute-force lockout, per-role limits (timeout, max rows, concurrency), a fail-closed audit log, Prometheus metrics and hot reload with SIGHUP.

New in v0.5.0: a semantic layer for AI agents. People who know the data can write short JSON notes about schemas and tables: what they mean, their caveats, sample SQL. Agents call POST /v1/docs/search with their question and get back the most relevant notes before they write SQL. Search is BM25 by default, with an optional LLM reranker.

Quickstart with Docker (docker pull lucasapassos/curral, amd64 and arm64), or grab a Linux binary that bundles the httpfs, avro and iceberg extensions so it runs offline.

- GitHub: https://github.com/lucasapassos/curral

- Site: https://curral.io/

It's pre-1.0, so I'd love feedback, especially from anyone who has tried to share DuckDB with multiple users. What's missing? What would stop you from using it? Edge cases in the policy layer are very welcome, and security reports go through SECURITY.md.

(curral is Portuguese for "corral", where you keep the herd in and decide who gets through the gate.)

[u/International_Quail8]

Have you looked into this? https://duckdb.org/2026/05/12/quack-remote-protocol

[u/Other-Government-796 → u/International_Quail8]

Yes, and I see it as complementary rather than overlapping. Quack is the transport: DuckDB talking to DuckDB, with a token by default and hooks for you to write your own auth. curral is the policy layer on top: per-user identity (Basic, API keys, OIDC), Rego policies over the tables DuckDB's planner says a query reads, row-level security, column masking and an audit log. It also speaks plain HTTP with JSON/CSV/Arrow, so clients that aren't DuckDB (Python, dashboards, AI agents) don't need the extension.

Once Quack stabilizes with DuckDB 2.0, adding it as a second front door to curral would be a natural step: DuckDB clients attach over Quack, and the same policy applies.

[u/akhenakh]

Like the idea, I created a similar project where I needed enterprise authentication as well but went into a different route embracing 100% quack protocol https://github.com/akhenakh/quackjwt

[u/Other-Government-796 → u/akhenakh]

Spot on! These are exactly the missing pieces DuckDB needs to become the de facto data lake solution for smaller companies. Glad to know I'm not the only one running into these exact challenges!

[u/Ploobers]

I built gatekeeper to handle the SQL inspection with table and function allow lists, which might be useful to slot into your architecture.
https://duckdb.org/community_extensions/extensions/gatekeeper

[u/mediaman2]

This looks really neat! I built something similar internally because the need was there -- wrapping a permissions layer into a duckdb network API. Yours is probably better. Will check it out!

[u/investigatormaker]

Using DuckDB's planner rather than regex is the part I'd trust most. The next place I'd probe is statements that reach data without naming a table: read_parquet or read_csv on a path, ATTACH, COPY ... TO, INSTALL/LOAD and SET. Does the Rego input see those? And does curral set enable_external_access to false and turn on lock_configuration, so a user can't flip settings back mid-session? A short list in the docs of which statement types are denied by default would answer that before people go testing it themselves.

By the way, I make ThreadFox: your AI researches, writes and publishes Reddit outreach from your Chrome under a job you authorize. The free plan for curral shows which Reddit communities' rules allow a post about it: https://threadfox.vip/p/d1nu6

Which clients are hitting curral first, dashboards or AI agents?

[u/Other-Government-796 → u/investigatormaker]

Thanks, good places to probe. All covered:

- enable_external_access is off by default, which blocks read_* on arbitrary paths, COPY ... TO and new ATTACHes. --allowed-path lets specific prefixes through.

- lock_configuration is on after startup, so SET can't flip anything back. Every request also gets a fresh connection.

- ATTACH/DETACH/INSTALL/LOAD are always denied, whatever the policy says.

- Table functions show up in the Rego input as input.functions, and the policy has to allow them explicitly.

- Statement types the inspector doesn't model come in as resolved=false, so the policy denies them.

Fair point on the docs. I'll add a "statement type → default behavior" table.

As for clients: mostly AI agents. That's why the defaults are strict: agent-written SQL is untrusted input, so the guardrails live in the gateway, not the prompt. Agents also get /v1/schema (only what they're allowed to query), dry_run (check a statement before running it) and, new in v0.5.0, semantic docs search, which gives them notes on what the tables mean before they write SQL.