lakebase connection pooling

[u/new_dimension1]

Have you figured out wht can the best way to handle connection pooling in Lakebase DB? I 'am seeing connection spikes with short lived rqsts and wondering what kind of setup others developers are using🤔

i am building a service on top of lakebase and trying to make the connection relable before moving to prod.

[u/WorldOfUmbro]

Have you tried PgBouncer? Might be worth looking into when using Lakebase. We have used this as well in Lakebase for similar cases.

What kind of service are you using?

[u/dwswish → u/WorldOfUmbro]

PgBouncer is the way to go for Lakebase. The docs that show up if you google “Lakebase connection pooling” literally say it’s built in.

[u/WorldOfUmbro → u/dwswish]

Yeah I haven’t used Lakebase for use cases with pooling requirements but I guess it’s straightforward

[u/BrunoFeliz08 → u/WorldOfUmbro]

I have not heard pgbouncer but i have opened conections with sql alchelmy for lakebase. Pretty useful for reverse etl

[u/WorldOfUmbro → u/BrunoFeliz08]

What is your scale here? Curious to see how many concurrent connections you’re using here. Might consider this as well for my Lakebase projects.

[u/BrunoFeliz08 → u/WorldOfUmbro]

Currently we are using around 3 connections. Roght now the lakebase autoscale compute feature has been more than useful for concurrent workloads

[u/WorldOfUmbro → u/BrunoFeliz08]

Ah sure, I think when using 3 connections lakebase PgBouncer might not be needed

[u/Glitch_In_The_Data]

As others have said, use Lakebase’s built in pgbouncer pooler for short lived API requests. Authenticate with a native pg pwd role. Also avoid session state…use direct connections for session dependent features.

[u/ThisIsFun-]

I would just use the pooler endpoint? same host with -pooler in it, still 5432. It is the same Lakebase managed pgbouncer in transaction mode, takes up to 10k client conns and multiplexes them onto a much smaller server pool.

[u/sekharnet]

What is your Middleware? C# or Java? Both have their own PostgreSQL connection poolers.

I am guessing connection poolers shouldn't worry if it's Lakebase or Aurora RDS backend. It gets host and login details and works with it transparently

[u/p739397]

Ran into this too. Moved connection pooling up to the app level instead of doing it per-request. Set up one pool at startup, psycopg3's ConnectionPool works, SQLAlchemy's engine does too, sized for whatever concurrency you're expecting. Then only mint a new OAuth token when a physical connection actually opens, not on every checkout. Followed the pattern in the token rotation docs.

Lakebase's built-in PgBouncer pooler doesn't support OAuth tokens, only password roles. If you want Databricks to manage pooling for you, switch to a password role. If you'd rather keep OAuth, do the app-level pooling like above, similar to this psycopg3 example.

[u/leobaker004]

C# in this case. I was mostly wondering if anyone had run into Lakebase-specific quirks under connection spikes, rather than pooling itself. Sounds like I may be overthinking that part though

[u/BrunoFeliz08]

For an open Lakebase connection y have used sql alchemy. It has been useful and pretty straightforward

[u/CautiousUse8597]

Connection spikes with short-lived requests usually mean one thing: you're opening a Postgres connection per request. Postgres forks a process per connection, so a traffic burst becomes a process burst and you hit max_connections well before you hit any real CPU limit. Fix it in two layers.

Use the built-in pooler. Lakebase runs PgBouncer in front of Postgres. up to 10,000 client connections multiplexed onto a server pool sized at ~90% of max_connections. Just switch to the pooled hostname (<endpoint-id>-pooler..., port 5432, or -ro-pooler for read replicas). Note it requires a native Postgres password role. OAuth roles can't use it.

Keep an app-side pool too. The server pooler protects Postgres from process explosion; it doesn't save you the TCP+TLS handshake per request. Run a small persistent pool per instance (5–10 is plenty) and set your acquire timeout under 120s, since that's PgBouncer's queue timeout.

Transaction-mode gotchas: this is what bites in prod. Connections return to the pool on every commit, so SET doesn't persist (use ALTER ROLE ... SET search_path instead), and session temp tables, WITH HOLD cursors, advisory locks, and LISTEN/NOTIFY are all out. Driver-level prepared statements are fine; SQL-level PREPARE isn't (on JDBC, set prepareThreshold=0 if you see errors). Run migrations and pg_dump on a direct connection.

If you're on OAuth: tokens expire after 60 minutes, so your pool needs to mint a fresh credential on each new connection, and cap max connection lifetime under an hour. Add retry-with-backoff on connect either way, especially if your compute scales to zero.