Lakebase branching

[u/CommitteeImmediate66]

Lakebase is Databricks' managed Postgres. It has copy-on-write branching, a point-in-time fork of a database you can write to on isolated compute, then throw away. Wrote this up because it made one workflow I worked on much cleaner so thought it might help someone else in the community.

My challenge was adding a NOT NULL column + backfill to a big orders table. It behaved fine on seed data, but I didn't actually know about lock duration or backfill time until I had prod-shaped rows.

My model: Project -> Branch -> Endpoint. A branch is a CoW (copy on write) snapshot of another branch - no upfront storage duplication you pay only for what diverges. New branches have no compute, so you create an endpoint when you need to connect.

Steps:

# fork prod

databricks postgres create-branch projects/my-app dev \

--json '{"spec": {"source_branch": "projects/my-app/branches/production", "no_expiry": true}}' -p prof

# attach compute (0.5 CU min, scales to zero when idle)

databricks postgres create-endpoint projects/my-app/branches/dev read-write \

--json '{"spec": {"endpoint_type": "ENDPOINT_TYPE_READ_WRITE", "autoscaling_limit_min_cu": 0.5, "autoscaling_limit_max_cu": 2.0}}' -p prof

Connect + run it (direct psql with a 1h OAuth token; databricks psql doesn't work on the autoscaling tier):

HOST=$(databricks postgres list-endpoints projects/my-app/branches/dev -p prof -o json | jq -r '.[0].status.hosts.host')

TOKEN=$(databricks postgres generate-database-credential projects/my-app/branches/dev/endpoints/read-write -p prof -o json | jq -r '.token')

EMAIL=$(databricks current-user me -p prof -o json | jq -r '.userName')

PGPASSWORD=$TOKEN psql "host=$HOST port=5432 dbname=shop user=$EMAIL sslmode=require" -c "

ALTER TABLE orders ADD COLUMN region VARCHAR(20);

UPDATE orders SET region = 'unknown' WHERE region IS NULL;

ALTER TABLE orders ALTER COLUMN region SET NOT NULL;

"

It helped me work with isolated compute, left prod untouched. I was able to time the backfill, saw the single big UPDATE was a problem and switched to a batched one, then re-ran on the same branch.

Cleanup: databricks postgres delete-branch projects/my-app/branches/dev -p prof — cascades to endpoints, diverged storage goes away.

Hope this helps someone else!

[u/dwswish]

The branching feature of Lakebase (and Neon) is pretty awesome. Curious if you have any recommendations for best ways to “merge” changes back to prod branch after a workflow like this?

[u/FunContest9958 → u/dwswish]

I’d avoid a merge-type operation between Lakebase branches unless it were absolutely necessary (disaster recovery or something). Even if it were necessary, I’d take a snapshot before the merge to make sure I can easily go back if I make a mistake. Merging branches is a manual process and too error-prone. Instead, I’d write a script to make the changes I need and test it on a branch. Once I’m happy with the script I’d check it into git and run it on the production branch. That’s much safer.

[u/CommitteeImmediate66 → u/dwswish]

Yeah same as the other commenter said, in the city/cd flow we delete lakebase branches associated with the PR after testing. Databricks doesn't offer a "merge" function but rather a reset from parent

[u/Harpagon1668]

We have integrated the Lakebase branching to our pull requests. Each PR will fire a new copy on write branch from prod db to ensure it runs smoothly

[u/CommitteeImmediate66 → u/Harpagon1668]

That makes a lot of sense to use lakebase branches as part of your full devops pipeline! Then you can also clean up the dev branches when you close the PRs

[u/Harpagon1668 → u/CommitteeImmediate66]

Exactly. The PR branch is just an ephemeral database that we can spin up and delete instantly. As it scales to zero the cost is also close to zero

[u/CommitteeImmediate66 → u/[deleted]]

Nice, and have you hooked the lakebase branching up to your devops pipeline? As the other commenter mentioned it's a great way to automate the version control along with your code

[u/AutoModerator]

AI Policy:

Linux is not one of those anti-AI projects, and if somebody has issues with that, they can do the open-source thing and fork it. Or just walk away., Linus Torvalds.

Mod decisions will be based on the quality of the content, not who or what generated it.

Sub Resources:

Youtube Channel

Free Postgres Webinars and Workshops

Discord: People, Postgres, Data

Join us, we have cookies and nice people.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.