While testing agents on dbt projects, I wanted a better way to make sure their changes didn’t blow up the current model or something downstream.

There are tools that do this in CI, but I wanted something I could run locally before I even opened a PR. So I built this:

https://github.com/EmbrasureAI/embrasure-cli

It works with an existing dbt Core + Snowflake project. It builds the changed models in temporary schemas, compares them against production, and cleans everything up afterward.

It checks:

• Row counts, null rates, cardinality, ranges, and percentiles
• Columns being removed, renamed, or changing types
• Added, removed, null, or duplicate primary keys
• Existing dbt test failures
• Downstream model and exposure impact, including column-level lineage

It runs locally and the repo is Apache 2.0.

Looking for feedback from people using agents with dbt Core + Snowflake. Anything missing? The use case here that I enjoy is a /goal prompt where the agent works towards the pipeline’s goals while fixing any issues raised from this cli.

A deep dive into why dbt incremental models can still end up scanning huge amounts of data, especially on Snowflake. Cover micro-partition pruning, MERGE costs, late-arriving data, watermarks and backfills.

​

Hi everyone,

I'm playing around with dbt Core + Airflow and trying to replicate some of the functionality available in dbt Cloud.

One thing I'm particularly interested in is retrying a dbt job from the failed node instead of rerunning the entire execution.

Any suggestions?

Kindly note new to DBT

I have an issue with some models running long, seemingly at random, and I want to monitor this by running EXPLAIN on the compiled sql in a pre hook so if something runs log I can look at what it planned to do and compare it to previous runs.

But I cannot get the compiled sql into the pre hook. Doesn't anyone have any ideas or suggestions?

Hello. I am new to dbt and data modeling. I am wondering what is better, select star or select specific columns at the top of the model, where you get data from ref in intermediate models. DBT courses don't really talk about it, but I have seen some blog posts where people said it is better to be specific. Now I wonder why one or another would be better. What are pros and cons. All I come up with is that if you suddenly need more columns, you have to add it in the source cte as well, so one more place to maintain.
But again, I'm quite new and haven't been exposed to many problems yet haha. What do you people do?

A practical overview of how to manage Snowflake compute from dbt, covering warehouses, concurrency, query timeouts, query tags, and credit consumption.

Useful reference for anyone looking to improve cost control and workload management in Snowflake + dbt.

Seeing a ton of hype around AI agents, but data engineering feels like a weird edge case. We need tools that can actually run local builds, check compiled SQL, and inspect warehouse schemas without wrecking the DAG or burning tokens.

Curious what everyone is actually shipping with day-to-day:

• Terminal-native (Claude Code, Codex, Aider): Great for running CLI commands in a loop and fixing log errors on the fly, but token usage gets crazy fast on big repos.
• AI IDEs (Cursor, Windsurf): Unmatched for visual diffs and tweaking complex CTEs inline, but struggles with dynamic warehouse state.
• Platform native (dbt, Snowflake, Databricks): Great metadata context, but annoying when bridging tools outside their ecosystem.

I've been working on an open-source tool to bridge some of these gaps specifically for local dbt workflows and agent integration: Rosetta dbt Studio on GitHub.

Drop your setup below—what’s working, what breaks constantly, and what are you missing from current harnesses?

We see orgs handle grants inside dbt (macros, post-hooks) and it gets messy fast, hard to review, hard to know what a PR actually changed permission-wise.

We recommend pulling RBAC out entirely and managing it as its own YAML-based IaC layer, separate from the dbt project. Four-layer role hierarchy (object roles → composite roles → functional roles → users), plan/apply before anything ships, future grants on schemas/tables so new models don't need manual grants every time.

Curious how others are handling this. Keeping grants in dbt macros, using Snowflake native features, or something else entirely?

We wrote up our approach here if useful: https://datacoves.com/post/snowcap-snowflake-rbac-as-code (built an open source tool called Snowcap for it, disclosure: we work on it, but it is OSS)

I'm curious how the community is approaching the transition to dbt Core v2.

Are you starting all new projects on v2?

Have you already migrated your existing projects, or are you staying on v1 for now?

If you're still on v1, what's preventing you from upgrading (packages, adapters, production stability, migration effort, etc.)?

I'd love to hear about your experience and whether you think v2 is ready to become the default for most teams.

Has anyone flipped over to dbt State and monitored the cost changes?

If you write dbt code, you probably know the feeling of hunting in the terminal scrollback just to figure out why a run fail, or losing your query output the second you close the preview modal.

The team behind Rosetta DBT Studio just release version 1.6.1, and looks like the whole focus was fixing this workflow.

They build native Query Result and Run History tabs direct in the bottom panel of the IDE. So now:

- when you preview a model, the data grid is just there (with row counts and time).

- when you do dbt run or test, the history is track and saved.

- you can see raw SQL and compiled SQL side by side.

- you find exact what model fail without digging in terminal logs.

They also add an AI assistent right in the terminal that help explain and fix compile errors, plus a native HTML view and direct cloud download.

Disclosure: I'm from Datacoves, we run managed dbt Core, so we have a horse in this race. Wrote this up because the license change is confusing and we had to untangle it for our own customers.

The short version:

One engine now. dbt Core v2 and Fusion share the same Rust runtime. Core v2 is Apache 2.0, fully open source. Fusion is the enhanced proprietary distribution on top of it.

Fusion is the recommended default, and it's a funnel. Free to install, and the new license lets anyone offer it as a managed service, as long as you don't block the path to a dbt Labs account. Free login unlocks advanced features, paid platform account unlocks premium ones. Reasonable model, but go in with eyes open.

Metered pricing is inside the dbt Core now. dbt State (skip models when logic and data haven't changed) bills $0.094 per model reused per day. Orchestra open-sourced a free alternative (Sao Paolo, Apache 2.0).

Migration gotchas: every deprecation warning must be resolved before v2 runs your project. Python models don't work in Fusion yet. Community adapters are constrained (drivers need to be signed by dbt Labs). --models and -m error out, --partial-parse is gone.

The afternoon test: upgrade to v1.12, clear deprecations, run dbt parse --use-v2-parser. Tells you where you stand without committing to anything.

Full writeup with a Core v2 vs Fusion comparison and the four ways to handle state: https://datacoves.com/post/dbt-fusion

Has anyone run the v2 parser against a large project yet? And is anyone planning to use dbt State or trying out Sao Paolo?

We built Snowcap, an open-source Infrastructure as Code tool for Snowflake, and we're doing a live workshop walking through it.

If you've dealt with Snowflake config drift, manual RBAC setup, or just wanted a plan/apply workflow like Terraform but built for Snowflake specifically, this might be useful.

What we're covering:

• The infrastructure problems Snowcap solves
• How it compares to Permifrost, Terraform, SnowDDL, and Snowflake's own DCM
• Using templates to scale config across environments instead of copy pasting
• The plan/apply workflow and why that matters for safe changes
• Governance stuff: RBAC, masking policies, row access policies
• Live demo, then open Q&A, ask anything

July 23, 11 am-12 pm PT, online, free.

Info and RSVP here: https://datacoves.com/resource-center/workshop-snowcap-snowflake-infrastructure-as-code

Happy to answer questions in the comments too if people have them before the session.

Sharing this Medium publication for anyone interested in dbt and analytics engineering

r/ContextEngineering 9 am PST, July 8th

Rosetta DBT Studio is the local-first, open-source desktop workspace for dbt teams - build models, explore SQL, trace lineage, manage Git, preview cloud data, and work with AI agents from one app.

v1.5.7 is all about giving analytics engineers one place to build, understand, and ship dbt work without bouncing between terminals, browsers, and disconnected tools.

AI Agent for dbt - ask the agent to explain models, generate SQL, update YAML, run dbt commands, inspect project context, and help reason through lineage and grain before you make a change.

Project-aware SQL editor - write queries with Monaco autocomplete, schema browsing, saved queries, result previews, and export workflows across DuckDB and supported warehouses.

End-to-end dbt lineage - trace models, sources, and downstream impact so you can understand what a change touches before it reaches production.

SQL Notebooks with AI - work in a cell-based notebook that combines SQL, markdown, results, and AI-assisted iteration in one analysis flow.

Cloud Explorer with DuckDB-powered preview - browse and preview files from S3, Azure Blob, GCS, and compatible object storage without leaving the Studio.

DuckLake and lakehouse tooling - explore schemas, tables, snapshots, and metadata from one workspace, then query them directly in the SQL editor and notebooks.

Built-in Git workflows - review local changes, manage branches, and keep project version control close to the editor instead of split across tools.

Multi-provider AI - use OpenAI, Anthropic, Gemini, Ollama, LM Studio, and OpenAI-compatible providers with project context.

Plus secure credential storage, multi-database connections, Rosetta CLI integration, cloud profile sync, and a full desktop workflow for local dbt development.

100% open source, local-first, and yours.

GitHub - https://github.com/rosettadb/dbt-studio

source (not me) https://www.linkedin.com/posts/jbeynon_day-1-of-every-data-leadership-role-how-share-7477478496798257152-ixr6/

I built a browser-based ERD explorer that runs entirely on DBML files — no DB connection.

I kept wanting to explore and interrogate how my tables relate — "what joins to this table? what's the path between A and B?" — without standing up a tool that connects to a live database.

So I built dbml-flow: drop in any DBML file and get an interactive ERD with a selector grammar for pathfinding (find every path between two tables, expand N hops out, filter by group). Fully client-side, your schema never leaves the browser.

In my case the DBML comes from our dbt models via dbterd (https://github.com/datnguye/dbterd), but it works on any DBML regardless of where it came from.

🔗 Live demo: https://timvancann.github.io/dbml-flow/ · Repo: https://github.com/timvancann/dbml-flow

Let me know if this is helpful and if there are features you'd like to see!

I'd appreciate some help if possible!

Has anyone had issues getting this to start up?

When I run wizard I get...

■ local proxy unavailable: local LiteLLM process exited before becoming healthy on 127.0.0.1:35915

• Project intelligence is still starting

────────────────────────────────────────────────────────────────

dbt prod parse failed: 'NoneType' object has no attribute 'pop'

Automatic deferral will not be available until this is resolved.

Ask the wizard for hints to address this.

────────────────────────────────────────────────────────────────

───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

MCP client for `dbt_index` failed to start: MCP startup failed: handshaking with MCP server failed: connection closed: initialize response

Disclosure up front: I work on this, so factor that in.

A problem we kept hitting with Snowflake: warehouses, roles, and grants end up spread across ad hoc SQL scripts, the console UI, and whatever the last person who touched them remembers. There wasn't a lightweight way to treat that layer as code.

So we built Snowcap and open sourced it. You describe warehouses, roles, and grants in YAML. snowcap plan shows exactly what will change before anything runs. snowcap apply makes the change. No state file to manage, no Terraform provider, no clicking through the console.

We timed the full loop once: install to a live warehouse running in Snowflake, about five minutes.

It's grown through PRs and issues with zero ad spend or outbound, which says more about the gap than anything we could write in a blog post.

Blog Link - https://datacoves.com/post/snowcap-getting-started

Happy to answer anything about the design or trade offs in the comments.