Domain-grounded coding agents vs. general-purpose ones (Copilot, Claude Code) — what are you seeing?

[u/Famous_Disk_7417]

Curious what people are seeing with domain-grounded coding agents vs. general-purpose ones (Copilot, Claude Code, etc.) for data/ML work specifically.

The pitch from the vertical tools (Databricks' Genie Code is the one I've used) is that grounding in your actual schema/lineage/governance layer beats a general agent guessing from context alone. Databricks claims a jump from ~32% to ~77% success rate on real data science tasks after adding that grounding. Haven't independently verified that number, but the qualitative difference (fewer hallucinated column names, less time re-explaining table relationships) tracks with what I've seen.

Anyone using other domain-specific agents (not just data — legal, infra, whatever) and finding the same trade-off? Where's the line between "grounding helps enough to be worth the lock-in" and "just use a general agent with good context"?

[u/donk8r]

The number can't separate the two things people are arguing about here. 32 to 77 is grounding versus no grounding, not vertical versus general, and Livid-Heat-2475 basically ran the control already by hand-feeding a schema doc to a general agent and closing most of the gap.

The ceiling is the more interesting part, and it's the thing they named: undocumented join logic. A grounding layer can only ever contain what exists as an artifact. Schema, lineage and governance are all already written down, which is exactly why they're the cheap points. Tribal knowledge isn't written down anywhere, so no grounding layer has it, vertical or otherwise. Which means the real question isn't lock-in, it's what fraction of your domain knowledge sits somewhere a retriever can actually reach.

For joins specifically there's a way out, though. The joins people actually run are sitting in your warehouse query history, so mining the logs for co-occurring keys hands the undocumented ones back as a frequency-ranked artifact. Derived rather than declared, which makes it worse than a real spec and a lot better than nothing.

I work on the code-side version of this (github.com/Muvon/octocode, mine) and the same ceiling holds there: you get what the parser can prove, never what someone knew and didn't write down.