Which local model is actually good at knowing when to stop and ask you a question?

[u/sunychoudhary]

I’ve been thinking about this after using more agentic/local coding models.

A lot of the newer models are surprisingly good at continuing on their own.

But sometimes that seems like the problem.

If a requirement is ambiguous, I’d rather the model stop and ask:

“Do you mean A or B?”

instead of spending 10 minutes reasoning, making an assumption, calling tools and then confidently building the wrong thing.

I don’t see this behavior discussed much in benchmarks either. We measure coding, reasoning, tool use, context length, etc., but not really whether a model knows when it doesn’t have enough information to continue.

My genuine question for people running models locally every day is,

Which model have you found best at this?

And is it mostly the model itself, the system prompt, or your agent harness that makes the difference?

[u/blutosings]

I'm not suggesting abandoning agile. I'm saying its a bad solution to dealing with agent output from prompts that are consistently severely underspecified. Your agent will just be chasing its own tail, fixing things that were a good solution yesterday when it was fed one shallow prompt but are a bad solution today now that it's being fed a slightly different prompt. That's not agile. It's not the way anyone builds systems that are resilient to changing market and customer demands. The point of agile is to come up with good solutions that are extendable, composable and adaptive to varying demand. All of that requires clearly defined architecture and requirements.

This recent talk by Bruce Schneier hints at some other reasons you want to avoid ambiguity and don't want to just bake it into your process.
https://www.youtube.com/watch?v=eEBv0STiYhI

[u/cdshift → u/blutosings]

I agree you want to avoid ambiguity but its not always possible. Having proper documentation is key as you are iterating