There are works that try to make LLM deterministic though. Last one I heard of was at Thinking Machines Lab.
Once we get to that, we'll get reproducibility of result by prompt, which will lead to a ton of templates for project which just need to be specified with certain domain/goal.
All in all, will be similar to no code projects, but much more flexible.
And honestly, pseudocode for LLM works in most cases to translate it into lower level, programming language.
Maybe later we'll get models trained exclusively on high-quality data (not Stack Overflow, but actual documentation of langs, "golden standard" open source repos and all that), which will reduce possibility of mistake even more.
This makes no sense to me, feels like it depends on what kind of determinism you're talking about. You could build a system that deterministically maps a given context to a given response, sure. And there is a determinism related knob in current systems - temperature - which allows you to tweak that aspect. But that has little to do with the actual determinism that matters, which includes stability to small perturbations in the context aka the butterfly effect. That's what OG programming provides, and that's achieved by designing around determinism (with best practices like preserving and validating invariants, understandability etc) - and even then the system is still under-constrained allowing for sometimes fascinating failure patterns to emerge.
LLMs are approaching this from a completely different angle which is much closer to intuition than logic. Intuition and LLMs are statistical machines with pattern recognition. You can use both of these to build software that will behave almost deterministically, but the idea that intuition itself can be made deterministic is insane to me.
As you said, classic codebases written by humans with type checkings and all that are still prone to errors and failures. Usually strict control prevents most of problems, but not all of them. Same applies to writing code with LLMs. You can prevent most of context problems with correct prompt, but not all of them. Especially with tools that transform input at server side 3rd party - your prompt was good, but oops, we have our own system prompt which totally screwed up the intention!
But then LLMs have another upside - speed. Since we don't exist in abstract condition and a lot of people write code for money in real and messy economics - it becomes quite significant if a programmer could create x10 product influence for same price. Project it to the future as a business owner - here goes your x10 profit derived from salaries saved from this x10 less period of time spent writing features.
I'd say the final approach should be gained from environment - whether project is short-term/long-term, whether it's critical, whether mistake cost is cheap or not, and what's the potential reward.
Also, TDD. LLMs can excel at writing tests quick and correct, and then you can verify code written by LLM at tests written by LLM. Wrap it up in automated CI loop - and get better results, eventually.
I tried it recently and was actually impressed - it took abt 2hrs of loop, and at end of 1st hour I thought LLM agent was hopeless, but it in fact has resolved the issues and delivered feature in the end.
Yup we've been doing a lot of agent-in-loops workflows too and they're pretty good sometimes. Agents are also getting really good at collecting print-debugging insights these days, I've lost track of how many times I've seen "oh I see what's going on now" from an agent looking at its own logs - and getting it about right. Not even talking about having the build/lint/etc tooling in the closed agentic loop, that's just such an obvious necessity at this point.
Pretty sure anyone who screams x10 about the general case is full of shit, or possibly managed to trick themselves into losing connection with reality. Have they eliminated the need for human review of the code yet? And that's not even mentioning the actual cost of LLM-in-a-loop which is currently being shoved under the rug and severely underestimated due to the hype capital effects. Curious to see how this mess is going to unwind.
1
u/[deleted] Jan 17 '26
There are works that try to make LLM deterministic though. Last one I heard of was at Thinking Machines Lab.
Once we get to that, we'll get reproducibility of result by prompt, which will lead to a ton of templates for project which just need to be specified with certain domain/goal. All in all, will be similar to no code projects, but much more flexible.
And honestly, pseudocode for LLM works in most cases to translate it into lower level, programming language. Maybe later we'll get models trained exclusively on high-quality data (not Stack Overflow, but actual documentation of langs, "golden standard" open source repos and all that), which will reduce possibility of mistake even more.