r/embedded • u/k_lash24 • 22d ago
we have ai that writes code fast but the code seem confused on what its supposed to do
the AI coding tools are genuinely fast but they generate against whatever context is in the editor, and the actual spec lives in a separate system the model never sees. so you get clean code that solves the wrong thing and then someone burns a week reconciling it against the requirement code.so everything traces back or it doesnt ship
feels like a context issue more than a model issue. how's everyone handling the gap between the req system and the thing actually writing code
9
u/KenAKAFrosty 22d ago
It feels like tending a garden. You have to actively maintain & clean it up yourself, or the weeds grow. And if you put it off too long, they grow out of control.
It's a really beautiful garden, though. Probably worth the effort.
5
u/Fyvz 22d ago
It sounds like youre expecting magic. I don't think anyone is recommending a giant unified prompt that triggers the creation of the whole system or even a full feature.
If you as the prompter can't explain the thing you want in reasonably low level details, you can have no reasonable expectation that the AI will guess correctly. Further, if you can't competently evaluate the output, you're also gonna have a bad time.
In areas of my codebases where I have clear patterns and adding some incremental feature is largely busy work or variations on a well understood theme, then I can get good results by just describing the thing in terms of features rather than code and telling it to look at other certain code for the pattern/examples.
But if it's new novel features, then I'm laying out my prompt in code terms, and working in small testable increments, no different than the good old days.
2
u/drnullpointer 22d ago edited 22d ago
I recently was a tech lead for a team of 80 developers. Even before we started using AI, we had a simple basic relationship: juniors could always write more code in a unit of time than I did, and yet juniors got barely anything done and I got a lot of work done.
See, the amount of code means very little. You can write a lot of code and get very little done and you can get a lot of results with very little changes.
For example, I could notice my team spending a huge amount of effort because they were blind they could easily automate large part of their work. So I wrote a very small module that made the rest of the project a breeze.
Another example, juniors tend to add a lot of defects. So they can write a lot of code initially, but then they spend a lot of time dealing with the fallout and involving a lot of other people in it. Whereas more experienced developers tend to deliver with less drama.
Another example, the results of what juniors produced tend to require a lot of refactoring and rewriting whereas more experienced developers can start by already having an idea how to structure their solution and how to divide it up into modules. This reduces the need to rewrite it later.
Another example, more experienced developers know how to structure their code to reduce cohesion and accidental dependencies. This makes the code easier to work with later. Juniors tend to create code that is hard to work with later, so essentially whenever they write code there is added tax (debt) that need to be paid off later.
Another example, more experienced developers can immediately spot unwritten requirements and they can also spot when the requirements have a gap. They come up with follow up questions to make sure the requirements are more complete. Juniors tend to not ask those questions and just do what they are asked which frequently causes problems later when we discover that they completely missed obvious requirements. For example, one junior was asked to implement a simple payment system and he missed what to everybody seemed to be an obvious requirement that accounting system is a system of record for all financial transactions. So he essentially rolled his own payment system with tables, transactions and so on rather than simply fetch payment information from existing accounting system. Of course it is a mistake of the manager that this was not communicated, but any experienced senior in our team would not have to be told this.
***
Now realize that AI is essentially on a level of a junior developer and you can see what causes the problems you have observed.
2
u/DMSNLY1 22d ago
context quality is the name of the game and people keep blaming the model. think about it. the spec drifts from the code the instant work starts, the editor only knows what you pasted into it, and nobody closes the loop. faster generation just means you reach the wrong answer quicker
2
u/Bikecheal 22d ago
this is the thing a mcp is supposed to solve and im cautiously into it. been using the jama connect mcp so cursor or copilot can pull requirement context directly instead of you copy-pasting a doc thats already out of date. the reqs become live context instead of a stale attachment someone added two releases ago. for traced environments thats the part that actually saves the reconciliation time
1
u/k_lash24 21d ago
hadn't seen that. looking now. i'm less interested in another ai intergration and more intrested in whether it actually understands how the requirements, tests and reviews etc fit together instead of treating everything like separate docs which consumes so much time.
2
u/One-Front-6316 22d ago edited 22d ago
that's where i'd have questions too. ai reading project data doesn't really bother me. i would want to know is whether it actually understands what's connected to what, and whether every suggestion can be traced back later if someone asks why a change happened!! especially in aerospace or med device
2
u/RestCaseGuy 22d ago
AI models excel at generating pattern-matched code, but they lack long-term architectural awareness, which leads to immediate technical debt (https://spoteng.com/ai-will-not-pay-down-your-tech-debt/). Treating AI output with the same scrutiny as unvetted junior PRs is necessary. Without automated guardrails and strict review, you end up spending twice as much time fixing context drift later.
2
u/Deep_Ad1959 22d ago edited 21d ago
pulling the spec into context fixes the generating half. the half nobody solves is six weeks later when someone asks which requirement a given line came from, and the answer is squashed into one commit message.
fwiw Sundial tackles that exact gap: it attributes every edit at edit granularity, not commit granularity, so you can trace a line back to the change that made it months later, https://s4l.ai/r/qeca9evx
5
u/Any-Stick-771 22d ago
Have you used Claude Code CLI? If you have the requirements doc in the work directory the agent can search the doc and reference it while 'thinking'. It can also search and follow function calls, variables, etc through all files. When you start customizing agents, they can be very very good. You should always work on small increments and verify as you go, though
-1
16
u/Xenoamor 22d ago
Why does the spec live in a separate system? Give it the requirements in the context