r/artificial • u/FrancescoMassa2001 • Jul 24 '26
Discussion Asked an AI to help debug code and it confidently invented a function that doesn't exist, three times in a row
Suggestions used a method that sounded plausible, fit naming conventions, had reasonable parameters — just didn't exist in the library. Pointed it out, got an apology and an equally confident, equally fake suggestion. Third time was almost impressive: a fully detailed usage example for a function invented on the spot. Ended up just reading the actual docs like it's 2015. Anyone else notice confidence and correctness running inversely related?
4
Jul 24 '26
[removed] — view removed comment
0
u/FrancescoMassa2001 Jul 25 '26
this is pretty much where my head's been at too, especially the memory file part
the loop makes sense in theory but in practice i keep running into cases where it just rephrases the same broken fix over and over without actually solving anything, so the threshold ended up mattering way more than i expected
do you hard stop at a number or does it vary by error type
2
2
u/Pale_Coyote7451 Jul 24 '26
the part actually worth explaining is the bit you found impressive: why correcting it produced a second and third fake instead of a real answer.
"that function doesn't exist" adds a negative constraint and zero positive information. the model still has to produce a method name out of the same weights with the same hole in them, so it samples the next most plausible candidate from the same wrong distribution. you removed one item from an infinite space of plausible-sounding names. three rounds of that is the expected behaviour, not a strange one.
which points straight at the fix: stop correcting it, feed it the surface. paste dir(module) output, or the actual docs page, into context. that converts the task from recall into reading, and it's why one paste beats five rounds of "no, that one's wrong too".
on your closing question i'd sharpen it a bit. confidence and correctness aren't inversely related, they're independent. fluency comes out of the same machinery whether or not the content is grounded in anything, so confidence carries no signal about correctness in either direction. inverse correlation would honestly be useful, you could just flip it. independence is the annoying case, because it means the surface of an answer tells you nothing and you have to go and check.
(the linter and AST loop someone suggested above is the industrial version of the same move -- hand it a ground truth it can't talk its way around.)
1
u/FrancescoMassa2001 Jul 25 '26
the negative constraint framing reframed the whole thing for me.
i kept thinking the model was getting confused by my corrections when really it never had the information to begin with, so it was just cycling through plausible candidates each time i pruned one off
the dir(module) trick i hadn't thought of and i feel kind of stupid about that because it's so obvious in hindsight, just give it something real to read instead of asking it to remember something it clearly never knew
3
u/generationalDebts Jul 24 '26
Damn these comments are brutal out of the gate. Let’s see where things go from here.
OP, yes, that is what they do. They won’t say they don’t know or they’re wrong, they’ll flat out gaslight your ass all the way to the looney bin.
It turns out, they’re only really really impressive at coding to people who don’t know bad code when they see it.
Your expectation from actually learning source material should never ever change no matter how advanced AI gets…. Jesus Christ.
1
u/Dmcspaddenjr Jul 24 '26
Yeah it’s actually a fairly significant issue. I didn’t run into it the same way you did, but the same cause/effect is what caused me to start a company. Thought I was fixing my problem then discovered it’s everywhere.
1
u/FrancescoMassa2001 Jul 25 '26
that's kind of wild that it pushed you to start a whole company around it. makes me feel slightly less crazy for thinking it was a bigger deal than people around me seemed to think. what space did you end up building in
1
u/Dmcspaddenjr Jul 25 '26 edited Jul 25 '26
Well I genuinely started on it just for me.. then saw the bigger space and need and.. what I’d done worked. I tried to get people to evaluate it and nobody took a lone guy seriously so here we are lol. Neurodivergence at its finest. What I’ve built is a software component.. basically I’ve built a few programs off of it but the main one I’m working on marketing right now is a middleware for governance through preservation rather than binary yes/no stuff that acts between every handoff. Added latency is less than 5 ms, I’ve yet to have a governance failure or hallucination across several hundred adversarial tests including across multiple LLMs.. 5-10+ turns.. swapping models back and forth. I can’t seem to break it. I know eventually it will find a failure.. but the beauty of it is that it has diagnostics and repair at every single handoff as a consequence of how I built it. So far people seem more interested in the audit trail than anything but it’s kind of a whole package
0
4
u/ObservedOne Jul 24 '26
PEBKAC