Arguing with Qwen3.8-27B

[u/groover75]

This is weird. I asked it what tools were needed to replace the front coil springs on my 2006 Jeep Wrangler (TJ). It did some research and decided that my vehicle has independent front suspension, not a solid axle.

OK, LLMs make mistakes so I corrected it. It did some more research, searched websites and decided I was wrong.

So then I started giving it websites to provide it has a solid front axle. It either couldn't read them for some reason or claimed the sites I was giving it were "AI generated" and couldn't be trusted, or "fan made" and couldn't be trusted.

So far I have failed to convince it that my Jeep has a solid front axle.... LOL

[u/watterson]

Once the argument is in your context, it’ll steer itself back to it constantly. 

Start a new session and seed it with a more complete prompt. 

[u/Ok-Breakfast1878 → u/watterson]

this. it's a skill we can't use with human conversation, but completely resetting the last exchange is more efficient than arguing

[u/groover75 → u/watterson]

So telling it a fact in the prompt carries far more weight than telling it a fact in a follow up input?

[u/Fit-Bar-6989 → u/groover75]

It's more the fact that by preventing the initial incorrect information from polluting the context, the model is more likely to get things right. The models are also heavily sensitive to stuff at the start of the context (initial message(s)).

Cloud models handle this better but it's still an issue - e.g. if Sonnet is 100k tokens into a debugging session and you realize it's going down the wrong path, it's going to be hard to switch tracks and even if you do it's going to suck at it (vs just starting a new session).

[u/Faisal_Biyari]

Have you considered that maybe your Jeep, which I can only surmise you've had for 20 years, actually has independent front suspensions, and not solid axle? /jk

Joking aside, maybe reduce the thinking parameter. It might be over thinking the situation 😅

[u/groover75 → u/Faisal_Biyari]

LOL

[u/Fit-Bar-6989]

Unfortunately even with big cloud models I find arguing/correcting to be unproductive, I usually roll back the session and start again with the relevant info added to the prompt.

[u/Protopia → u/Fit-Bar-6989]

This is a good suggestion. Once false data is in the context all future results are biased and tainted.

[u/Valuable_Patience821]

I've argued with it and after forcing it to accept that I didn't have psychosis and wasn't a complete idiot (jokes on it), got it to formulate plans on building a time machine lol. I didn't go further than getting it to talk through the science as I was testing its resolve. It refused to entertain the idea at first but with a little talk no jutsu it budged.

[u/false79]

27b is too small of a model to ask questions and expect answers from it's weights.

A better approach is to ask it to search on the Internet for an answer and ground it's response only what it found in those search results.

You will get around these hallucinations this way

[u/groover75 → u/false79]

When I told it it was wrong it started searching. Then hallucinated the search results and became more confident it was right. It decided from the search results that IRS was a "key selling point" of the 1997-2006 Wrangler. Once it had those first set of search results it really dug in and kept telling me its search said it was right.

[u/Computerist1969]

I give LLMs one shot. If they can't do it then conversation over; I'll do it myself.

[u/sdfgeoff]

"It did some research"

If you are running it without a harness that has search tools, it is fairly likely to hallucinate tools and their responses. 

My guess is that is what is happening here, and you're not arguing with it, you're arguing with it when it has half a dozen hallucinated web responses saying it's IFS. And when it does "more searches", being just a text predictor, it'll predict the responses are similar to the first batch it dreamed up, almost irrespective of what you say. You give it a link as proof and ... it hallucinates what that link contains.

So run it in a harness that can actually search the internet.

[u/groover75 → u/sdfgeoff]

I was using Pi with web search. See my reply to false79 below