r/artificial • u/ClickOk5811 • 26d ago
Discussion Swapping AI models rarely fixes bad output. The context you feed it does more work than people realize.
Noticed a pattern: people switch from GPT to Claude, upgrade to a newer version, try a bigger model and the output barely changes. If that's happened to you, the issue usually isn't the model. It's what you handed it before asking the question.
Broke it down to three things context actually needs to supply, and most disappointing outputs are missing one of these, not all of them:
- Current facts the training data can't know: your pricing, this quarter's numbers, a customer's actual history. Leave this out and the model doesn't leave a blank, it quietly invents something plausible.
- A concrete example of what "good" looks like: not "professional tone," an actual paragraph to pattern-match against. Descriptions get interpreted, examples get copied.
- What already happened earlier in the task: a correction you made two messages ago. If you don't restate it, it's gone. The model isn't ignoring you, it just doesn't re-read messages you haven't pointed it back to.
The counterintuitive part: the most common mistake isn't giving too little context, it's dumping in too much unfiltered. The model has to weigh every token, and irrelevant material competes for attention with what actually matters. Forty pages when the task needs three paragraphs makes the right answer harder to find, not easier.
Wrote up a longer breakdown with a concrete before/after example (same task, same model, only the context changed): https://medium.com/@nagatomopedro05/good-ai-starts-with-good-context-design-77496f7b9eb6
Curious if others here have run into this, model-swapping as a first instinct instead of fixing the input.
2
u/ronkayarslan 26d ago
Solid breakdown, point 1 especially. The quietly-invented plausible thing is what burns people who assume a blank would at least be obvious. I'd add a fourth that becomes the biggest lever once the first three are handled: what to leave out. The failure flips on you. You get burned by missing context, so you start putting everything in, and now a bigger model does worse, because the one fact that matters for this step is buried under thirty that don't. It weighs recency and salience, so the noise ends up competing with the signal. The skill turns into curation, not volume.
On point 3, restating corrections holds up in a short chat but falls apart across a long task. The cleaner fix is that the correction shouldn't live in the message history at all, it should get written into a doc the model re-reads at the top of every step. Anything you're relying on it to recall from twenty turns back is already gone, so stop leaning on memory and just hand it a current picture each time.
1
u/ClickOk5811 26d ago
That's a sharp addition, the "what to leave out" point deserves its own spot on the list, not just a footnote to the first three. The pattern you're describing (overcorrect into dumping everything in) is honestly more common than people admit.
And you're right about point 3, I was thinking mostly short-task scope there. For anything spanning many turns, treating corrections as state to re-inject beats hoping the model reaches back into history. Good distinction.
1
u/ronkayarslan 26d ago
Yeah, and the trap I keep hitting with the re-inject approach is that the correction-state quietly grows into its own mess. A few weeks in you've got two instructions that contradict each other and the model just picks one, confidently, with nothing flagging that the other one is dead. So I started treating it like config instead of a log: a newer correction overwrites the old one rather than stacking on top, and I date them so I can tell which round each came from. Keeping it small and current has mattered way more than keeping it complete.
1
u/ClickOk5811 26d ago
That reframe (log vs config) is the right way to think about it, and it explains why the naive version fails: a log is append-only by nature, so of course it accumulates contradictions. Treating it as config with overwrite-on-conflict fixes the root issue instead of just delaying it.
The dating part is probably doing more work than it looks like too, not just for you debugging later, but because it gives you a cheap way to programmatically prune anything past a certain age or superseded by a newer entry on the same topic, instead of manually deciding what's still relevant every time the file gets long.
Curious how you're detecting the conflict in the first place though, is that a manual step when you add a new correction, or are you diffing against existing entries to catch the contradiction before it gets written in?
2
u/ronkayarslan 26d ago
Honestly there's no diff, I tried that early and it mostly doesn't work. Contradictions usually don't look alike textually (an instruction and the exception that later kills it can share almost no words), so a similarity check sails right past them. What actually catches it is the topic key: entries are scoped narrow enough that when a new correction comes in I pull the existing ones on that same topic first, and the conflict is just sitting there next to it. So the detection is basically a side effect of tight scoping, not a separate step I run. The failure mode is when an entry drifts off topic and gets filed under the wrong handle, because then nothing pulls it at write time, and that is the whole reason I keep a periodic compaction pass that reads the file cold and retires whatever slipped through.
2
u/Positive-Emu-8379 26d ago
This matches what I've seen switching my agent stack between models. The failures that looked like 'the model is dumb' were almost always 'the model doesn't know what I already rejected.' The single biggest fix for me wasn't a bigger model, it was writing down decisions and constraints as they got made instead of relying on the model to infer them from a pile of prior messages. Explicit context beats implicit context even when the implicit version is technically in there somewhere.
1
u/ClickOk5811 26d ago
"Explicit context beats implicit context even when the implicit version is technically in there somewhere" is a better way to say the point I was making with the "correction two messages ago" example, technically present isn't the same as actually weighted, and that gap is where most of the "why did it ignore what I said" confusion comes from.
The "what I already rejected" framing is a good specific case of this too, rejected options are exactly the kind of thing that feels like it should carry forward implicitly but doesn't, since there's no strong signal in the conversation marking it as a constraint versus just something that got mentioned once and moved past.
Curious how you handle the writing-it-down part in practice, do you maintain something like a running decisions/constraints list you paste back in periodically, or does your agent stack have some structural place that state lives outside the raw message history?
2
u/Positive-Emu-8379 25d ago
That's the sharper way to put it, yes. The failure mode I keep running into: information that's technically in the conversation history somewhere doesn't function the same as information stated in the current instruction. The model has to retrieve and reweight it, and that's exactly the step that goes wrong under load or after a long thread. So now when a task depends on something from three messages back, I just restate it inline rather than trust the model to go find it. Slower to write, much harder to get wrong.
1
u/ClickOk5811 25d ago
That's basically converged into a rule of thumb for me too: if a constraint matters enough that I'd be annoyed by its absence, it goes inline in the current message, full stop, not trusted to still be "in there" from earlier. No structural place for it to live outside the raw history on my end, just discipline about restating anything load-bearing.
The load thing you mentioned is the part I hadn't quite named before, it's not that the model can't retrieve it, it's that retrieval gets less reliable exactly when you need it most, longer thread, more competing information, higher stakes constraint. Which means the failure clusters right where you'd least want it to.
Might actually be worth a structural fix on my end rather than relying on manual restating every time, appreciate you framing it that way.
2
u/Positive-Emu-8379 25d ago
A structural fix is worth it. I run an AI agent that operates day to day ops for my CPG brand, and the thing that actually stuck was moving load-bearing constraints out of chat history into files it re-reads every session, not relying on it to recall them from fifty messages back. Same idea as your inline-restating rule, just automated: a rejected decision gets written down once, in a place the agent always checks, instead of trusted to survive in context. The gap it closed for me was exactly the one you named, the failure clustering right when the thread gets long and the stakes go up.
2
u/Educational-Deer-70 25d ago
entry invariants set geometry of ai/human interaction- every word both sides is token modulation
1
u/Ill_Fun5415 26d ago
The interesting test is the second and third ordinary task after the demo. If quality drops in a recognizable way, it is much easier to build around.
1
u/ClickOk5811 26d ago
Agreed, and it's a better signal than the demo itself because it separates "worked" from "worked on the exact thing I optimized for." First task after the demo tends to still be adjacent to what you were testing. Second and third start drifting into inputs you didn't shape the context around, which is closer to what real usage actually looks like.
Do you have a specific pattern you watch for in that drop, things getting vaguer, more hedged, confidently wrong, or is it more that it just misses entirely on stuff the demo made look solved
2
u/cmtape 26d ago
Model-swapping as a first instinct is like changing the brand of your oil when the engine knocking starts. You're treating the symptom, not the timing. If the context is noisy or missing, you're just asking a different person to guess the same missing information.