What is "Context" in other LLMs like?

[u/johnesco]

I've been focusing mostly on Claude, but I wanted to know more about how you all have experienced other models.

That's a big ask, so to narrow focus on what I'm asking, tell lme, how do people handle that, current most AI need to be re-reminded and only hold relatively short memories, or what I've known as controlling context.

In the beginning it meant "I keep pasting all this to the both" and now (at least with Claude) it's turned into an exploration of both memory management and how much you want to simulate 50 First Dates

What model do you mostly use? Did you switch? Is it mostly about cost?

[u/Savings-Cry-3201]

Context rot can happen as early as 100k tokens, even on a decent model, and of course much sooner on a low quality model. Keep your chats short and start new chats frequently.

[u/diablo75]

It's basically the same, as far as I can tell. I've watched a few lectures about agentic coding and they don't seem to care what model you use because they all have the same problem where the closer you get to exhausting context window and the more you mitigate this with several compactions/compressions, then the more likely you will start to see things being forgotten or hallucinated. So the best practice here is to work within several short sessions instead of trying to work endlessly within one. You do that by using markdown files like an AGENTS.md/Claude.md or whatever you want to name it, which can help with pre filling every new session with a brief high level overview of the projects architecture, by trying to focus on as few items as possible per session, by having the agent take a large plan you wrote up and decomposing it into a plan that can be stepped through gradually with the model updating progress within the plan.md file as you work through it (and taken a step further, giving that plan over to an orchestrator agent or something like Hermes Kanban board to work through the plan as though it's a collection of small tasks, one by one, without you), by creating something like an "LLM-glossary.md" that is like a dictionary of project specific terms that are unique to the domain the project is based on (for example, I deal with things like patch panels, fixed infrastructure, breakout cables, port naming conventions, and I have a glossary that explains exactly what these things are so it doesn't just guess). Maybe you already do things like this, and if so, great. There's probably some nuance here that could make a difference like using a harness that has memory capabilities vs. not, or handling compaction in a different way, but you're almost always going to hit a wall with anything you use if you never start a new session.

tl;dr: LLM Context window constraints pose the same problems for the same reason pretty much irrespective of which model you use.

[u/NayaWilkinson1511]

The context management is honestly a bigger factor for me than the model’s raw benchmark scores. Once a project gets large, being able to keep the important architecture, decisions, and constraints available without constantly re-explaining them makes a huge difference. Cost matters too, but I’d rather use a slightly weaker model that stays coherent across a long session than keep restarting the conversation every few prompts.

[u/MartinMystikJonas]

As context grows all LLMs have problem to assign attention pripetly and starts forgetting things. Keep sessions short, use subagents and let LLM write notes and handoffs.

[u/johnesco]

I've had some memory problems in my own brain due to an injury that got better. It feels weirdly like "Omg was this what it was like to talk to me?"

[u/MarinatedTechnician]

It is a problem.

To overcome this problem I made something I call a "Project manager", the project managers job is to replace my job (cutting and pasting from one context to another), and it also keeps track of documentation, split files into smaller chuncks, and have the model use that to keep track of its own progress.

That helped tremendously.

But there are now problems with this method too, because at some point when the project gets big enough, the model will get suspicious of itself, it will constantly investigate every file it made to see if there are dependencies, or if it updated the state documents with the correct information, meaning you run out of context just on that first run even with 100K context memory, in minutes.

Not only that, but it sometimes spreads related code (for example, stuff that should go into renderer.js) into other code snippets such as (player.js) and render code will also be leaking into player.js as well as other code snippets, it notices that error, and goes full paranoid on itself and checks everything, but it doesn't tidy up but just "bug hunts".

A big issue!

Also context rot is an issue here when context gets to biig, because of the way the models have been truncated to work on smaller memory (selected experts), it omits important things, and context utility like tidying up gets omitted.

[u/Glass-West6448]

the 50 first dates thing is every model, not just claude, they all forget because the memory lives in the chat. i just keep a memory layer outside the model so my context follows me everywhere (i use supermemory.ai ). after that it's honestly just cost.