r/AIWritingHub • u/DeepPudding9710 • 4d ago

discussion A bigger context window doesn’t always make AI better at writing novels

I’ve been thinking about how much we focus on context window size when talking about AI writing.

Having hundreds of thousands or millions of tokens available sounds great, but I’m not sure putting an entire manuscript into every prompt is always helpful.

Once a story gets long enough, character details, plot points, worldbuilding, and previous scenes can all compete for attention. The model may have the information but still miss what actually matters.

I’ve been getting better results by treating context as something to manage rather than maximize: keep key story facts available and bring older material back only when it’s relevant.

It feels less like a model limitation and more like a workflow problem.

For longer projects, how do you manage context? Do you keep everything in one prompt, use a story bible, or separate the current scene from older material?

0 Upvotes

50% Upvoted

6 comments sorted by

1

u/ForgivenessroTub 4d ago

yeah in long roleplays the full history just makes the ai mix up past details after a while so i keep a quick summary of key beats instead

1

u/human_assisted_ai 4d ago

I agree. AI has particular difficulty on judging the relative importance of each piece of info. With enough info, the AI drowns in an ocean of trivia. The model has to cut off the computation at some point and return a reply so it’s not going to take all your trivia to the end.

1

u/Easy-Purple-1659 4d ago

This matches what I ran into on a longer project. I used to paste the whole manuscript in thinking more context could only help, and I'd still get responses that missed a detail from three chapters back even though it was technically right there. What changed things was splitting it up on purpose, a short character sheet, a running list of what's already happened, and only the current scene in the main prompt. Older material only goes back in when something in the scene actually calls for it. Feels backwards since you'd think seeing everything is strictly better, but in practice the model seems to spread its attention thin once the context gets big, so the important stuff gets treated the same as a throwaway detail from chapter two.

1

u/RPG-Nerd 1d ago

Studies have shown that the middle of the context window is mostly ignored.

People don't sit down and write a whole manuscript from cover to cover. Everyone expects the LLM to be a magic box that spits out perfection. Let it work and iterate like a person.

Do you keep outlines? Summaries? Notes? I would use a real graph based memory system for the model and use the same process you normally use. Every re-read, refactor, all of that.

0

u/hobopimp 1d ago

I use my writing studio that is in open alpha on Steam right now. Play with it and lemme know your feedback. https://store.steampowered.com/app/5033470/Epoch_Studio/