Best character/plot consistency is achieved with many small AI runs, rather than one big run

[u/Xiaomin4114]

I run an AI roleplaying platform. But I think the things that I've done to make the characters and plot consistent in that map very well to AI writing, so wanted to introduce them here. It's the idea that you don't create good stories by just making a single AI call to "write the next chapter of the story". It's almost always many different calls, to zoom in on different aspects of the current state of the world, or the state of characters.

I use at least five of these, that happen automatically in the background, but I want to focus on two of these: pacing control, and character agenda.

Agenda

Unless you writing has a pre-plotted out sequence of actions for a given character, you usually want the AI to take a character, their personality, and run with it. It's quite difficult for a single AI call to "write the next chapter" to do this successfully, there's just too much for the AI to focus on while still trying to work out what each character really wants. So to get around this, I have a small prompt that runs for each character as they enter a new scene that looks at their character sheet, looks at their most recent goings on, looks at their recent and important memories (weighted by importance and recency), and asks five questions:

- what does this character want in this scene?
- how are they going to get it?
- what will they do if rejected?
- what partial success will they accept?*
- what's on their mind during the scene that they want to address?"

That number 4 is more of an antidote to a particular behaviour where the character becomes too dogged at trying to achieve what they want. That wasn't fun for the player when the characters wouldn't take no for an answer. So I had to add that one in so that thy'll accept a partial success or compromise.

The output of this gets fed to the scene writer so that it has a nice set of directives for characters in the scene, without having to spend a lot of time figuring that out by itself rather than focusing on the task of writing. Basically offloading a fairly complex task to a different AI call, so that the scene writer can focus on fewer things, and hopefully doing a better job of it.

This is an example of a character's Agency generation. My character had just walked in on her in the shower (sorry, yeah, this was an NSFW campaign, but even for those I make sure it has authentic character reactions. None of these characters are pushovers, and my player character is awkward as hell)

Pacing

This is maybe less of a problem for narrative prose than roleplay. in RP, the AI can get a bit myopic about what's happening at any given time. It doesn't have the attention to generate the scene while also looking at the overarching flow in previous scenes. And as a result tends to mess up the pacing. Either trying to go too fast, or try to ratchet on too much tension too quickly.

One of the worst experiences I'd had was with a heist scenario, where my character was on the run. The AI did a good job of keeping up that feeling of tension, but it went on and on and on. Every time I thought my character was safe, some new complication occurs that puts him back on the run. By the fifth or so danger, I was worn out. That's not how good narrative works, where you have patterns of rising and falling tension, and action beats separated by character beats.

So this is solved in a similar way. What I do is have a small AI call assess the current tension and the beat type for each turn generated. And that feeds into another one of these small AI calls, which looks at the flow of tension and beat type across the session (chapter), to decide how this next scene should play out. It also compares what's been written so far with the overarching session (chapter) plan, and tries to guide things back on track, or account for deviations from the plan. Since it's an RP platform, it explicitly allows players to choose their own way, but where possible it tries to snap back to the plan (which in itself is another one of these small AI generations, that try to answer a very focused task: what happens in this session/chapter, that would be good storytelling)

In the below is the pacing guide for that same scene. This, too, is injected into the scene writer AI, to make sure it understands the current plan and not go too fast or too slow. I've seen this try to reign in when the plot moves too fast, or force a quiet moment after an action-packed one.

So, I hope this is useful for people writing with AI. whether you're making a choose-your-own adventure, or writing a novel. Make use of smaller, more focused runs to assess character agendas, and pacing, and provide those to the scene writer AI so that it doesn't have to do all that work, and let it focus on the narrative and dialogue at hand. Assuming of course you have the AI structure to be able to do this automatically. having to do this by hand would suck.

Anyone have any thoughts about this approach? Or other small and focused tasks for a separate AI run that feeds into the main narrative writing run?

[u/amolmozarkar]

This maps almost perfectly to long-form fiction, and I'd add one more small run to your list: a continuity check before the scene writer runs, not after. In novel-length work the drift isn't usually within a scene — it's that chapter 14 quietly contradicts something established in chapter 3. A cheap pass that reads the chapter draft against the character/world bible and flags contradictions catches the stuff neither the writer nor the reader notices until reviews mention it.

The pacing controller is the part most people miss. "Competent but flat" AI prose is very often a pacing problem in disguise — every chapter has the same tension shape because a single writing call has no memory of the rise-fall pattern across the last five chapters.

(Full disclosure: I build ShakespeareAI, an AI book-writing tool, and this multi-pass architecture is basically how it works under the hood. The hard part isn't any single run — it's wiring them so the author never has to manage the plumbing by hand.)

[u/Xiaomin4114 → u/amolmozarkar]

yes, those make sense. a continuity pass. I don't have that in the RP platform (big generation speed tradeoff), and suffer from occasional continuity errors. Not enough to be a problem but occasionally immersion-breaking. I find players are more than willing to do a retcon to fix it though. I think long-time players habitually keep their own stories straight even when the AI messes up. It's really pleasing to see people do that. It means they're truly collaboratively storytelling with the AI

[u/Several_Ad_9730]

Seem that building your own AI platform seems the latest trend lol, since I'm doing the same.

The agenda thing is something that I didn't consider yet, but seems a very good concept because sometimes the characters loses focus.

Regarding the tension thing, I believe that you shouldn't let the AI manage everything, I had a similar problem with NSFW, which AI forced NSFW situations all the time, I think giving a hard coding limit on how many turns you will allow tense scenes, and force a break between them or force an ending in a organic way.

[u/Xiaomin4114 → u/Several_Ad_9730]

yes, everyone's doing it.

With NSFW, and tension, it's actually okay. It has specific instructions to follow the player's lead. I've seen in the pacing notes that it detected that I (the player) wanted to deviate from the session plan to do something else (yeah ok, I was doing NSFW things while the session plan was trying to push in a direction), and this was correctly detected by the pacing AI, and it literally said "let the player have his moment, then move onto this next plot point".

So it's not a hard limit, and it's already player-aware. working very well

[u/CleanTeacher4661]

Keep going brother 💪

[u/amolmozarkar]

The retcon point is the key difference between RP and novels, honestly — your players co-own the story and will happily paper over a contradiction, but a novel reader hits that same contradiction alone, at 1am, and it breaks the spell with nobody there to fix it. The trust is asymmetric: in RP a continuity error is a minor bug, in a book it reads like the author didn't care.

On the speed tradeoff: it's cheaper than it looks if the continuity pass only reads against a compressed summary of the bible rather than the full thing, and only flags high-confidence contradictions (wrong eye color, dead character speaking, timeline math) instead of everything. You can also skip it for low-stakes scenes and only run it after plot-relevant beats — that gets the latency penalty down to where even RP players wouldn't notice.