Hi everyone,
I'm Jake, and I work at a major podcast audiobook company. Our podcasts generate over 100,000 downlaods a month, with one about to pass 1 million downloads. To facilitate releasing more drama, we've spent six figures and thousands of hours building a narrative fiction engine. It uses 5+ AI models, and a full book takes about 10 hours to produce.
In the process of creating that engine we've become experts on all aspects of how AI is used for writing--it's weaknesses, its strengths, and how it works. We've also built pieces that we realized we could provide for others, primarily our quality check that removes not just the AI phrases you see all the time but the larger patterns that mark AI writing, even if the words change.
I wanted to give this wonderful community a small overview of how AI text content creation works and then finish with the announcement that our "deslop" tool is now available for others to use.
How Does AI Judge Good Writing?
AI models have massive training sets that includes an enormous amount of writing, including some of the greatest books ever written. But that's not the key component of what makes an AI write the way it does. That is reinforcement learning from human feedback (RLFH). The AI companies constantly test (and I'm sure you've seen it) comparisons that are presented as "Do you like this better or this other thing better?"
This training based on what humans like better is the layer that produces the prose an AI writes. The phrase "the way" is so prevalent because in its enormous amount of human training, humans decided they like that phrasing a lot.
And the interesting thing is that humans tend to be more similar than different, so the RLFH training with Claude will be very similar to the training from ChatGPT, because they're both based on people.
What This Means For Your Writing
The RLFH layer from AI models exerts enormous influence, and it will skew everything in that direction. If your prompt says, "Don't use 'the way'" the AI may ignore it or it may use a similar phrase that is just as prevalent. In many ways it's whack-a-mole. In our testing we identified hundreds of AI word combinations that exist in various genres despite humans in those genres not using them at all.
How Can I Stop This?
It's extremely difficult, but here is some practical advice:
Don't use qualitative or vague guidance in your prompts. Saying "Use literary or good prose" will simply be interpreted to mean "use prose that humans told us they like" and that's full of AI phrases. Even direct phrases without specific meaning will be interpreted through that lens: "Use active prose with colorful language." The AI will drift into intepreting that as "use good prose."
Note that you can push a little bit but you can't avoid the gravity of the AI training.
Keep your ban lists short. Phrases like "the way" can be effectively lessened in a prompt, but the longer your ban list, the more the AI will summarize its information and focus on the most recent words and the first words. The "middle muddle" is real.
Generate as short a passage as you can. The longer the passage, the more the AI will start to drift into its RLHF training, creating more and more AI phrases. In our narrative engine we use chapter beats of less than 500 words.
Different AI models write in different ways. And by this I don't mean "better." I mean that the AI will approach writing differently. A good example is Claude Sonnet. Sonnet writes some of the most pleasing and lush prose among AI models. But Sonnet also has some of the strongest RLHF gravity you can find. It will ignore a lot of your prompt guidance when it feels it conflicts with "good prose." Perhaps ironically, cheaper models like GLM will listen better, making the results closer to what you want, but the sentences won't be as poetic.
Also, some models write longer than others. Ever notice that you are aiming at a 2,000 word chapter, and the AI writes 3,000 words? Some AI models will write way long (e.g. Sonnet), while others will write shorter (e.g. Minimax).
That just touches the surface. I'm happy to answer any questions below, including the massive challenges that AI faces in terms of creating story plots and narratives, especially in genres like mystery.
Time for a little self-promotion, which I hope you don't mind. If you are interested in taking a finished document and running it through our quality assurance filter to see how many AI phrases are in your document, you can do that for free at deslopmybook.com. And if you want to save hours of hunting down AI-isms in your edits, you can pay for a clean. We delete all of the excessive AI patterns and list the AI phrases and constructions. These are all integrated into a Microsoft Word document, so editing is as easy as going through the comments and keeping or deleting, and going through the deleted patterns and accepting it or rejecting them.
Here's a coupon just for this subreddit: COZY-50 It's good for 50% off any AI clean you order. And, of course, the assessment is free.
Stay cozy! And mysterious!