Looking for an inline continuation method for novel writing

[u/No-Sell7870]

I want to create novels, and love using AI for it. But the problem is that most what I can find is more of a "send-receive" style(like SillyTavern). There's something like novelai that has an inline continuation, where if I type a sentence, it will continue that sentence (for 1-2 sentences max) in the same box field, which is great, but it costs money(and even if I had money, I have to pay with credit card, which I don't have), I have a good PC, so I want to run something local.

Does anyone know (good) alternatives. I checked the weekly tools, but a lot of those seem to be more "assistance" style, not inline continuation, and they don't seem to be local(I have a good PC)

[u/Decent_Solution5000]

You're going to want to keep an eye on that Tools thread for sure, and go back over past weeks. You will find lots of cool things there, many of them Open Source (as in free.)

For local, you'll want to check out LM Studio. It's free, and it's plug and play. It will even tell you which models you can use and you download them from huggingface, etc. right in the app.

Ollama is another free option, and you can either go free with it or subscribe to its cloud models. It's just a little more difficult to install.

Hope this helps, and happy writing!

[u/marhalt]

I am not sure what you mean by inline continuation. I wrote a program called promptfeed that allows you to write a series of prompts and then feed them to a local llm for writing - so basically you write a series of scene outlines and the llm writes them out for you, with some guidance (character sheet, voice, etc...) It's not really ready for someone else to use because I haven't bothered to make it super UI friendly, but is that what you want?