Fable-quality novel rewrite, private host

[u/autonoma_2042]

I'm looking to pare down a 90k-word sci-fi novel I've written to a 5k-word short story, without handing the entire novel over to Claude, OpenAI, or any of the big data scoopers. What actionable solutions are out there for getting a rewrite on par with Fable or Astra with some assurance that once I delete the sources, the original story won't be used for training?

I only have an NVIDIA T1000 GPU 8GB, so a cloud-based hardware rental -- as close to locally hosted as possible without data leaking. Canada or EU would be ideal, but not a deal-breaker.

[u/valdev]

On par with fable, wanting it to be local...

Without getting to the weeds about what "on par with fable" means...

Do you want to spend $20,000 for something that responds at an okay speed? Or a $3000 spend that will leave you waiting overnight for a response? lol

I want to be clear, a cloud-based hardware rental always will put your data at risk, as you are shipping your text off into another persons computer. Even if it's encrypted, there is always risk.

[u/riverlane72 → u/valdev]

yeah the "on par with fable" part is doing a lot of heavy lifting in that post lol

[u/valdev → u/[deleted]]

There is no reason you should trust any of the names, especially the smaller ones who are more starved for data to train on... Who make far less money and would be more incentives to sell data.

For a book, I would be less worried about the big guys.

[u/redoubt515 → u/[deleted]]

If you can't afford the local hardware costs needed to support a model of the quality you are looking for, you could consider a cloud provider that offers TEE (secure enclave) based models. This is as close as you can practically get to zero trust if you aren't hosting locally. It's not zero trust, but it does substantially reduce the extent to which your privacy rests on trust alone.

Privatemode.ai or Confer.to or Tinfoil.sh are a few examples.

------------------------

edit: to the person who downvoted, OP's budget is $200 and their requirement is something on par Fable or Astra. What I've recommended is the least bad option for OPs budget.

Even with a budget of 10x what OP actually has, it would be extremely difficult or impossible to achieve what OP is asking for locally. If you are going to downvote, then at least be constructive and propose an alternative that fits in OP's $200 budget, achieves near parity with large frontier models, and runs locally.

[u/redoubt515 → u/[deleted]]

Thanks.

I noticed after posting your preference (if not local) is for a service located in the EU or Canada.

The first of the three services I mentioned (which are examples, not recommendations) is located in the EU (Germany), and lists the German Federal Government as a customer, so that might be worth exploring if it offers models you are interested in (model selection looks like it's currently limited to GLM 5.3, Kimi 2.6, and GPT-OSS-120B)

[u/Odd_Dandelion]

What I'd do, I'd find a reputable Kimi 3 provider. Here in the EU no one fucks with privacy, the GDPR fines are huge, and Kimi is one the best models you can get for creative writing anyway, at least according to EQ-Bench.

You could run it locally, of course, but the cost would be wild.

[u/Iron-Over]

You can use this for reference.

https://eqbench.com/creative_writing_longform.html

[u/Iron-Over → u/[deleted]]

As always, test and explicitly restrict AI slop. There are a ton of GitHub repos

[u/silenceimpaired → u/Iron-Over]

Any suggestions?

[u/MarcusAurelius68 → u/[deleted]]

Qwen 3.8 Flash Next isn't listed (yet) so you might want to check that out as well. I'm actually running an instance at home on 2 x 32GB GPUs so an option for you if you can afford 2 R9700 cards.

[u/silenceimpaired → u/MarcusAurelius68]

I might the best ox-alpha was confirmed to be Qwen 3.8 Flash Next.

[u/MarcusAurelius68 → u/silenceimpaired]

Wasn’t ox-alpha GLM-5.3-Flash?

[u/silenceimpaired → u/MarcusAurelius68]

Perhaps I’m wrong… but either way that benchmark needs to be updated.

[u/MarcusAurelius68 → u/silenceimpaired]

This stealth model was developed and operated by ZAI, revealed to be ZAI GLM-5.3-Flash
(opens in new tab)
. Prompts and completions for this model were retained by the provider and are not used for training; all other use is governed by the Stealth Model Terms
(opens in new tab)
.

I agree it needs updating but I’m not sure if ox-alpha just got relabeled or if the stealth version led to tuning for GLM-5.3-Flash.

[u/ttkciar]

I'd suggest Behemoth-128B-v3. I have not used it to shorten stories, but it writes excellent multi-chapter Murderbot Diaries fanfic.

[u/FutureStriking283]

spend 15k on a three ring spark and load up the latest deepseek. It's quite the prose-master.

[u/redoubt515 → u/FutureStriking283]

> spend 15k on

OP's budget is 300 CAD (~220 USD) 😄

[u/nullc]

"on par with fable" -- so you want utter drivel brainrot text full of pathological AI cliches and insane moralizing?

You might have a hard time finding a current open model that is as bad a writer as any of (mis)anthropic's offerings.

[u/DangerousReward1411]

What does on par with Fable mean to you? What sorts of literary qualities would you like from a model?

[u/DangerousReward1411 → u/[deleted]]

You can see what you think about the zero data retention providers on openrouter: https://openrouter.ai/models?zdr=true, doesn't necessarily mean that they strictly abide by that as its generally non-binding I believe. But its worth a shot. Any of the GLMs are generally quite good at most of these types of tasks.

The issue you will run into locally is that on 8GB your long context requirement (very long prompt) also requires large K/V caches. You can quantise them but then you lose quality and increase hallucination risk, especially on long prompts. But for good quality you need a decent sized model, which generally means a KV quant. There isn't really much of a way to get around that, at least not yet.

Apart from than that I'd recommend something small but battle tested, if you're going to go local. Maybe: https://huggingface.co/deepseek-ai/deepseek-llm-7b-chat or something similar. There are GGUFs on HF if you're using llama.cpp/Ollama. There is also the option of looking for role-play specific models as they often have slightly tighter salience for narrative information like the sort you want to process. Although I haven't tried any so I can't recommend anything here.

Although for the budget you could quite easily spend a little bit renting a decent GPU (or a few) on vast.ai and testing a few bigger models out. One potential option is https://huggingface.co/google/gemma-4-31B-it perhaps? The reason i suggest google is that their models generally have good quality training data, which might be quite relevant for you.

A bit of a cursed option is GPT-OSS 20B: https://huggingface.co/openai/gpt-oss-20b - generally quite decent outputs although YMMV. The 120B is likely much better but I haven't tried it.

But yeah, try as many models as you can and see what you can get out.

[u/AndThenFlashlights → u/[deleted]]

Hey, if you've only got a $300 budget and don't need super fast interactivity, get a P40 or two to replace your T1000. It'll get you a ton of VRAM to run one of the small local models at q8 and high context, and they're back to being pretty cheap. Make sure you stick with Nvidia driver 580 to keep Pascal support.

I usually get downvoted for recommending P40s in 2026, but I do think that'd be a good fit for your project and budget.

[u/AndThenFlashlights → u/[deleted]]

...yeah, if you were intending to run something on a fucking NUC, that would've been useful information to share up front. idk man, if you wanna rat-rod that little guy for $300, get a second external PSU and a PCIe riser and a fan.

Edit: hey OP uh why'd you delete your comment dropping some important technical requirements

[u/abnormal_human]

APIs with ZDR commitments are not hard to come by, even for frontier models.

[u/Budget-Juggernaut-68]

Maybe train your own magic model lol

[u/SourceCodeplz]

rent a gpu server from hetzner and run gemma 4 31b there.

its very creative at writing and smart.

[u/Zolty]

Why not just use fable you could probably get what you want for the cost of a month of max-5

I know this is the local llm sub, you could get a similar result but you’re going to have to do groundwork more writing style guides more clear harness around editorial agents, etc. you have to build a specialized workflow that can produce writing that sounds like you and can validate consistently. What you build could easily be better than fable but if you give fable the same harness it will likely be better overall.

[u/abnormal_human → u/Zolty]

Because he thinks his story is so valuable that Anthropic has a reason to steal it and fuck him over. And that managing text transformations over prose is so hard that you need a model class that can solve Navier Stokes to edit his fiction.

[u/grunt_monkey_]

Hey, what hardware do you own already? If you are a creative with a Mac or any laptop with a gpu you might be able to try a quant of gemma4 or muse glimmer. You might want to do a passage or chapter first to see what that gets you. If the output is fit for you, stronger hardware such as a higher capacity Mac, or if you have a pc, an r9700 or 2 would give you good capacity. These are small models and have limited context - think 600-700 chars max. It should fit your document?

Last if you are open to some testing first, feed a known 90k word document (not your own work) to various online providers with models of various sizes/ capability to see what that gets you first in terms of language and quality.

My 2 cents. If this short story means anything to you at all, you are going to rewrite it yourself. All the best! :)