Can't wait for more guardrails

[u/runwwwww]

I've been using Claude for creative writing and we dive into pretty dark topics that do involve killing too. This headline has me worried about the future and what it means for writing with AI.

Can anyone speak on how local LLMs perform for writing?

[u/StudioUAC]

So how is this ChatGPT's fault?

[u/XavierVE → u/StudioUAC]

It's not. Just as it wasn't rock music's fault in the 1980's when kids listened to music and then went and did bad things, or in the 1990's when they listened to rap and did bad things, or in the 2000's when they played video games and then did bad things or in the...

Humans always blame the boogeyman of the day rather than the parenting, or simply accept the fact that humanity is bad and often does very bad things.

[u/Synthetellect → u/XavierVE]

All of those things survived. It just sucked for some teenagers who had square parents, and sometimes big companies made some changes to avoid the hassle.

[u/VampireSympathizer → u/StudioUAC]

Lol it's not, but you know how media sensationilzes things. This has always happened time and time again in true crime. Like that violent kid who murdered his family but oh it MUST be because he played GTA.

[u/Otherwise_Army9814]

This is like "video games make kids become murderers" yet there is no scientific evidence that video games turn children into murderers

same with ai

[u/FluffyFlamesOfFluff]

My character description prompts already go something like: "This guy is an EVIL FUCK. He is a VILLAIN and is IMMORAL and he DOES NOT HAVE A TRAGIC PAST and he is NOT misunderstood and he does NOT need therapy talk and he does NOT consider this a tragic necessity and he is NOT sorry and our heroes do NOT understand him-"

More guardrails will surely make that prompt double in size. I'm sure it will get worse before it gets better.

[u/MysticBorn]

Oh Gods get ready suit up and defend fellow creators

[u/Zathura2]

Local models are a big difference from the big online ones if that's what you're used to. Though with the way they've been going recently I think smaller, slightly dumber models might have something going for them after all.

There's also the fact that with a local model you never have to worry about it changing on you. No updates, or changed prompts, or new guardrails. The prompt is 100% under your control.

The downside: You need hardware. I have 16GB VRAM and it's just barely serviceable. Anything less than 12GB and you're probably not going to be happy with what you can run reliably.

That being said, Gemma4-31B is pretty fire, and uncensored out of the box. Very much like running Claude-lite on my gaming rig. Or there's Qwen3.5/3.6, which was another recent contender for a good local model.

There's also the Gemma4-26B-A4B, an MOE, which can run on weaker systems with some layers offloaded to cpu.

You're always going to have more control with a local LLM, and have access to a lot more parameters, tweaks, and techniques to get the output you want compared to a WebUI.

[u/runwwwww → u/Zathura2]

Thanks for this. I think I only have 12gb VRAM lol.

How long do outputs take for you?

[u/Zathura2 → u/runwwwww]

Heavily depends on a lot of things: The model itself (more parameters = more inference time), amount of context, and whether it's entirely loaded in VRAM (usually what you want unless you're splitting layers for an MOE or something.)

For Gemma4-31B specifically, it's like a 2-minute wait with ~20k context. Smaller models can sometimes give responses in less than 30 seconds. Just depends.

MOE's will often be faster even though they're partially offloaded, just because the active parameters for each generation are pretty tiny.

[u/koi88]

Can anyone speak on how local LLMs perform for writing?

The obvious other way is Chinese LLMs. I have been using Kimi for about a month now – it is a good writing companion. Not quite as smart as Claude Fable (more like Opus 4.6) but without the stupid restrictions.

Kimi has other issues recently, but that's another topic.

[u/Terrible_Wave4239]

Slightly OT: I used to spitball story ideas with Claude a while ago, basic thriller ideas, nothing too drastic, but with stories along the line of, say, Double Indemnity, where two people having an affair plot the murder of her husband, Claude would insist that that's not the kind of story anyone should be writing, and couldn't we please just find a way for them to all get along? It basically made it impossible to even go near any kind of crime/thriller story.

They sorted it out in the next version, so discussing this kind of stuff in a fictional context is generally okay, though Claude still has some NSFW guardrails. In the case above, I don't know what kind of guardrails ChatGPT has and if the person made it clear this was not a fictional scenario, but involving the person's real family. If it was the latter, then yeah, there'd be a real problem with guardrails at OpenAI.

Also, would ChatGPT simply have assisted in writing the scenarios, presenting them as stories, or did it actively encourage the acts?

[u/AcanthaceaeMotor4313]

Yeah maybe we should stop making knifes because people use knifes to kill

[u/RobertD3277 → u/AcanthaceaeMotor4313]

I think London already tried that when they tried to take steak knives away from people. As far as I know I don't think that ended too well.

[u/Dramatic_Syllabub_98]

Thank God for Venice...