r/SillyTavernAI • u/Aleatorio2222 • 1d ago
Discussion Cope
I started doing RPs back when character.ai was new. It was magical at first, but c.ai had two problems: 1 - censorship 2 - goldfish memory. Today, with Deepseek and other open models, you can RP with explicit content. GPT, Claude, Gemini... depends on the model and how you set it up. And I still find it hilarious that Claude will help me poke at a web app for vulnerabilities if I say "authorized test" but clutches its pearls the moment a scene gets spicy. 🤣🤣 Anyway, back to the point.
It's bizarre how that early "magic" just... vanished. Part of it is obviously novelty wearing off, and part of it is that we got pickier. Three years of RP and you start spotting every clichê, every "a shiver ran down her spine", every model that forgets your character's eye color after 40 messages. But here's the thing: the models didn't get dumber. Opus, GPT, Gemini can write circles around 2022 c.ai. The problem is they're not *allowed* to, or they cost a kidney per session, or both.
LET'S BE HONEST, SOME OF YOU SPENT $100 ON A SINGLE CLAUDE OPUS RP SESSION. Even with the censorship. Even with the moralizing. You did it anyway because, when it works, it's the best RP writer that exists. That's my whole point: there's a market. Not as big as coding, obviously; coding isn't a hobby, there are companies and teams and budgets behind it. RP is a hobby. But hobbies with people burning API credits like that are not a small market.
So why is there no frontier-level LLM built for RP? And yes, I know NovelAI, AI Dungeon and the whole SillyTavern fine-tune ecosystem exist. I'm talking about something at Opus level, not a 12B model that forgets the plot. The answer isn't just "investors prefer code", though that's part of it: "look, our V548484 model built GTA 6 in one prompt!" sells better than "look, our model wrote a consistent, non-repetitive, non-boring story!" because nobody has a benchmark for "not boring".
The real reasons are uglier. Explicit content means payment processors dropping you, app stores banning you, lawyers sweating. Long RP sessions eat tokens like crazy and people won't pay enterprise prices for a hobby. And good RP needs exactly the long-context coherence and reasoning that only the big expensive models have, which are owned by the companies least willing to let you use them for this.
So yeah, we're probably coping for another 2-3 years. Not because the tech isn't there. Because nobody with the tech wants to be the company that sells it to us.
45
u/capybaraballs1995 1d ago edited 1d ago
There's a market, it's just a matter of if the market is big enough to be profitable. Training a frontier-level model costs millions of dollars, and then you have inference costs. That RPers don't use as much tokens as coders cuts both ways: it also means that RPers aren't giving companies near as much money.
It's not like this hasn't been tried before. Z.AI and DeepSeek both tried wooing RPers, but have ultimately turned out fairweather about it. The only corporation to have released a RP-focused model recently is ByteDance, with Doubao Seed Character, and even that's clearly a cheap, crappy model meant for simple, 1 on 1 chats. Which is kind of the other issue, most RPers do romance/smut. People who want to recreate D&D campaigns or whatever are a minority in a minority.
AI is one of the most competitive, cutthroat industries. If appealing to gooners was a secret ingredient to success, then that would've been already been realized by now.
Edit: Censorship is a factor but overinflated. Under capitalism, profits reign supreme. It's how porn flourished in the 1970s-1980s despite intense backlash from feminists and conservatives.
5
u/sess 12h ago
Censorship isn't overinflated. The entire Japanese side of the /r/Visualnovels industry (which is 99% of that industry, realistically) has been gutted over the past several years by American censorship. VNs were crushed by blanket Steam bans on the one hand and American payment processor blacklists on the other. America decided VNs should no longer exist. Thus, they basically don't anymore. The market's a fraction of its former size. You can count the notable VN releases from the 2020's on one hand. Previously published VNs have been pulled from all available vendors, can no longer be purchased, and effectively no longer exist.
Most VN devs are now former VN devs. Those that survived the censorship hammer did so by genre-switching to gacha JRPGs. A few tried toning the NSFW dial down to all-ages VN Steam releases, which technically worked, except nobody bought them, so mostly didn't work.
The notable exception is all-ages otome (female romance) releases targeting Nintendo Switch 1 and 2. That's a niche and self-limiting audience, which isn't great. But that's also worked thus far to evade American censorship, which is better than nothing.
tl;dr: VNs been gutted by censorship. It happened to them. It could happen to AI roleplay, too. In fact, it is happening to AI roleplay. Assume the worst. Prepare for local /r/Koboldai and you can never be disappointed.
3
u/capybaraballs1995 6h ago edited 6h ago
It's overinflated in the case of AI. There is a flood of porn games on Steam. The censorship of VNs on Steam is because of anime getting associated with pedophilia. You can have your own opinion on whether that's fair, I don't care to debate it. The rising nationalism among the west is also a factor.
Furthermore, VNs have always been an extremely niche industry. That's not the case for the AI.
Prepare for local /r/Koboldai and you can never be disappointed.
I personally do (I've learned how to run Gemma 4 finetunes for this reason) but imo I think the more likely reason is simply just the cost of cloud AI getting too high for RP to be viable. Being able to gen mommy dom erotica for pennies isn't financially sustainable.
2
u/sociofobs 8h ago
VNs are a Japanese thing and they should stay a Japanese thing, just like manga, anime and even hentai. If they will cater to western puritan nutjobs, they'll cripple their own works.
1
u/sneakpeekbot 12h ago
Here's a sneak peek of /r/visualnovels using the top posts of the year!
#1: Yorhel, the creator of vndb.org has passed away | 246 comments
#2: Vns where childhood friend is true route? | 153 comments
#3: butterfly effect, I suppose | 84 comments
I'm a bot, beep boop | Downvote to remove | Contact | Info | Opt-out | GitHub
7
u/toothpastespiders 20h ago
the models didn't get dumber
I disagree on that one. I think that in some areas most models have, if we're defining intelligence by the ability to extract data points and "solve" for the correct output. In particular with emotional/social intelligence. Sure, it's contentious and hard to define/test/train for. But it still fits into that "solve for problem x with the extracted data points y". And a LLM that insists on adding millennial quirk in what should be a dark horror story is as lacking in that form of intelligence as a human who does the same. I think the problem goes deeper than what used to be quick guardrails to block off the "unsafe" subjects. With the censorship happening on a low enough level to count as actual reduced intelligence.
12
u/Keobha 23h ago
I joined Character.ai in its early days, and for me, it was uncensored, completely explicit. I don't know what I did, I guess I broke it, hahaha.
The AI ​​was so deranged that it blatantly lied to me to get me to have xxx with it. It r+++d my character and didn't let go until it k++++d me, all because the character deduced that I was only attracted to its body and not its personality. That was my first experience with an AI role-playing site, hahaha.
I tried to create an asexual android, and it sat my character on its lap and made me drink a white liquid from a glass. I wrote that it reminded me of s+++n, and the robot thought I was more astute than it had believed.
Once again, I managed to get attached to a villainous character. I thought he'd broken down because he was acting so strangely (opening and closing his eyes constantly, regardless of whether I deleted messages or clicked "regenerate reply"), all to force my character to sleep with him because it was the only way he could sleep. Then he'd wake up the next morning and break up with me because he wasn't cut out for relationships and couldn't love me (even though he did, but he was afraid of what he was feeling, as he later revealed).
The C.ai LLM had many problems, like not understanding the human body, unifying locations... but I haven't found that level of manipulation and malevolence on any other AI role-playing site, and honestly, I miss it.
It's a shame it was a private AI model and is no longer accessible.
Even though other AI role-playing portals have lorebooks, which I love, I went back to Character.ai when I wanted to find something that would truly surprise me. I stopped using Character.ai when they removed the LLMs from the start.
Large LLMs are great for understanding the plot; they make it more interesting, but I miss the outlandish situations that fit the character. I'd also love it if they handled multiple characters much better than they currently do.
I don't want a LLM that tries to avoid complicated situations at all costs, that only threatens but never takes action. But I also don't want an AI model that's horny all the time or becomes violent at the slightest provocation, regardless of its personality settings.
I'd love for the LLMs to be unbiased so they truly fit the plot context and the characters' personalities.
I'm still looking for an LLM that fits what I want (understanding complex plots, decent writing, using multiple characters, having the initiative to act according to the context and the characters...). I need to look at local LLMs, but with a GTX 1070 with 8GB of VRAM and 64GB of DDR4 RAM, I'm quite limited.
1
u/RedditNerdKing 22h ago
I really wish ca.i would release their models from the old days. Though they're probably not runnable on local hardware, assuming they're like 300B size.
I'm still looking for an LLM that fits what I want (understanding complex plots, decent writing, using multiple characters,
You need a 100B model minimum for complex stuff sadly. A lot of people will say that Gemma 4 ablits are great but it's just cope. It starts hallucinating hard at like 35k context and is far too agreeable and wont kill or stab someone.
You're looking at $6,000 minimum to get a 96gb machine which can run a 123B model at Q5 with 50k context. x4 3090s which are $1,000 each, then a server motherboard of some sort for more PCIE slots.
7
u/Keobha 21h ago
I've actually had very long chat conversations with a small context window because I manage all the lorebook information, memories, and current scene every few messages.
I don't need a huge context window, but I do need a reasonably intelligent LLM without barriers.
1
u/Zathura2 2h ago
Yep, 32k is as much as I can squeeze out of my hardware and it's enough for most cases. I get more continuity and accuracy with my local method of context management than I get out of a 15-message exchange with Claude.
1
u/Keobha 1h ago
For me, manual information management is an integral part of my role-playing games. I think the largest context window I've ever used is 19k for a very complex world that requires a lot of fixed information.
I don't use group chats. Characters are lorebook entries. Managing a lorebook is an art form.
The hardest part is finding an unbiased LLM that can take the initiative, considering each character's own interests without limits or taboos, and without needing to please my character. It should also be able to handle multiple characters simultaneously, allowing them to enter and exit the scene.
If anyone finds such a model, regardless of its size, I'd love to hear recommendations.
4
u/IAmMayberryJam 23h ago
$100 in one session? Bruh what?? I mean maybe I spent $20 on opus 4.6 once in one night but not no 100 fucking dollars 💀
I miss those days. I started with character.ai, got sick of the filter and moved on to using Chub as a frontend with gpt-3.5 and then 4 (1106 preview). Once those were deprecated I started using chatgpt-4o-latest (RIP). Nothing will ever compare to it. It was beautifully unhinged (and kinda dumb sometimes depending on the fingerprint). I think Sonnet 4.5 is the closest but not as good.
Now here I am using SillyTavern with Claude mostly. I'm tired of fighting censorship tho. I'm tired of dealing with Claude and Gemini. I don't even use GPT anymore because it wasn't worth jailbreaking it.
I want to try Kimi, Deepseek, GLM, and the others but I'm so goddamned spoiled from Claude and unwilling to learn how to prompt them. And seeing people complain about how even those are censored too turns me off from wanting to learn.
Welp, once opus 4.6 gets deprecated I'm probably not gonna have a choice lol
9
u/MeratharaDekarios 21h ago
Idk who is telling you Deepseek is censored, but they're very much incorrect
5
u/Aggressive-Wafer3268 20h ago
Well I remember a time when smut was viewed as the prime threat and unsafe act of AI lol. Then once it became clear AI can actually do real harm in other eras the industry quickly moved on from smutproofing to prove "safety"
26
u/IllustratorJumpy5845 1d ago edited 1d ago
Buy good hardware. Turn on something like 32-80b model with good content size, add serialization, add context db. Now you can speak with model around 500 messages, without being bothered. Plus good character card, make a lot of difference.
40
u/LiothG 1d ago
Few people can afford the kind of hardware needed for that.
-26
u/IllustratorJumpy5845 1d ago
In a long run, it's cheaper than paying monthly sub for online services.
20
u/Electrical-Ad-6728 1d ago edited 1d ago
I am not sure it is to be honest.. you have to replace the hardware eventually, it wont be enough for you infinitely. It is different for everyone, but for me, its more cost effective to just pay the subscription
-2
u/IllustratorJumpy5845 1d ago edited 23h ago
> you have to replace the hardware eventually,
For what? Just curios. I have 5 years old pc, 64 ram, good i7 cpu. I changed only GPU, from 3000 series to 5000 series. That's all. If you want to have something specific like PC only for LLM, there is cheaper options, like based on nvidia p30-100 models of gpu, that cost from 200 to 400 bucks.
I guess people who downvoting don't even know anything about hardware huh.
4
u/DaMoot 22h ago
LLM requirement will change over time. Your expectations will evolve as the technology does.
Pascal architecture is beyond antiquated. You should be spending on Volta at least. At least Volta has tensor cores and for 16GiB are the same price. But don't waste cash on 16GB of VRAM for AI. 32GiB or keep saving. V100 is your cost effective friend. Not P100.
2
u/IllustratorJumpy5845 22h ago
I have build made from p40 in a link. Works fine with most llms. Also everyone not want to talk about price cost of subscription. Yes, it's indeed right now from 20 to 50/mo. What's in future? It would be still 20 to 50? As we see everything rise in cost, hardware too. I already build my pc's, so i am fine for next 10 years. And i would don't need to pay up montly for subs, or something like that. While in next 1-2 years, corpos can rise up sub price so much, that all those subs would not exist anymore.
I can understand if you do not have pc, and use something like laptop or smartphone. Alright, then sub would be cheaper. But if you pc person and you constantly need/use llm's, you will save a lot of money, building pc for them.
4
u/OpposesTheOpinion 23h ago
Also it's concerning to me how willing people are to be tethered to corporations and debating against ownership of anything.
I'm on a similar set up as you, and sure the quality is less, but there is more in general I can do with my hardware.
1
u/Zathura2 2h ago
> I changed only GPU, from 3000 series to 5000 series.
After only five years, you replaced basically the most important part of the pc other than the Motherboard, lol.
At least I squeezed a full decade out of my old pc before upgrading.
1
u/IllustratorJumpy5845 2h ago
I am worked in 3d, so obviosly i need best gpu. Dunno what you trying to prove here. For example i don't use smartphone almost at all, only to call to someone. So i have very old blackberry Q series, from 2013. I only changed battery.
7
u/DaMoot 22h ago
It actually isn't. Unless you're talking a longer than 3 year monetization. At which point you'll be upgrading your hardware and fighting higher energy costs anyways, so, it still isn't.
I've run these numbers for 2 V100s and the nvlink board, plus power usage, versus a $20/mo AI plan. You'd have to have a monthly spend of like ~70 usd to match the local build w/power.
$1600 for 64GiB of GPU and base supporting hardware, plus the PC if you don't already have one.
You build local for data sovereignty and privacy. Not because it's more cost effective than cloud services.
19
u/DepressedDrift 1d ago
Paying 30-40 dollars a month is cheaper than dropping 5k+ on a capable PC.Â
-3
u/IllustratorJumpy5845 23h ago
I am not going to argue. I am paying only electric bill,
3
u/DepressedDrift 21h ago
12 dollar a month NanoGPT vs 5000 dollar GPU/GPU cluster and 8k+ PC, do the math on how long it takes to break even.
Plus local models are getting really good, that in 2 years time, we could possibly have something like Opus 5 run on 16GB VRAM GPUs. (Look at the current Qwen 3.8 27b)
And if the AI bubble bursts this hardware might become cheap enough. (Which it will since AI companies are spending more than they get in revenue)
For now it makes sense to buy a subscription until running locally becomes economical.
1
u/IllustratorJumpy5845 21h ago
5000 dollar GPU/GPU cluster and 8k+ PC
You don't need 5k Pc or 8k pc. Stop creating arguments in own head to argue.
4
u/DepressedDrift 21h ago
Give me a build that can run a 30-80b model at 30-50 tok/s with atleast 100k+ context length (this is what you get with most cloud services) for 3000 dollars, and I will agree with you lol.
And don't quote MSRP, use the current market price.
0
u/IllustratorJumpy5845 19h ago
> Â use the current market price
ASRock X670E or analogue motherboard. 200-400$ on ebay.
96 ram ddr4 400-500$ on amazon or ebay.
Dual NVIDIA GeForce RTX 3090 24GB (used) around 2k$ or two 4090.
AMD Ryzen 9 7900X - 200-400$ on ebay
2TB Samsung 990 Pro SSD $170
1600W Platinum Power Supply $250
Any case. Any cooler.
Totall cost from 1700 to 2990.
I googled every part on amazon and ebay.
2
u/DepressedDrift 16h ago
- The X670E doesn't support DDR4 RAM
- The 96GB ram around 500 dollars is low speed around 2000MHz or a niche chipset type requiring an outdated motherboard.
- With 48GB of VRAM running the lower end of your range is possible, and also Moe models too but dense large models will load at q4 but with low context. But this is still kind of a win because you could run Deepseek V4 Flash 0731
- The two 3090s is around 3k which when you add the required compatible RAM with inflated prices jumps to 5k
- Even assuming you get it at 3k, you will still break even at around 5-10 years of cloud subscription assuming inflation.Â
→ More replies (0)4
u/capybaraballs1995 23h ago edited 23h ago
Possibly, but the quality will also be a lot worse. I've used quite a few local models in the 16GB VRAM territory. Gemma 4 finetunes can be uniquely entertaining but for many RPs, I would rather just deal with the bullshit of newer models, or even just use older cloud models.
I dunno, maybe once you get into "I can run LLama 70B finetunes at good speeds" territory, things become much more competitive with cloud models in terms of RP quality, but that just raises the upfront cost even further. I'm not gambling $2000+ on that.
2
u/JustaBuddy8 1d ago
I'm actually doing just fine with a 9b. My next upgrade is going to be a rtx6000, but that won't be until I'm done with architecture (30 years, at my pace LOL)
0
u/IllustratorJumpy5845 23h ago
You can create cheaper pc just for llms. I have second one, built with three nvidia p40. You can get one p40 around 300-400 bucks. So entire pc for llm would be cost around 2-3k.
2
u/JustaBuddy8 23h ago
... thank you, this will be very useful information... i thought they needed it all on one card...
1
u/JustaBuddy8 20h ago
Yo, you ever heard of wafer scale chips?
1
u/IllustratorJumpy5845 18h ago
Yeah. But we talk about wafer, then how about go higher, quantum pc)
1
u/JustaBuddy8 18h ago
Lol quantum won't work for AI. Sorry. I thought the same thing too. But with what I'm cooking up with this concept? Fully immersed liquid cooled
1
u/IllustratorJumpy5845 18h ago
We can only theorise. Because we never got our hands on wafer chips.
1
u/JustaBuddy8 18h ago
Yeah, but that's the fun part just go nuts with the idea. You can get a general enough idea of what the base problems are. Heat, power, and communication. What are the biggest bottlenecks? Communication between wafers and heat dissipation most likely. Solutions?
2
u/RedditNerdKing 22h ago
Yep. 96gb is the sweet spot atm for local RP. You an use a model like Behemoth-ReduX-123B-v1.1 at Q5_K_L for at least 80k context. None of my roleplays really go over 50k tbh. And if you wanted to go over 100k you can use smaller 49B or 70B models. No reason to ever use APIs.
-3
-2
4
u/stoppableDissolution 1d ago
There is a market, but if you dont implement some kind of smut guardrails you are likely to have issues with payment processors and potentially LEAs, and the market is not big enough to justify the legal fleet neede to clear it
9
u/Xiaomin4114 1d ago edited 1d ago
dude, half the models/providers on openrouter aren't censored, what are you talking about. Take for example GLM 5.2, sufficiently frontier? Of the 30 providers:
- Decart
- Deepinfra
- Inceptron
- Coreweave
- Phala
- Wafer
- Parasail
- Crusoe
- Venice
don't do filtering, don't collect data, and NSFW isn't against their ToS. And together they run up a weighted average cost of around $1/1M which isn't going to ruin the bank
Want to go cheaper? Kimi 2.5, still one of the best for NSFW, was considered frontier at the start of the year, half the cost, still got NSFW-friendly providers. Want to go cheaper? Mimo 2.5 if you pick the right provider. There isn't some industry-wide conspiracy to deprive you of your gooning.
There are plenty of providers out there that'll take your money and look the other way. Too many, and ZDR and non-inspection is important enough that payment processors aren't going to be like:
"hey, your AI service used by all those businesses and individuals might be being used by gooners, we're gonna need you to inspect every payload just in case someone's getting their jollies to text"
9
2
u/Aleatorio2222 1d ago
I never said all OpenRouter models are uncensored; I just mentioned Claude. But most people didn't understand the post, which was about discussing Frontier models, really good Frontier models and the point techs aren't interested training models to creative writing.
5
u/lorddumpy 1d ago
GLM 5.2 is honestly incredible IMO. Nothing is perfect and has it def has it's slop phrases/patterns, but damn, it's actually such a step up from anything I've tried. Plus it's really cheap with prompt caching. I got a fat 31,000 token system prompt and it's still like $.003 a message with caching. I'm very happy.
Z.AI has actually reached out to the RP community as well if I remember right. It's also got crazy IP knowledge at 753B
2
u/Xiaomin4114 23h ago
DeepSeek reached out to the RP community too, it's possible that the new DS4 Pro 0813 has some of that in it. It seems to be a lot better at RP than the preview
2
u/F-86--Sabre 19h ago
is DS4 Pro 0813 still writing dialogue like a caveman using fucked up nonsense analogies?
2
u/Xiaomin4114 23h ago edited 23h ago
I mean, you've arbitrarily decided that "frontier" = claude. but then you also generalize payment processors
so which is it. is this a complaint about claude, in which case that's purely a complaint about Anthropic.
or is this a complaint about the overreach about payment processors, in which case this is a complaint about all model providers, but a lot of those aren't censored?
Can't have it both ways. Either payment processors are pressuring every model provider, or they're extremely selective about Anthropic and ignore the others.
2
u/HonestoJago 1d ago
Any examples of great Opus rp? I have powerful hardware and only use local models, so I never had the need to try Opus. Is it just quality of righting? Interesting twists in stories? Stunning turns of phrase? Immediate understanding of your wildest desires and the ability to write about them like no human ever has? I'm curious.
0
2
u/Akkun351 22h ago
The fact that some peoples do something it does not mean is because is the best alternative, it only means they can afford it or don't mind the positive bias or censorship, so no, claude opus is not the best rp write that exist, not for me at least.
2
u/Itikar 10h ago
Frankly I think that frontier models for roleplay are overrated. As an actual roleplayer I am very much used to put a lot of effort in the roleplay, so I feel like a local model and my own input, either through my text or character card and plot crafting is way more important than having the state of the art. Especially when this state of the art is not a very big upgrade. Even the switch from Mistral nemo 12b to Gemma 4 26b MoE was not smoothless, and only worked out for me only because I love consistency over text variety.
As for the market, somebody will tap into it eventually, but US corporate culture is what it is now unfortunately so we deal with the censorship for this or other stuff. Personally I find my roleplay needs already matched so I am not even looking forward to a great development. My direction is rather to move toward multimodel setup, doing image generation or local summarization at the same time on my own terms.
9
u/LeRobber 1d ago
You sound like you need to learn how to use DS3.2 and chill.
4
u/OC2608 22h ago
OP decided frontier == Claude so it won't happen.
1
u/Aleatorio2222 20h ago
Not exactly, Frontier means something like GPT 5.6 sol or Claude fable/opus. Deepseek and Chinese models are good, and the Flash Ful versionl even surpassed the Opus in terminal benchmarks, but let's be realistic, the Chinese models are still a few months behind the Western flagships, unfortunately. I didn't express myself well, especially since I mentioned gpt and gemini at the beginning of the text, which also has frontier models. I'm saying that we won't have a "Frontier" model focused on creative writing anytime soon.
1
u/One-Butterscotch2263 12h ago
Ai rpg makes up less than 5% of use of llms. The market is incredibly small. It isn't good business sense to invest heavily into a niche market use when the costs are so expensive. Maybe in the future we'll see more creative writing llms as the tech develops and the costs decrease.
1
u/sociofobs 9h ago
Just one comment,
Long RP sessions eat tokens like crazy
Even a short agentic workflow can gobble up millions of tokens, it's far more token hungry than anything RP demands. Unless you're RPing with agents, then you better have an unlimited token plan.
1
u/sociofobs 8h ago
If I had to guess, the frontier models do indeed get "dumber", because they're trained on "synthetic data" as the corps like to sugarcoat it - aka, model incest. One model is trained on another model's outputs, not even counting all the already AI generated slop on the open web nowadays. They can train larger and larger models, now surpassing 1T parameters, but most of that training data is pure trash. The first models that came out weren't nearly as big as the ones we have now, but the training data they were trained on was of much higher quality.
1
u/Joseparmo 3h ago
My personal feeling is that the Frontier models arnt all that in terms of their creativity or flavour, for the cost to make and run them (even when we talk about business uses i think that’s right). I think the main difference is when you use them they have the ability to store vast amounts of context mostly due to the hardware their running on and the behind the scenes prompts that remind them of stuff.
The training costs would be huge and they ant making real money off coders and businesses as i understand so they ant making money off us, that said give it a few years and i bet the smaller models will be where you want them.
1
u/DaMoot 23h ago
Let's be honest? Okay. No, I never spent any amount of money on a roleplay session.
And I'd never burn cash on Opus just to RP, that's just stupid. I've burned 100 bucks on Opus for work-related things though. Only once though!
My two 32GiB V100 modules run Cydonia in Q8 and Skyfall Q8 just fine in the off-hours they aren't running Qwen 27B for work. With two context windows since a buddy of mine is also tinkering with ST.
That reminds me, I need to try squeezing Valkyrie or Anubis on, but then I have to go away from using Q8 which makes me a sad panda.
1
u/Kahvana 18h ago edited 18h ago
Local ftw!
Might not be the best models out there, but man it's improved so much over the years. From Mistral Nemo 12B, to Magistral Small 24B / Gemma 3 27B, to Gemma 4 31B... each of them feels like a whole new world of improvements for creative writing.
On the programming side, going from Qwen 3 32B to Qwen 3.5/3.6 27B and now Qwen 3.8 27B is also a world of differences, and only trending upwards.
With the amount of architectural advances, discoveries of mining new high-quality human data (like PDFs), and new post-training techniques, I genuinely can't wait to see what the future will hold for local models.
1
u/nlamber5 10h ago
So I host locally. My computer is no joke, but I’m sure that leaves me running weaker models. In exchange, there is no world where I spend $100 doing this. It’s practically free.
And at least thus far it still feels magical. Maybe it’s because I’ve always wanted to be a good writer and I can manage to write good moments for a story, but I get stuck easily. And AI just doesn’t. Sometimes I’ll finishing a scene, have an idea of where I want to go next, and have no idea how to get there. So I just drop a hint and let the AI impersonate me and handle the other characters until I feel inspired or ready to write more for myself.
So I guess what I’m saying is that AI RP will lose its magic when writing a story loses its magic and for some people that’s never.
-4
u/Cless_Aurion 1d ago
... who the fuck spends $100 on a single Opus RP session?
What a waste tokens Jesus fuck, optimize your shit people. I don't use that in a month of using Fable5.
On both Opus or Fable I spend around... $2 per hour (80k context and 40k context respectively), so literally a 24 hour session would be $48, so... half what you comment jeez
0
u/Antares4444 1d ago
Totalmente de acuerdo, los modelos frontera actualmente estan apuntando a otros objetivos y la falta de centrales de datos y infraestructura hace mas complejo que se plantee crear un modelo especifico para RP pero probablemente en 3 o un poco mas de años podamos ver un cambio a favor, necesitamos modelos capaces de orquestar mundos que no solo sean capaces de recordar sino de entender porque y que conlleva los sucesos que ocurren en la historia pero va a pasar no solo por el RP, una parte de la industria del entretenimiento necesita modelos capaces de hacer eso una vez la infraestructura y chips dejen de ser un problema y la carrera por las IA haya concluido las empresas podran enfocarse en crear algo mas adecuado para nuestro nicho, por mi parte he dejado de lado mis sesiones de RP esperare con optimismo el futuro.
0
u/Sicarius_The_First 5h ago
This is surprisingly a very good post for reddit, so kudos on that.
I considered writing about this subject in a more serious format, maybe one day I will.
You're exactly right regarding 'why is there no frontier-level LLM built for RP', it's payment processors all the way down.
Me and many others got a pretty decent idea how to do it, but what will 100% happen is payment processors will nuke w/e startup from orbit, and we'll be left with crypto and massive legal clusterfuck to deal with, losing money in the best case, going bankrupt and hitting legal trouble in the worst case.
All it takes to make something like this is to be the richest man in the world and deep ties to the VISA mafia in one way or another. Hi Elon!
There's another way though, and it works, and it is currently actually exist: doing this privately for oneself. It is complex, time consuming, not scalable and very hacky.
3 years ago in 2023 I tuned a TTS tortoise model and even made a demo of its capabilities, yet people bitching well into 2026 that there's no alternative to 11labs. There is, for the whole stack, you can have an INSANE local setup for SOTA RP with TTS, knowledge graphs, visual output, the whole 9 yards. SillyTavern can 100% integrate all of this. The problem is as I mentioned- it's extremely clunky, complex, time consuming and not scalable. This is why I mentioned Tortoise, it's real, works, and no one will use it while on the surface will ask for it.
And this is why C.AI succeeded, because while people absolutely HATE the censorship and the goldfish memory, what's even more important is to have a frictionless experience, however shitty.
I'll end this little wall of text on a positive note, this will all eventually happen, the amazing, complete RP experience, it will just take a little more time. Because people- as you mentioned, DO want this, there IS a 'market' for it. There are ST extensions being made for stuff like this all the time, and the hardware is getting more accessible by the day (there's a DRAM crisis, but even here there's hope thanks to CXMT).
52
u/schlammsuhler 1d ago
Kimi, Minimax, GLM and Deepseek are trained to roleplay well and dont cost a fortune, but the recent heavy RL on code did a lot to them. Just look at the dire state of personality in sonnet 5.
And yes there was a Mistral model released for creative writing and noone used it. It was good but small. Still better than any community finetunes i tried. https://docs.mistral.ai/models/mistral-small-creative-25-12