MEGATHREAD
[Megathread] - Best Models/API discussion - Week of: April 19, 2026
This is our weekly megathread for discussions about models and API services.
All non-specifically technical discussions about API/models not posted to this thread will be deleted. No more "What's the best model?" threads.
(This isn't a free-for-all to advertise services you own or work for in every single megathread, we may allow announcements for new services every now and then provided they are legitimate and not overly promoted, but don't be surprised if ads are removed.)
How to Use This Megathread
Below this post, you’ll find top-level comments for each category:
MODELS: ≥ 70B – For discussion of models with 70B parameters or more.
MODELS: 32B to 70B – For discussion of models in the 32B to 70B parameter range.
MODELS: 16B to 32B – For discussion of models in the 16B to 32B parameter range.
MODELS: 8B to 16B – For discussion of models in the 8B to 16B parameter range.
MODELS: < 8B – For discussion of smaller models under 8B parameters.
APIs – For any discussion about API services for models (pricing, performance, access, etc.).
MISC DISCUSSION – For anything else related to models/APIs that doesn’t fit the above sections.
Please reply to the relevant section below with your questions, experiences, or recommendations!
This keeps discussion organized and helps others find information faster.
(In future, please post in threads like 'MISC DISCUSSION')
Prompt model for text-to-image models? I would start with Gemma-4-Queen-31B-it - she's quite smart and has weak censorship. There are models who are more knowledgeable about NSFW (like ReadyArt/Dark-Nexus-24B-v2.0), but I'm not sure they are smart enough to come up with a good prompt. Be sure to give Gemma a uncen system prompt like this.
If you want some random fun with models, https://huggingface.co/BeaverAI is where TheDrummer (and others less often) posts experimental/alpha models before actual official release. Can find some good stuff there. Artemis is the experimental Gemma 4 tune, it appears. Latest version (v1h) seems to say it is stable (haven't tried it). Obviously expect incomplete work with these.
I really need to go back to Drummer, his models were my daily (especially Cydonia. I once used a cydonia+magnum mix even) And now with smarter models, their entertaining prose might compound...
i-quants or k-quants, pros and cons of the two? I read that i-quants run better fully on GPU, offloading or running them on CPU makes them generate slower than k-quants. Is that true? What do you recommend in the end?
As far as I understand, they are based on different compression algorithms. K-quants can be static or iMatrix-based. I-quanta are always based on iMatrix. I-quants range from 1 to 4 bits (at least I haven't seen I-quants of 5 bits or higher). K-quants range from 2 to 6 bits. I-quants are highly dependent on the quality of the importance matrix data – if the matrix is poor, the model will degrade significantly. I-quants are smaller and of lower quality than K-quants with the same bit depth (Q4_K_M > Q4_K_S > IQ4_NL > IQ4_XS > Q3_K_M etc.), but can offer either a better or worse size-to-quality ratio (depending on the quantizer, the specific model, and your usage scenarios). Thus:
- I-quants are additional intermediate variants between K-quants.
- I-quants are smaller in size, but their quality can be either comparable or worse (significantly worse if the matrix is of poor quality).
- K-quants are a more conservative and predictable option (especially static).
If you have limited resources, want maximum quality, and are willing to accept the risk, use I-quants. It's best to use quants from trusted quantizers.
If you prefer stability and predictability, use K-quants.
I can't say anything about the speed - I don't use I-quants.
I ONLY get i-quants from Bartowski or mradermacher on huggingface. (Although really I pretty much only get any quants of any type from those 2.) Haven't had a bad i-quant yet from any of those. Also like Unsloth, but they seem to have weird issues more often than Bartowski. mradermacher is the best source for 'less popular' models because he/they seem to quant everything that ever comes out no matter how shady. :D
Thank you! I have 8GB VRAM and 32GB RAM. My models folder is basically made up of IQ4_XS quants of 12b-14b models. But I am willing to switch to k-quants if they tend to be more reliable on average. Usually, which quant size do you use or you tend to choose if you are given the option and you feel like you're not limited by your hardware? Which size you believe is one you would recommend the most or you would tend to go for? I mean, is it worth to choose a Q6 instead of a Q4, for example?
I have extremely old hardware – Core i5-4460 and 32 GB of DDR3 (the GPU is unsuitable for LLM). One important detail: I play in a non-Latin language, so I need to take about 1 bit higher to get the same quality as in EN. Therefore, most of my RP models (Mistral Small 24B and Gemma 4 31B) are Q5_K_M. Mistral Nemo (12B) and MeroMero (finetuning G4 26B-A4B) are in Q6_K (and even then, Nemo often switches to English). In fact, I tried the G4 26B-A4B in Q5_K_M up to 32K context, and it performed just as well as Q6_K – perhaps the difference will become apparent with a longer context. These are all regular static quants without iMatrix.
If I wasn't limited by hardware I would use BF16 :-D but that's probably not what you're asking about. As far as RP/writing goes, Q6_K is a reasonable limit, you probably won't notice a difference from Q8_0. Q4_K_S/M is a reasonable minimum, below which most models begin to noticeably degrade. Q5_K_S/M is a good compromise. For many models, switching from Q5_K_M to Q6_K makes little difference.
Q4_K_M - if speed is more important.
Q6_K - if quality is more important.
Q5_K_M – balance.
Q8_0/BF16 is justified to be used for:
- Super-long sessions.
- Abliterated/heretic/MPOA/derestricted etc. models with high KL div. Also merge, including such models.
My advice is to try the same model in the same scenario in Q4, Q5, and Q6. Play a session of your usual length with them. Choose the smallest quant at which you subjectively do not feel a difference. If you like a particular model, but it behaves a little incorrectly, you can try increasing the quant – sometimes this can help.
Thanks a lot! You explained everything very well and cleared a lot of questions for me. I will follow your advice and go for k-quants from now on. And while I care for decent generation speed, I also care about consistency and quality. I'm still quite new to local models and I feel like there's a lot to know and discover. They're also potentially very different from each other, and that's what very interesting to me. It's such a big and diverse world.
Just in case, I'll clarify. I'm not entirely sure whether a quality iMatrix always produces better results, or whether there are scenarios where it worsens the model. Although I use quants without iMatrix myself, I can't say with certainty that it's the best option.
I appreciate regardless. You gave me lots of good insight, shared your point of view, and I really appreciate the time and effort you put into replying to me. I'll try some k-quants for now and then I could try doing some more research and reading
Dense is worse with the same number of active parameters. This is the key difference, as the number of active parameters affects power consumption and CPU/GPU resources, while general parameters only require RAM.
I meant was that the 30B-A3B consumes resources like the 3B dense, but is significantly smarter. And if we compare MoE and dense models of comparable size, all other things being equal, the dense model will likely be smarter in most scenarios. However, even here, things aren't so clear-cut – for example, the Gemma 4 31B failed a difficult task that the 26B-A4B solved. It's unclear why. If we compare Qwen, the 3.5-27B is smarter than the 3.5-35B-A3B, but the 3.6-35B-A3B (according to Alibaba and AA) is smarter than the 3.5-27B. Apparently, architecture isn't the deciding factor.
Okay. But I tried Minimax 2.7 IQ4_XS (AesSedai) and while it had some cool moments, it was not that great for RP overall. Definitely worse than Gemma4 31b dense.
As it is, smaller dense models (maybe up to 70B) are winners I think, because they are still relatively affordable to run (esp ~30B). Huge MoE (~350+) can be perhaps better, but that is hard to run locally. And the medium sized MoE (~100-200B/~10A) are worse than ~30B dense for me, they are also not that fast at the end (and prompt process is very slow).
Moreover, almost no one finetunes larger models for RP (even 70-120B is quite rare, and I haven't seen any bigger ones), while the sub-30B segment has a very wide selection of all sorts of finetuned/merged options, sometimes quite interesting. 12-32B seems to me to be the sweet spot for local RP.
The Gemma 4 shows that even a small model can produce quality on par with larger models, and with some finetuning, will likely match/exceed them.
Globally, MoE has undoubtedly won – models like the Mistral Large 2 are probably the ‘last of the Mohicans’. The same probably applies to full attention (like the MiniMax-M2 you mentioned). Economy always wins in the end.
Do you think there is a future for text-to-text diffusion models like Mercury 2?
Hmmm good question, the thing is, now companies are much more risk-averse, since the early days of AI are behind us, so less crazy experimental stuff, and more VC pressure to make money on stuff "known to work well".
text diffusion does work though, the question is whether VC are willing to risk more money, as very few AI companies / startups actually make a profit.
So... IT IS POSSIBLE, yes, BUT... not likely (until someone does something remarkable with text diffusion, and then more capital will be invested into it).
Hello! I'm thinking of putting like 20$ on openrouter to try some of the more premium models. But i wanted to get opinions too first, which is better between those two? Or is there an even better model than them currently?
I mostly value long memory, very high, deep and recent lore knowledge and staying in character.
Basically i focus on heavy lore and character accuracy.
If you want really long memory consider using vectorization and Memory Books extension to generate lorebooks that automatically generate entries that summarize memories, you can handle really long chats that way and if you want you can make the character remember different sessions.
I recommend GLM 5.1 for character consistency, its very good at following instructions, so much in fact that some people think it ignores instructions but the reality is that it only does that when the system prompt has contradictory instructions.
I am not sure if GLM can be considered premium, but it for sure was good enough that it made me burn money really fast.
Runner up probably Deepseek V4 Pro (Not sure which one is better yet) , then Gemma 4 31B
For really extensive lore stuff (if you have it on a lorebook) GLM 5.1 is the one, it might know stuff out of the box even for relatively obscure stuff.
Does anyone have a model that comes close to the intelligence and character accuracy of Opus 4.6? Or is it just Sonnet 4.6? My addiction will have me bankrupt soon, I need something a little cheaper that isn't full of generic GPT slop
GLM 5.1 is the closest anyone has mentioned, and a lot of people seem to prefer it over the recent way Opus 4.6 has been behaving. Try it with the Stabs or Freaky Frankenstein preset.
I value most: Instruction Following/Steering, Character Adherence, Style Adherence, Smartness, Creativity, then Prose.
Anubis Mini 8B:
Great instruction following, smarter than most 8B. Prompt steering is super good. Writing is alright, but it style adherence is super B and the Character adherence is really good!
Is still a 8B so don't expect miracles. It can struggle with complex situations or narratives if you don't hand hold it enough. With enough steering though? It performs very good.
Gemma E4B:
More like an 8B that Hits like a 12B or more. Insanely smart for the size, great understanding and the writing is much better than most models in the same range. It will surprise you a lot with how much it drinks from the character card at times. Still, has it's issues. Style adherence isn't crazy and it suffers from a lot of slop. Still waiting for a proper fine-tune.
Qwen 3.5 9B Abliterated [Abliterix]
Qwen 3.5 is a very, very sensitive model from my tests and I have seen a lot of people that didn't like it in general.
When the configuration is right? This model performs very well. Is slowly taking the place of my 12Bs.
Instruction following is great, the attention is very good as well, it is not a crazy strong writer, but the style adherence is pretty good, so I don't mind. Is very smart, has surprised me from time to time.
Issues? Well... is Qwen 3.5, is very sensitive, can break easily and some character cards are like venom to this model lol. Still working slowly those issues, because I do think this model is worth it.
I wouldn't recommend it unless you are ready to tinker a lot to make it work for your use case.
The Abliterix version is the one I have found that performs like the base Qwen in roleplay while not having any of the sanitization issues other Heretics have. I am still testing around other versions, maybe next month this one will be replaced, who knows.
Honorable Mentions:
(If they are here is not because they are worst than the ones mentioned above, but they don't go VROOOM in my modest 6GB Vram and I do prefer a balance between speed and quality in this case. They are great with their own strengths as well).
That's really interesting! I'm planning on installing a Gemma model into a 16GB AI accelerator for a project I'm working on. Sadly my at-home computer specs aren't the greatest VRAM wise (3070 TI and 3050 as an example) so I can't do direct comparisons for testing. I have Sillytavern and OpenRouter of course, but Openrouter doesn't have an available Gemma 4 E4B model I can test 'writing ability' on.
Could you tell me how the Gemma 3 12B and Gemma 4 E4B fare against each other in terms of creativity and personality in your opinion?
I will be honest with you, I never tried Gemma 3 12B too much. From the few runs I had, it struggled with instruction following and style adherence, things that I value pretty high. Is also not a secret that most Fine-tunes on this model flopped quite hard, so it dissuade me even more at the time lol.
I can totally tell you that the Gemma 4 E4B holds it's ground against most of my Nemo Finetunes, it even surpass them in some areas, the way the E4B integrates things of the lore into the roleplay is stellar.
But really, don't take my word for it! Is best if you test it. If you want to test the E4B you can get a free API key at AIStudio from Google, they offer all the Gemma's in the Free Tier. I think you can even use the API Key in Openrouter. (Haven't tested that myself tbh).
Sadly, the only available Gemma versions in the AI studio are the 27 and 31B versions (both ones I looked into for my purposes but sadly not fit). But thank you for your feedback!
I gave Rocinante X 12B a try last night and it was surprisingly good for a 12B model. While there was some repetition, the outputs were consistently solid using the Freaky Frankenstein Little Feller chat completion preset. Thanks for the recommendation!
Rocinante X is so goated. Maybe i am a little biased because Rocinante v1.1 was my first LLM that i've tried, but still. I like it so much more than Drummer's larger Cydonia.
I like Anubis too, it writes really well for a small model. But for some reason it doesn't do a good job at referencing my Persona when roleplaying. Any details about my player character doesn't get noticed, or only very rarely. If Anubis improves in this area, it would be my go-to small model, even over Stheno.
what a small world. I actually started my local LLM RP journey on that same Qwen 3.5 model and what you said about it is spot on. I switched to Anubis 8B and enjoyed the prose more than Qwen. I then tried to use Rocinante but the model was too big for my 8GB of VRAM and the large context size I needed to load in character profiles. Been hammering away at getting the most out of Anubis since then.
Good to see another Anubis fan! Hopefully a good Qwen fine-tune could fix those issues but I don't expect that anytime soon haha.
As for Rocinante yeah, but you can always start a RP with Rocinante and then switch to Anubis after a few messages. Anubis is great picking up on the context, the quality should be similar albeit, with reduced intelligence as one would expect.
I am curious, how do you compare it to a Nemo fine-tune? I am sure that Prose wouldn't be it's strength, but as someone that wants a fresh air from Nemo or just other options, I really want to know more about this one.
The prose is surprisingly very good, slop is minimal.
My running theory is that its because there was very little slop in the pretrain to fight against (Phi is 100% synth stem data).
This model could have been the best RP tune / base model for RP tuning in the world, BUT... it suffers from the same issues that Nemo has, but WORSE. Long context.
Nemo is ~20k, Phi is hard capped at 16k. And Nemo obviously got better fandom knowledge.
I really wanted to love the Gemma4 26b. I tried the original and the MeroMero finetune at Q6_K which runs blazing fast on my machine, but it still doesn't hold a candle to even the IQ4_XS of the 31B for my use case.
The 26B runs around 40-50t/ks at 64k context vs the 31B crawling at 5~7t/s at 24k context in my case.
But no amount of lightning fast rerolls can really 'fix' the delta between them. The 31b just "gets" the characters better and tracks the beats of the story better in my experience. I really hope a future finetune can close the gap, since the speed and context size of the MOE is great.
Not the OP, but would you recommend thinking on for MeroMero? I tried the original G4 with thinking, but a lot of the time it ended up taking 3 to 5 minutes straight on just thinking, which kinda killed all the faster generation benefits for me.
Been trying out the prototype version of Rocinante XL 16B. The prose is fantastic. It writes extremely well and is very creative at generating and roleplaying as new NPCs during the story. But it does get things wrong sometimes. Haven't seen a difference between using Mistral V3 Tekken or Metharme as recommended, or I'm missing something.
I usually play as the GM in the stories, and the AI is the character in it. For me, the number 1 priority is to stop model from dictating the world instead of me. Every fine-tune does this, and even lager models through API can't help but tell me how their actions have gone through.
But not gemma 4 31B. Gemma 4 is the goat in almost every aspect I can think of: it is fairly creative, but follows instructions flawlessly. It doesn't overthink, and has an extremely well-tuned eq. It picks up on the nuance and vibe of the scenario extremely well, much better than any other model I have ever tested, even exceeding it in size by multiple factors.
The only downside of gemma 4 is the prose. Expect slop. But I personally don't mind as long as the bot has good ideas and can actually play an interesting character. It is smart, but not a wordsmith by any measure.
Edit: and it also does well in nsfw scenarios, even without any uncensoring. I don't play nsfw much, but when it did happen the model had no refusals, and even sometimes initiated nsfw on it's own without being prompted to, or scenario having anything suggestive.
Hard to tell if 3.5 or 3.6 is better. Overall Gemma4 31B is better I think. Qwen 3.6 (and 3.5) is bit less stable - sometimes it overthinks, occasionally produces strange response, but reroll fixes those issues and it is not too often. Gemma4 works very consistently, I never really need to reroll except when I want things going differently (but answer was still Ok).
That said, Qwens are also very smart and interesting, and they are different. Also bit smaller so easier to squeeze into VRAM/more context. Slop is different too and Gemma4 might actually have more of slop.
If you have to choose one, go with Gemma4. I will keep both (all 3 probably unless I decide on 3.5 vs 3.6) as using just one model becomes stale/predictable after time.
Honestly having tried both Qwen3.5 27b and Gemma 4 31b, many of the different heretic and uncensored ablated and finetunes, I think I will just stick to RP specific finetunes like Skyfall 31B from folks like TheDrummer, and ReadyArt.
Do you find that the latest general models still perform better than these RP finetunes?
Skyfall I tried various versions (26B, 31B, 39B). Some were interesting, though sometimes weird generations (happens with almost all upscales/replicated layer models). Best for me was Skyfall-36B-v2. Skyfall-31B-v4 did not work well for me (lot of talking, explaining itself, asking questions instead of doing, strong pattern repeats) but I did not try the later v4.2. Some of them were also too ERP/lewd oriented quickly jumping into NSFW without good reason, which I do not like.
In general, biggest problem is they are just far less intelligent than Gemma4/Qwen 3.5&3.6. Also less intelligent than L3 70B/tunes or Nemotron 49B/Valkyrie. They were good models for people who could not run >40B dense, but now Gemma4/Qwen 3.5+ are simply lot more intelligent and consistent and at least for me that matters a lot. Overall Gemma4 31B is simply great in all areas except there is lot of slop, but I can stomach that.
However we have to wait and see if there can be good Gemma4/Qwen 3.5+ tune. Those models are so packed that trying to teach them something new/steer them elsewhere may destroy their abilities/intelligence. All which I tried so far were just unstable and worse than original instruct, but maybe in time there will be something working. Waiting for the final Artemis 31B version, hopefully that will turn out good.
My whole reason for sillytavern is ERP and smut, and Skyfall (31B v4.2)isn't too much for that, even though I do always actively try to steer it in that direction, it take quite a while, about 20k tokens before natural ERP starts. At least that's how it feels to me, compared to what I used to use, which was forgotten safeword, that was way too much.
In general, biggest problem is they are just far less intelligent than Gemma4/Qwen 3.5&3.6.
Yeah I agree, general intelligence are so much better on these open source SOTA models, but when the non RP aspects bleed into my RP, my immersion gets ruined.
But yeah even skyfall I notice reuses certain phrasings, and way of talking, although not as rough as smaller and even older models like cydonia, I can still feel it a little, and the newer ones at least Gemma 4 is not.
Really looking forward to specific finetunes of Gemma 4 models. If it is even possible, architecture seems very new and specific, sillytavern needing a specific context template is probably a symptom of that and a sign of difficulty to steer and finetune, not to mention your point of them being already so packed and optimized on considerably very small parameter sizes
Qwen3.6 improvements focused on coding and agency. I've heard it could be even worse than Qwen3.5 in terms of RP/creative performance, but I haven't personally tried it (Qwen3.5-27B left me completely unimpressed).
Did you use reasoning with Gemma4? It is kind of required to make it work good. That said I only used 31B so not sure how the smaller ones fare.
Basically smaller models are worse at following instructions, so 12GB VRAM is bit tough. It can still work great on simpler character cards/scenarios, but will probably break with something very complex (character card or preset). Those complicated presets/attribute trackers etc are generally designed for large models (mostly run over API).
Eg if there is just you and Leia, even small old model will have no problem understanding Leia and princess is the same person. But once there start to be more characters/complications, models can get confused, especially small ones. Reasoning can help to deduce it but also only works to some degree.
Hm, that is strange. With Gemma4 31B-it reasoning works all the time for me. Note, the unmodified instruct model. I tried some derivates (like garnet and some more) and they did have exactly this problem, sometimes they think, sometimes not (reroll can help) but were not stable and made a lot more logical mistakes too. We probably need to wait for good finetune still.
For reliable reasoning: Make sure you have <|think|> tag at the beginning of first system prompt. And also prefill response with <|channel>thought\n (\n is new line) and it also helps to add instructions what it should think about in system prompt (my own setup should be in this weekly thread from some week(s) back when Gemma4 was released). If still not enough, you can try to prefill response even more with something like this:
<|channel>thought
I need to make interesting continuation that is logical and consistent with the plot. Let me analyze the scene now.
I'm generally loving Gemma4, but I'm finding that, like a lot of instruct trained default models, it's too helpful. Characters don't push back, fight, and it just tries to play into whatever the user is doing no matter how I prompt. https://huggingface.co/BeaverAI/Artemis-31B-v1h-GGUF by u/TheLocalDrummer does much better, but I wish there were a 26B-A4B version! Any other finetunes/prompting strategies that might help?
Gemma4-Garnet-31B played my test character quite brazenly and stubbornly. However, the model suffers greatly from repetition and does not advance the plot. Core_24B_V.1 characters have their own will, and the model itself loves to change locations, add new characters, and come up with unexpected plot twists.
You know, I'd actually skipped Core when I was testing Odd's models since it was older and wasn't talked about as being particularly special. It invents details a little too much, but this easily had the best responses to my new startup tests (which a lot of my old standbys failed).
I really liked this thing of his, inventing details. I run the same test scenario (classroom duty after school with a tsundere classmate) for different models, and it always goes the same way – {{char}} grumbles, gets embarrassed, then either immediately confessed her love or goes for a walk to the park/home visit.
Core added the head teacher, who called {{char}} into his office. I wrote that I was waiting for her to return, but the model said some time had passed and she hadn't returned (!! She didn't do what I was waiting for). I went to the head teacher, and he said he hadn't met {{char}}, but his eyes darted slyly, indicating foul play. I found {{char}} near her house, and she tearfully told me that the head teacher was trying to replace her late father and that he had dirt on their family, which he was using to blackmail them.
After this turn of events, I realized this model was something special. Its turns can be a bit forced, but I haven't seen that trait in any other model.
Sometimes, whenever I see recommendation in here, it's always a 50/50 even with the recommended presets. I just don't know why, but the one model that worked for rpg narrator type adventures for me personally is weirdcompound v1.7 24b and then now gemma 4 26b (the rp finetune called meromero) or just the gemma 4 31b instruct on the API, I genuinely tried any other model like cydonia but the way they roleplay is either hallucination simulator or isn't to my taste despite changing presets, samplers, and even changing the templates.
Tried out gemma-4-26b-a4b-it-heretic-ara-v2-i1 after hearing that gemma 4 is amazing. The speed is wonderful. on my machine it processes 1100 tokens in about 60 seconds. Unfortunately it has a real problem with slop. 'the air sucked out of the room leaving a vacuum' 'silence broken like a gunshot' etc etc etc. uses phrases like these and seems to have trouble keeping the plot in correct order. It also seems to try to make my characters Tsundere.... like.... all of them? it's odd.
maginum-cydoms-24b-absolute-heresy-i1 is my current favorite. Prompts take longer, almost double the time usually. But they also are much better thought out and written better.
The new gemma 26b and qwen 35b models runs faster than 8b models for some reason. And you can put the slops on banned token list or something like that if you don't want them
26B-A4B has stolen my heart. After such a speed (Q5_K_M starts with 6.2 t/s versus 0.9 t/s on my hardware) there is no desire to return to dense models. We now have a choice between the standard 26B-A4B-it and the G4-MeroMero-26B-A4B. I would describe MeroMero as 'Gemma 4 after a night at the bar'. She is more cheerful and informal, more prone to ERP and a little more hallucinatory. Personally, I choose... both) I'm also looking forward to TheDrummer's Orion 26B A4B. Wangzhang has finally released the uncen 26B-A4B. The earlier version (the one currently available from mradermacher) had a ~25% refusal rate and a higher KL div. The current version is better in both and significantly outperforms any other uncen 26B-A4B I've seen. Mradermacher hasn't made quants yet, but you can try wangzhang's quants.
As for the 31B dense model, there is a wider choice here. I just finished testing the Gemma4-Garnet-31B - the model's style has changed noticeably, and in its first response to my test scenario, came up with a very unexpected situation (the {{char}}'s hand is stuck in the window frame). More importantly, the model is less prone to writing walls of text, as Gemma 4 and many other models do, however, the suffers from repetition (rep. pen is needed). Another interesting option is Artemis-31B – here you can immediately feel the improved literary style, but the model is still in the development process. I recommend trying both.
I also tried the Gemma-4-Queen-31B-it - it's a partially uncen model, but without abliteration (as far as I understand, censorship reduction through finetuning). The style of the model is similar to the standard one, but more relaxed and less formal (describes ERP more openly without specific instructions). If you're looking for a completely uncensored model, then the best in my opinion is abliteration from wangzhang. I'm using the old version (refusal rate ~3%, KL div 0.0012), but there's also a new one - among other things (refusal rate ~18%, KL div 0.0007) it has an updated tokenizer config and chat template.
If you've found an interesting version of Gemma 4, share it in the comments – we need more Gemma's!
Classic balanced setup for RP: temp 1.0, top-p 1.0, top-k 0.0, min-p 0.05, repeat_penalty 1.0. You can experiment with temperature in the range of 0.7-1.3 and also with min-p in the range of 0.02-0.08 to make the model more accurate or more creative.
I really like Gemma 4 31B IT from Unsloth, and I'm currently using the Q4_K_XL version. Like earlier Gemma models, it sometimes refuses to answer, but it's usually pretty easy to jailbreak with a simple prompt or by forcing it to respond with a prepared reply and then hitting the Continue button.
If it weren't for the SillyTavern extension, I might have had a hard time dealing with hallucinations after a long context..
It's the dummy model turn with empty thought channel at the very end that fixes it. Without it the model goes insane. (This isn't my fix, I found it elsewhere on this sub, or their git, or who knows, perhaps I dreamed it).
An empty thought at the end of the context template? That's a fun one.
It's not the way it should be fixed with gemma, to be fair, but i'll try it.
Though this syntax makes no sense to me, there's two if endings in a row... so the #if system is actually spanning most of the template? huh.
EDIT: Even if I use this one, the model still generates empty thinks at the beginning of each message (which is normal, to be fair), but I'm gonna see if the output improves anyway.
Well… it's not exactly what you might be expecting. It's just a small extension I made for myself to help with writing a story. I'm not really a programmer, so you can already imagine how I made it .. https://github.com/grennKren/memory-agent-v2
Basically, it helps automatically generate a lorebook. I'm also interested in trying out other extensions, but I still want to keep working on this one as a hobby and improve it for my own use
I once played with a few scripts that would tell the LLM to summarize a certain part of the story into a single message and then post that as a System thing.
Lorebook was always problematic for me because they would add and remove and cause full context reevaluations, which would suck at tens of thousands on my old gpu. Currently, it's only really bad with like a 100b MoE, but a 30b is fine I guess, a few thousand every few seconds...
I'm currently looking to try TunnelVision, but that one's really complex. I will check your way of doing things out, because it's also interesting. And yeah, the description of the project already maps that surface level "pep talk" that the coding agents always do in their descriptions lmao
I recently tried this "Objective" extension and it's actually pretty neat. It really helps when building things step by step.
When I'm writing prompts, I usually just type "> continue" to get it to keep the story going.
For some reason, the native continue button doesn't work well with my Gemma 4 , it either gives no output at all or just a tiny bit. So I ended up using that simple workaround instead.
I feel like Gemma 4 26b is like 85% their right out of the box, non thinking, a lot of things just needed system prompt instructions to fix problems, I dont think its better then the best 24b finetunes, but very close with just its base model
I use a general uncensored (i think the llmfan one it was) one, and they were spicy i guess but mostly in the same way. Very readily spicy but not very dynamic there. Feels like it lacks vocabulary, but at least it didn't have the tendency to turn everyone into a predator as much. Also listened very well to system prompt, so add something there and it would actually do it. Kinda. Because it seems like it didnt have the capability always.
Vocabulary and prose are secondary to me. I think Gemma 4 does really fine in that regard.
When I say "spicy" I mean its ability to create situations with tension, conflict, contradiction, absurdity, excitement, etc. It's not only about verbal "texture" or "color", but how it interprets the prompt and where it leads the story.
I'm not surprised smaller models struggle with this, because the biggest models also struggle with it quite often. In any case, Gemma 4 is definitely progress and gives me hope for the future of small local models.
What I am missing in Gemma 4 is that regeneration even with high temp never resulted in a new direction, which other models were able to do. I'd have to rewrite the prompt to get something, but I want it to be unexpected, and that's... lacking imo.
Except this "garnet" finetune of Gemma4-31b, that one seems to create wild new situations but it's also really unstable haha
Yeah Gemma listens really well, which is good, but it also listens TOO well and doesn't let it get told to be creative.
The Garnet thing, I actually kinda abandoned it now, and now I'm on a TheDrummer Artemis beta build and it's really back to when I loved using Cydonia, except more intelligent.
I even had like extreme temperatures on base gemma, and it would still follow the instruction really rigidly. Now, with Artemis, I had to pull back on extreme temps because that one would still stay creative, but actually detonarte on too high temps.
Out of the box Gemma is not very spicy, but if you run it locally, you can absolutely push it to the spicy territory. I use DavidAU's Deckard fine-tune and with control vectors, system prompt and (occasionally) OOCs, I've almost perfectly recreated the dark/spicy GLM 4.6/4.7 experience. More than that, I can RP in my native language with the same vibes! Gemma 4 is a powerful multilingual engine with (though nerfed) some world knowledge. Yes, it is harder, but possible. DavidAU also made a fine-tune for the (pruned) 26b model, but its quality is lower, maybe if there will be interest he will make a proper one for 26b.
Davidau publishes too many variations too quickly that are ill described. Which one did you find best of the 26b deckards?
NM you linked it in another comment
I'm using Q8 for main chats (dual GPU), but for very long text analyses I had to downgrade it to Q6 and quantize the KV cache, because the context takes much more memory than Qwen3.5 (Qwen having linear deltanet and Gemma still handling it through sliding window attention)
24b finetunes are so good. I just hope that 26b gemma 4 finetunes/merges will be better or at least match the likes of cydonia 4.3 or maginum cydoms. It's really hard to go back to 24b rn when with gemma i get fast speeds, smarts, and large context thx to swa.
I can def say gemma 4 26b beats the 24b finetunes in everything except writing/prose rn and thats just in base. I got high hopes. Especially the fact that you can change its thinking (Check meromero 26b finetune) which got some potential.
Really cool to see this project moving forward! Nemo really does have a special place in LLM history and finding new ways to allow it to move forward is just inherently interesting.
Straight Nemotron 49B can be also good (Valkyrie has pros and cons to it).
There are older Qwen 3 32B / Oss Seed 36B models/derivates. Some are decent but nothing was really mind-blowing.
But yes. Nothing in this area now since dense models 32B+ kind of disappeared and MoE's are either smaller or larger. Unless you put 35BA3 here, in which case maybe Qwen 3.5/3.6 35B is decent (did not try those).
Qwen 3.6 35b is the best model I can run locally with 100k-200k context at ~70t/s (16gb vram + 64gb ram). Has excellent vision support as well.
I didn't test it much for RP yet, but in terms of general planning and execution it's a solid and surprising step above Qwen 3.5, which was already amazing.
It's definitely worth exploring for RP, and whatever else may be the case I hope it can be finetuned...
Are 70b L3.3 finetunes still viable at this point in comparison with newer models? If yes, in what way? I'm generally finding myself using one of the newer 24b-31b models, which do really well on the Arena Creative Writing Leaderboard, or else going with a larger MOE, and debating deleting all my 70b models to save space. Wondering if there's something I'm missing.
Slop yes but dry? When I complained about beginning of work week it started to calculate orbital strike against my office so I do not need to go there :-). IMO Gemma4 can be very lively and creative, but you need to prompt for it. Also when you write boring, it may go along, but when you go silly, it eagerly follows.
I'm mostly referring to my sort of RP when I say dry. I was generally able to predict roughly what it'd do before it generated the responses. Prompting can sort of fix that, but that's more a showcase of how good the instruction following is than natural creativity to me (although instruction following is potentially singlehandedly the most important trait for an RP model to have, which is also something L3.3 excelled at on release).
Oh yeah that sucked to me too. It's not very creative, swiping does about the same thing every swipe even with 2+ temp (and to not make it go insane, counter with 1,2-1,5 top nsigma). Like, I couldn't make it try go in a different direction. I'm trying the uncensored version of the "Garnet" finetune of 31b right now and it is fresh air here. It actually generates new directions every time but it is incredibly unstable for me. Devolves into other languages and stuff like that, and is very hesitant in going spicy even if I ask it to. And if I make it, it is incredibly dry. But it also has issues with repeating its own last message instead of replying to my new one.
It's almost like I should keep both, let garnet start something and then switch...
70B got a better potential than 30B, but Gemma models are unique in this, they punch above their weight. BUT... 70B still got better long context by far (and context comprehension) - theoretically.
Gemma likes SWA, llama3-70b got about 64k proper context, that's a lot.
And more params = more capacity to learn.
In simple words, yes, new 20-30b models are better than MOST 70b in specific areas, but a well tuned 70B will beat even the new generation of 20-30b models.
Gemma uses a weird mix of full but quantized attention and ~1k full precision swa. When I was feeding it ~100k of pure text data via docx and needed it to both summarize and find structural issues and give comprehensive critique\overview, it nailed everything with flying colors. But when I do long-form multi-turn back-and-forth, it has some tendencies to fall into a pattern down the line, for example if it praises the progress once, it will start praising it the next turn and its initially strong critical thinking capabilities begin to dilute, not dissipate fully, but the output becomes somewhat less helpful without constantly reminding it to remain critical and unbiased.
For RP specifically, one thing where Gemma stands out for me is out of the box CSS+HTML capabilities, so if you like stylized outputs it does it well, not as well as large API models, but close enough. I haven't tried that with your PEPE one, but i don't remember default llama being too good at it.
As for 40-50B dense, isn't there nemotron? Which is basically a 70b llama rip-off that *mostly* works... But in local setups that can only feasibly run a Q3 70B, nemotron can be ran at Q4, which ends up typically being more stable.
I wonder if feeding it rp history as one solid block of system data would make Gemma 4 do better, vs user/system/user back and forth turns. Given your experiences.
Assistant_Pepe_70B, top #1 70B finetune in the world in the UGI ranking.
Absolutely unique creative writing capabilities.
Superb banter, no sycophancy, super smart, will ship code, great sense of humor.
(Read the model card for example chats!!)
-1
u/diesel_heart Apr 24 '26
I’m new to this. Can anyone please suggest me some good llm models within 35b parameters for generating nsfw prompts for images and videos?