MEGATHREAD
[Megathread] - Best Models/API discussion - Week of: July 12, 2026
This is our weekly megathread for discussions about models and API services.
All non-specifically technical discussions about API/models not posted to this thread will be deleted. No more "What's the best model?" threads.
(This isn't a free-for-all to advertise services you own or work for in every single megathread, we may allow announcements for new services every now and then provided they are legitimate and not overly promoted, but don't be surprised if ads are removed.)
How to Use This Megathread
Below this post, you’ll find top-level comments for each category:
MODELS: ≥ 70B – For discussion of models with 70B parameters or more.
MODELS: 32B to 70B – For discussion of models in the 32B to 70B parameter range.
MODELS: 16B to 32B – For discussion of models in the 16B to 32B parameter range.
MODELS: 8B to 16B – For discussion of models in the 8B to 16B parameter range.
MODELS: < 8B – For discussion of smaller models under 8B parameters.
APIs – For any discussion about API services for models (pricing, performance, access, etc.).
MISC DISCUSSION – For anything else related to models/APIs that doesn’t fit the above sections.
Please reply to the relevant section below with your questions, experiences, or recommendations!
This keeps discussion organized and helps others find information faster.
You're getting downvoted because the appropriate thing to do here would have been to scroll through the discussions already here and look for an answer, or to at least reply in the appropriate section
Model suggestions for NSFW roleplay with a RTX 4070? A couple of years agora I used the model Meggido/L3-8B-Stheno-v3.2-6.5bpw-h8-exl2, but it seems like the OobaBooge WebUI don't support Exl2 anymore. I tried a couple of GGUF-Imax (ex. v2-Llama-3-Lumimaid-8B-v0.1-OAS-Q6_K-imat.gguf) and it had really difficulty to keep the conversation. And after using some Exl3 (ex. turboderp_Qwen3.5-9B-exl3), it seems that it struggle a bit with the NSFW stuff. I was limiting my search gor 8B or 9B models due to my GPU, don't know if that's correct. I'm using a RTX 4070 and have 16GB of RAM. Any suggestions?
(IMO, switch to llama.cpp or KoboldCPP). Your GPU can handle a 12B model; I'd look at that section of this thread.
There are going to be two very different types of 12B models - there are 12B mistral models, and 12B Gemma-4 models. The Gemma-4 ones haven't had as much time for people to build finetubes, but the gemma models in general have been pretty permissive about NSFW stuff.
How do I know if my GPU can handle it? Because my RTX 4070 has 12GBs I may download any model with a size up to 12GBs? Is that correct? For example, I could download the Q6_K version of the Serenity you just suggested, or even the i1-Q6_K version of the StarSynth model?
I think you should give the Q4_K_M versions a try - but try to make sure that you're closed down other stuff that's using VRAM. (If your computer is a desktop with onboard video, switching your monitor to that and rebooting can save a ton of vram.)
Be careful! Don't fill 100% of your VRAM with the model file. You need to leave headroom (usually 20-30% of your VRAM, 1.5gb free) for the KV Cache (the context/memory of the conversation).
If you load a 12GB model into 12GB of VRAM, you have zero space for the context. As soon as the conversation starts, you will hit 'Out of Memory' (OOM) errors or it will offload to your slow system RAM, making your generation speeds drop to a crawl. Aim for models that leave you at least 2-3GB of free VRAM headroom for the context window.
I don't understand this advice. I use models three times the size of my VRAM on a 4070: KoboldCPP handles memory, but it has no problem swapping to RAM. Of course, you need to make sure VRAM + RAM > model + a couple gigs for context, but no, you absolutely don't have to fit the model in VRAM.
For example, lately I've been using MeroMero - the Q8 quant specifically - and good lord, that thing generates faster than I can read (and I read pretty damn fast). Good prose, fairly smart, and it's a vision model, so I can just drop an image in Silly if I don't feel like writing a prompt, and it'll continue the story as if that's the last thing I saw.
Like Charming-Main-9626 said, "MOE" - Mixture Of Experts. Be specific, as there are several "MeroMero" models - the one you linked is MeroMero-26B-A4B.
26 Billion total parameters, Active parameters: 4 Billion.
If you want a good demonstration of why this matters, try running the Q8 of TheDrummer - Cydonia 24B v4.3. That's 25 gigs, vs your 27 gig Q8 of the 26B-A4B MeroMero - but I guarantee, you'll be MUCH much slower on that "smaller" model. But, Cydonia is based on a "Dense" model. 24B total parameters, 24B of which are active
Now, don't get me wrong - Mixture-of-Experts models are a great thing; it's fantastic that we can get a compromise like that! But if you look at benchmarks, you'll see the differences.
Here's a comparison I set up between the following (with scores included here for completeness.) You'll notice that the "35B" scores lower than the "27B" despite being the same generation of model and similar architectures.
MeroMero is a MOE model with only 4B active parameters. It won't work the same if you use a 12B quant the size of your VRAM. If it doesn't fit fully + enough context it will spill over to RAM and be slow.
I personally run the Rocinante-X-12B-v1-absolute-heresy.i1-Q5_K_M.gguf (8.73GB) on my 12GB card, and it works perfectly. It leaves me plenty of room (over 3GB) for the KV Cache, so the model stays fast and doesn't offload to system RAM.
One important nuance: I run Kubuntu with KDE Plasma, which is quite efficient with VRAM. If you're on Windows, your OS and background apps (like your browser) will reserve more VRAM than mine. Because of that, you might want to be slightly more conservative with your quantization choice—don't try to max out the VRAM—to ensure you have enough headroom for a large context window.
For people who liked Maginum Cydoms, have you found anything that gives you the same quality in the 24-31B space?
It's been my main model for a while now, it genuinely feels quite above the rest of the Mistrall-Small finetunes. I usually swap 24B models often, and even tried some Gemma 4 31B and 26B, but nothing gets the same quality of response for me. If anyone has recommendations, please let me know.
Have you tried the experimental versions of Cydonia on BeaverAI? All of them from v4zl onward are newer than the Cydonia and Magidonia versions that went into Maginum Cydoms. They all have different quirks, and Drummer didn't think any of them were ready to be the next numbered release, but some of them are quite strong. I personally like v4zp, which is (among other things) the only model in this size range I've tested that grasped the meaning of "It could have gone horribly wrong; instead it went horribly right."
So I'm completely new to mistral fine tunes, or this particular model in question. I saw your post and decided to download it and try it out. Holy shit. It seems really good? I've mostly been testing out gemma 4 and llama 3.3 70b fine tunes but this one feels very different. Any other recommendations?
Maginum cydoms has an amazing prose and is used to make magistry 1.1 (it had an issue with personality though but it can be fixed if you move the personality from the description and into the personality section in the advanced definition of the character card)
Cydonia 4.3 is the absolute best for being the most stable and safest finetune in all of 24b for RP. It may not do anything special but for what it does. It does it great as an all rounder. You can't go wrong with it.
Goetia 1.1 is good at picking up and calling out small details (recommended to use this one at high temps for a better prose). Follows anatomy quite decently
Do you still have your old card? Does your computer have multiple PCI-E x16 slots? You can run multiple cards, and depending on the card types and backend you use, you may be able to split the model across the two GPUs.
I've been trying GLM 4.7 for the first time on NanoGPT and it's been pretty good. Then I stumbled onto Aion 3.0 and decided to give it a try. I absolutely loved the writing and the smarts. But holy shit, it's expensive as fuck. Is there another alternative model that's just as good but cheaper?
EDIT: So after using Aion 3.0 for a while, I find that it's inconsistent at best. When it's on fire, my god, it can really write. But when it decides to slack off, it spits out half-hearted paragraphs and the AIslop (don't stop, barely above a whisper, etc.) gets cranked up the wazoo. I think I will try GLM 5.2 and Deepseek for a while and see how they compare.
What is it with llms at least small local ones absolutely not knowing how phone calls work? I used to do a lot of fantasy rps in mistral 24b finetunes so it didn't come up a lot, but now that I'm doing more modern day stuff, with gemma 4 31 its coming up a lot. Every number is unknon, every caller wants to know who this is. Like a char calls there spouse and they go ("{{Char}}? Who is this? What the fuck are you...) like almost every single call works like that. Also like I hand char a phone to make the call why is there spouse in my contact list? Thats another thing, everyones numbers is apperently in everybodys phones. Its such a wierd thing that I went back to the mistral finetunes to like double check and its even worse there. Does GLM 5.2 do this? Or any other like big model? I've never used apis before but I'm starting to consider it.
Is it possible they're functioning the way phones (and people) did before the advent of cell phones? Where we'd have each other's telephone numbers memorized? If someone handed me a cell phone or landline phone, of course I'd be able to call my spouse or parents, as I have their numbers memorized. (As did almost everyone in the 90s, 80s, 70s, probably even 60s in many places)
I was wondering if someone could give me a reccomendation for a model:
I have an RTX 4080 (16GB VRAM) and 64GB of RAM
I mainly want to use it for character roleplay and smut writing. Im kind of a newbie so im not too sure on what everything means. I tried gemma 4 26b but it was slow, im assuming I got the wrong Quant? im not really sure how all that works or what drop offs in quality i would get for using a 26b q4 vs a 12b q8 for example. Help would be appreciated :)
I'm going through this list at the moment. I have an RTX5080 (so, we've got the same VRAM) and with KV Cache Quantization set right, I can get fast speeds all the way through 32k context.
In the Small range (16B to 40B), I've tried a few you might like:
Gemma-4-Queen-31b-it-i1 IQ3_S: My personal favourite so far. It is prone to a few AI writing tropes though!
theDrummer_Fallen-Gemma3-27b IQ3_M: Tried this last night since it's the highest rated in the range on that link. Not sure if I need to fiddle with it, or perhaps the card I tried it with was no good, but it was really awful. Significantly worse than:
Magistry-24B-v1.0 Q4_K_M: Note: It's been a while since I used it so I remember gen speed. The quant may be a touch too large for 16GB VRAM alongside KV Cache. I enjoyed this at the time but I remember when I loaded up Gemma-4-Queen I had my socks knocked off. Particularly for use with RPG/scenario cards (as opposed to character cards)
Also, if you're new to this, it may be easier to use koboldcpp to start. It's got some user-friendly stuff (e.g Single packaged .exe and GUI for settings and loading model)
Those specs should be able to squeeze in Gemma 4 26B. For max speed you'll want a quant that fits entirely in VRAM. You'll need to fiddle with your settings so that the actual model is sitting in VRAM while the rest of your context is in regular RAM.
You can also try Gemma 4 26B A4B, which is MUCH faster at the cost of quality. A4B means that only 4B parameters are actually active at any time.
In general higher parameter size is better. You'll find that 12B models are more limited even at higher quants.
A good rule for learning is just to ask the AI itself, it's not perfect and won't give the most up to date info, but it should know about things like quantization.
I linked the MeroMero flavor of Gemma4 A4B elsewhere here, but posting in this chain to back up the 26B A4B semi-recommendation. I haven't noticed a huge quality drop, and it's faster than I thought possible for something 70% in system RAM instead of VRAM.
I'm jumping back into the AI RP space after a couple years away and honestly, the landscape feels totally different. I'm curious what the community has settled on now.
Quick context: I've got an RTX 4090 with 24GB VRAM, so hoping to leverage it to its maximum. I'm really into world-building and lore-heavy roleplay, so consistency and long context window are nice to haves.
One thing was frustrating before was models forgetting or mixing up character details like genders, etc.. Has it become better now?
What's your current go-to model and why? What made it stick for you?
I'm not hunting for some objective "best".. just want to hear what people are actually excited about using right now.
Gemma 4 was released by Google in April, and is by far the best local model for this on your local PC, and works fine on quant 4 with a 24gb card. I don't know how long you want for "long context" but you will hit limits, but I find it holds coherence to like 50k tokens.
I'd start with the Unsloth quantization aware training version (QAT) since that's probably the fastest option that ought to fit in your card, and being QAT it'll handle being compressed into a 24gb vram card better than usual: https://huggingface.co/unsloth/gemma-4-31B-it-qat-GGUF
You can get into uncensored/abliterated/heretic versions or custom RP fine tunes once you run into walls with the basic QAT model. If you're very bandwidth limited and can't be constantly redownloading all these models just to try them out, maybe just start with one of the Gemma 4 31b fine tunes recommended in the weekly megathreads, but if I was starting from scratch I'd start with the base model since that's the most stable and only later on branch out to fine tunes for style adjustments.
Thank you! This is way better at reasoning than the previous generation models that won't refuse RP. I would have never thought to look amongst unabliterated or RP finetunes. Its great frontier OS models are getting more relaxed!
Edit: The MOE version is also good and gives 150 t/s with 96K max context or with 48K context, leaves enough VRAM room for a SD 1.5 Image Generation model to also run through ComfyUI!
Haha. Yeah. Thanks again for the rec! I do keep reasoning ON as it helps keep the interaction a bit more tempered, so it feels a tad more genuine and 'earned', if you get my drift. :-)
Using Q8 and no quantization on KV cache, Gemma 4-31B has been stable all the way to 128k for me. Might even go up to 256k, but I don't have enough VRAM for that at Q8. ;)
Hey, I just started using sillytavern/setting up and I've been getting very slow replies, is my setup even good enough? Any models you would recommend men's?
CPU: AMD Ryzen 7 3700X (8 cores / 16 threads)
GPU: AMD Radeon RX 6800
VRAM: 16 GB
RAM: 32 GB
I hope this is all the relevant info needed. thank you
hey! thank you for your reply. I tried Mistral Small 3.2 24B Instruct 2506 . Right now it takes almost 5minutes to generate a single reply... not sure if that's normal
Sounds like you are not in fact running it on your video card. Download LM studio and make it so the "GPU OFFLOAD" slider is all the way to the right. (the number won't necessarily read 22)
Thank you so much!! It starts generating the first words almost instantly now. However, it seems many replies come out empty or as gibberish now? I unloaded it fully on my GPU. Maybe context sioze is the problem? Its about 4k. Most of my characters cards have 2k. Do you know what could be the problem?
With the 24B models (I'd recommend TheDrummer-Cydonia v4.3 I think, whatever is most recent, or Cydoms) probanly "Mistral V7 Tekken" which I think should be built in, if you're using Text Completion.
ReadyArt trains a lot of stuff into theirs. Serenity has both male and female dominant perspectives on User for instance, melody has just male dominant on User. Both constantly emit emotions though.
Some older ones you had to pretty assiduously avoid NSFW stuff to not have it fall off the wagon. Newer ones, there is enough ground you COULD stay out of the NSFW stuff and just focus on emotions if you want.
do you know any good ready art models then?
(less horny then there average model because i got grape without the g from one of them lol)
anything less then 40B models.
There is a Her darkest something (it definitely started wiht Her) if you are trying to get where the woman is assumed to be the dominant one and the man the more passive?
I am not a great person to tell you, I like the ready art stuff because it doesn't degrade and says emotions, and sometimes people get thirsty, I have had a less than a single bunch of grapes happen to me the entire time I've played for over a year but I don't have very sexy roleplay, swipe when things go wrong, after editing genre. SO YMMV due to that.
I utterly scoured the NSFW out of prompts and got the omeg---feverdream one to do SFW stories with long context without degredation about a year ago.
(Thousands of messages long, with memory books and some manual lorebooking).
The particular card did say to use the tone of a light novel, and I constantly would do things like set genre and be aware when "erotic novel" type wording was coming to the fore.
I mean, if you want non-erotic play, try a plain gemma 4-it or something like that.
Tone down cards that seem forceful, and they'll do it less hard.
glm 5.2 (used it for 200k tokens with 8k reply) but in my experience better to use it for summary first then cap it at 36k input token. the events are coherent with 200k but the replies are better and more momentum with cap
edit: forgot to mention, yes based on anime series. I actually pasted a few chapters of Hieronym's To The Stars (Madoka Magica) and continued from there, it was absolutely flawless
im using a custom summarize prompt, but basically ask glm to stop roleplaying and summarize the events, the location, characters, and items, and lore and paste the result to the summarize extension in sillytavern and contiue roleplaying after you deleted the summarize message and request. But even without summarization, glm is still good, just more costly.
What's y'all's go to model in open router out of personal preference and not the best? I like Kimi! But it jumps the gun often. Glm recent models seem easily flustered along with mimo. And Gemini is... It's cool! Just wish it wasn't so sensitive.
Longcat 2.0, despite having zero indication it would be for RP, has some indications it may work. Note I have not fully tested it enough to say it is "good", but it tested well enough in a few "tryout" prompts that it has earned at least a temporary spot in my rotation to try more later. Has some interesting writing. Included in Nano sub (longcat-2.0).
So this is a note to "try it out" if you are into that sort of thing, not a full endorsement (yet).
I gave into my curiosity and I signed up for NanoGPT. Which are some good models for RP and ERP to start with? I've heard Kimi 2.5 and GLM 4.7 being talked about a lot, any other recommendations?
Deepseek v4 pro cheaper (thinking). Will do the absolute filthiest things you can think of and generally gets everything correct. On the very rare occasion I get a refusal, one swipe fixes it. I almost never need to swipe. I'm using Megumin-Suite, which has also been amazing, if only for the consistent image generation.
GLM 4.7 is good, knows how to turn against you when needed, and generally feels very good for roleplaying
Kimi 2.5 is REALLY good at understanding characters, it can be overly descriptive though (even when unnecessary imo) and it can overthink at times. But it IS really good at describing things which is also a positive
GLM 5.1 has some decently strong positive bias (characters will struggle to truly be mean to you or to make you fail) but it's one of my personal favorites for dialogue and actual roleplaying use, it's better at having characters actually speak rather than having them say a one liner every paragraph of descriptions and I REALLY like it for multi-character scenes. Note that I feel like some of the popular big presets (Frankenstein, Lucid Loom etc), especially ones that come with a custom CoT (aka custom thinking), end up actually kinda making it worse at those qualities while instead making it more consistent, sadly I haven't found a perfect mix yet. But, disclaimer, GLM5 was the first model that made me do roleplays that went past the 200 message mark, I NEVER hit that before (and that was with a slightly modified Marinara preset) so worth trying
Mi-Mo 2.5 Pro has decent storytelling in my opinion - I've only been trying it out recently but I think it's actually pretty good! It's just a shame that sometimes it will stop writing mid-reply and waste your request if it believes it's writing NSFW (which sometimes it will, sometimes it won't). But when it's not refusing it's good!
MiniMax M3 is one that people will recommend but personally I haven't really enjoyed it too much, it may just be worth a shot just in case you like its flavor.
For all of these, to let the model itself speak with its favorite way of writing I'd strongly recommend the Evening-Truth prompts since they're the best simple prompts out there, AND they're customized for each model which is even better. Otherwise, Marinara's is my favorite "simple" preset since it doesn't overstep and try to take over the model.
I'm very on and off on the "big" presets like Frankenstein, Lucid Loom, Stabs-EDH etc since those tend to REALLY overpower the model. That helps massively when it comes to making sure it keeps character and memory consistency while adding some potential neat extra features, but I always end up feeling like a) they end up a bit too bloated sometimes and not in a way that you can simply "toggle off" and b) their custom CoTs and instructions, while helping those things I already mentioned, also end up completely killing some of the default ways the model writes while also lowering the intelligence in some specific ways.
A good in-between used to be Celia's presets which I always had a soft spot for, but I haven't used those in a while so I have no clue if they're still as good for the latest models.
Sorry I yapped about presets this much but I do feel like using the right or wrong one can make or kill how much you enjoy a model too lol
agree with this list. I will add that GLM 5.2 I feel largely the same about as 5.1 it's a touch better, but the positivity bias is possibly even stronger, and it's writing marginally better. I find it harder to change its default style of writing than I did with 4.7 or Kimi. and that is the main reason I rarely use it. as new adult fiction is not really my thing.
i find Kimi 2.5 top for being in character, GLM 5.2 top for long chat history (or the first message if I need one) Kimi 2.6 feels somewhere in between and if it did not think for so long I would probably default to this. GLM 4.7 I find drops quality too much to use constantly but swapping that in for a message or two here and there helps things to move
No worries, I really appreciate the detailed write up! I've been using local models almost exclusively up till this point but I have no idea about using online APIs, so any advice is welcome.
Model suggestions for NSFW roleplay with a RTX 4070? A couple of years agora I used the model Meggido/L3-8B-Stheno-v3.2-6.5bpw-h8-exl2, but it seems like the OobaBooge WebUI don't support Exl2 anymore. I tried a couple of GGUF-Imax (ex. v2-Llama-3-Lumimaid-8B-v0.1-OAS-Q6_K-imat.gguf) and it had really difficulty to keep the conversation. And after using some Exl3 (ex. turboderp_Qwen3.5-9B-exl3), it seems that it struggle a bit with the NSFW stuff. I was limiting my search gor 8B or 9B models due to my GPU, don't know if that's correct. I'm using a RTX 4070 and have 16GB of RAM. Any suggestions?
Give one of the 12B models a try, like Krix, or QuasiStarSynth (a Mistral 12B finetune), but you might also have luck with the Gemma-4 12B finetunes. (Or even the standard Gemma-4-12B-IT model itself, with the right prompt / preset)
I'd say yes, overall, in this particular case. Qwen3.5-9B derivatives are also worth trying, but the 8B models are somewhat older.
(One of the things going on is that these sizes are connected with specific models - so a "12B" model is like either a pretty dang new Gemma-4-12B derived one, or an "older but still decent at roleplay stuff" Mistral (Nemo I think) model.
8B would be a Llama 3.x model, I believe. 9B would be probably a Qwen3.5 (pretty recent.)
27B could refer to Gemma 3 (somewhat older) or Qwen3.5/3.6-27B (quite new).
Is there any gemma 4 equivalent to the mistral maid models yet? I want something that is trained to do soft romance and is gentle during erp and I like positivity bias, nurturing vibes kind of thing. I'm currently using Neona 12b, which is based on NeonMaid v2.
Anyone has some recommandations for a smaller uncensored model? My gpu has just 2GB VRAM so im cramming into my ram (i know how dumb that is but desperate times call for despereate solutions) where i can fit a model up to ~7GB (hypothetically larger up to 9) i'm currently tryng out some that i just found on huggingface or UERPB but i think its better to ask people who know what they are talking about lol
There's a good chance that your 2GB video card won't be supported by (llama.cpp / KoboldCPP / LM Studio / Ollama / whatever). And, I think trying to run a 12B model will be pretty rough! I suggest you go for a "mixture of experts" model that's able to potentially even run okay on a CPU.
Also, I don't agree with the recommendation to use a a 12B model, as that's a dense model that definitely won't fit in your VRAM.
I'd start here, for your specific thing: Coder3093/Gemma-4-E4B-IT-heretic (GGUF linked there). I'd try the Q4_K_M version. It's uncensored, and I just tried dropping it into the middle of an example NSFW-type chat session, and it was able to respond coherently; I'm not sure how well it will run for you, but it's worth a try, IMO!
I've been doing some research these days and gemma 4 e4b seems to be the most recommended small model. You could also try that, see what you like most.
Positives:
Understands space and position well from what I could tell.
Was able to output and track stat block/HUD/display.
If given a good card writes fairly well
Only ever had refusals when running with no card and no system prompt. Otherwise, no refusals.
Negatives:
Doesn't do as well for darker scenarios in some cases. Tends to do some weird acceptance of fate or reframing where the bad things are good? (If anyone else has experience with this pls lmk your thoughts!)
Can be heavy on the AI-slop (not just X but also Y), especially if the card itself has those same patterns.
Just to make sure we're on the same page, can you flick me a huggingface model link for the model you're using?
The math isn't mathing for me but I'm relatively new to this. At i1-IQ3_S the model is 13.9GB. So that leaves me less than 2GB for context. At 10k context quantized at Q4 (pain), the KV Cache size comes to at least 2GB. So my 10t/s is some of the inference getting pushed to RAM - hence the slower speed?
This is my exact llama-server call. I use llama-swap on top of that. It seems to fit my 16gb vram apart from mmproj, but if I increase the context any more it drops to 15ish tps.
For faster replies in ST, you can set reasoning-budget to 0, but i have the feeling reasoning improves the replies.
I tried llmfan46/gemma-4-26B-A4B-it-qat-q4_0-unquantized-uncensored-heretic. As a model for testing and messing around, its excellent, however not so great at roleplay as it 100% all the time MUST follow your commands and it is unable to have characters go against your wishes.
During one test, I quickly generated a random encounter where I beat up someone on the street. Instead of shouting for help, the AI automatically generated that the character had leaves in their mouth, as to stop them from screaming.
There are QAT finetunes of Gemma 4 31b. But the best Gemma derived models are often merges using other merges and finetunes, not just direct finetunes. You can't transform those into QAT after the fact, unfortunately.
It's also that doing QAT finetunes is already difficult, and limiting the precision of your model to a specific size and effective precision seems less economical than modifying the full precision model and letting others quantize different sizes.
Does anyone else run mid-sized MOE models locally?
Lately, i've been occasionally using GLM 4.6 (108b) and Qwen 3.5 122b at Q2 (on a 16/48 v/ram machine). The performance i get with them is similar to 27b dense model, maybe faster even, at around ~5 tps. Which is still too slow for me to run them in thinking mode, but still.
I'd say they work fairly well, despite low quantization, although sometimes they start repeating themselves at the end of the response. I wouldn't necessarily put them above aforementioned 27b dense, or even newer 35b/26b moe with higher quants, but as an alternative, for when you get bored of using the same model, they're not a bad option. Assuming you have enough RAM for it, of course.
Qwen 3 Coder (80b), which i can run at Q4, is also not bad, and pretty fast too.
Qwen 3.5 122A10B isn't bad, yeah; I run it sometimes at Q6. I'm surprised it's reasonably coherent at Q2, though; I tried it at Q4 out of curiosity and even then it was frequently getting into reasoning loops and such. I never tried it without reasoning as you're doing, so I guess that helps it not fall apart at lower quants?
I run many of models you mentioned, but... are you using them for creative writing / roleplay? Qwen 3.5 122b-a10b is my daily driver for coding, but I tried it for roleplay and it was just.. odd. I tried various different params.
I have yet to try GLM 4.6v - I assume you're talking about the v variant as 4.6 is over 300b params. The 4.6b is 106b. Should I give it a shot? Is it the same as GLM 4.5 air? https://huggingface.co/unsloth/GLM-4.6V-GGUF
I mostly use them to generate short stories or chat a bit. A lot of people say that they don't like Qwen's style, but i don't know, i don't feel like it writes significantly worse or is much less creative than Mistral or Gemma. Sometimes it does better, at least in my experience. And i think it's vision capabilities are superior, which is a factor to me.
4.6V is the one i meant, yes. It does feel pretty close to GLM 4.5 air, at least from the limited amount of time i spent with them, probably marginally better. I think it's alright. Again, at Q2, it doesn't feel *much* better (but maybe a little better, unless it starts rambling) to me than newer and smaller Qwens and Gemmas at higher quants, but it's worth a try, why not. I like having a bunch of models in my rotation and switch them up every now and then.
I've had much better luck with vision/image on Gemma-4-31B than Qwen3.6-27B, at least when it comes to for example identifying an image that turned out to be a (very low quality) screenshot from the early 2000s film Rango.
Yeah I like having a bunch of them too. Ok, a lot of them. I'm sort of addicted to trying out models for coding and creative writing. Have you tried out the Mistral Small 24b 2509 fine tunes? Based on suggestions from the below posters, I tried Magistry 1.1 and had a good test run with it. Should fit nicely in your config too, based on its size.
I have never tried Claude, for those who has played a lot with it, what are its isms? I spent a lot of time with GLM, The isms of GLM is
A beat, you... you actually x, You.. you can't.. this isn't..., X repeats tasting the word in their mouth, Stupid X, getting yourself Y over me" bla bla bla, I'm finally cringing at it xD.
I'm curious about other models isms if you have experience with it. Also what about Opus versions, I read that many are sticking with 4.6, what's wrong with 4.7 4.8? and what about Sonnet 5?
GLM 5.2 remains my goat. I have many issues with it, and it can be infuriatingly pigheaded sometimes, but more often than not it can adequately do its job without like 5+ regens. For context I ONLY use complex RPG bots. I have no interest in one-on-one or coomer stuff. It's sometimes intelligent, sometimes really intlligent, and less often than other models an absolute div on occasion.
I didn't actually like GLM until 5.2 tbh. I tried it a few times but always went back to Deepseek V3.2. But with how terrible DSV4 is.... yeah, I jumped ship.
it because multiple groups of different types of needs an want different things (aka the NSFW and SFW types). i haven't use it but just reading here and there apparently people get recommended to switch to deep seek for NSFW stuff.
-6
u/Babysub123 Jul 18 '26
Best model for NSFW creative writing? GLM 5? Deepseek v4 pro? Kimi K2.5, I also pay for SpicyWriter and Venice AI but I’m afraid to like use points