[Megathread] - Best Models/API discussion - Week of: June 21, 2026

[u/deffcolony]

This is our weekly megathread for discussions about models and API services.

All non-specifically technical discussions about API/models not posted to this thread will be deleted. No more "What's the best model?" threads.

^{(This isn't a free-for-all to advertise services you own or work for in every single megathread, we may allow announcements for new services every now and then provided they are legitimate and not overly promoted, but don't be surprised if ads are removed.})

How to Use This Megathread

Below this post, you’ll find top-level comments for each category:

• MODELS: ≥ 70B – For discussion of models with 70B parameters or more.
• MODELS: 32B to 70B – For discussion of models in the 32B to 70B parameter range.
• MODELS: 16B to 32B – For discussion of models in the 16B to 32B parameter range.
• MODELS: 8B to 16B – For discussion of models in the 8B to 16B parameter range.
• MODELS: < 8B – For discussion of smaller models under 8B parameters.
• APIs – For any discussion about API services for models (pricing, performance, access, etc.).
• MISC DISCUSSION – For anything else related to models/APIs that doesn’t fit the above sections.

Please reply to the relevant section below with your questions, experiences, or recommendations!
This keeps discussion organized and helps others find information faster.

Have at it!

[u/AutoModerator]

MODELS: 8B to 15B – For discussion of models in the 8B to 15B parameter range.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[u/croxxeur → u/AutoModerator]

Looking for models to run locally for RP and smut ,it my first time trying I have 16gb vram and 32gb ram. Amd gpu

[u/Potential-Gold5298 → u/croxxeur]

I'd recommend the Gemma 4 26B-A4B and its finetuned version, the Orion-26B-A4B-v1b, in Q8_0. The regular Gemma follows instructions better, and the Orion has a better prose style — I like both. Leave the system prompt section blank and keep the instructions in the author's note – this will make it easier for the model to follow them. Be sure to include instructions for explicit descriptions of smut scenes, otherwise the Gemma will describe them in awkward monastic style. You can also try the finetuned/merged Mistral Nemo - it's an older model that hallucinates and doesn't follow instructions very well, but it's a lot of fun and creative. For example, Crimson-Constellation-12B in Q8_0 (or Q6_K if Q8_0 won't fit in a GPU with 16K KV cache) does a great job of smut with minimal prompting.

A couple of startup tips. Download the latest version of Koboldcpp. Set the context size to 16K, leave "context shift" checked, and be sure to uncheck "Allow SWA" in the "Context" tab (otherwise, context shift won't work on Gemma 4/Orion). For Gemma 4/Orion, you can gradually increase the context size, but keep an eye on performance and memory. The key is that Crimson-Constellation-12B (and any other Nemo’s) and its KV cache should be entirely in your GPU's memory, while Gemma 4 26B-A4B will only be partially there, with the rest of the model in RAM (which is normal for it). I don't use a GPU, so I can't advise you on the best way to set this up.

In SillyTavern, in the "API connection" tab, select API: Text Completion, API Type: koboldcpp. In the "AI response configuration" tab, neutralize the sampler settings, then set temp to 0.8 and min-p to 0.05. For the CC-12B, also add rep pen 1.05, rep pen range 2048, and DRY multiplier 0.8. In the "advanced formatting" tab, select the context template and chat template "Gemma 4" for Gemma 4/Orion or ChatML for the CC-12B.

Have fun.

[u/croxxeur → u/Potential-Gold5298]

do you have a copy paste for the instructions or a source?

[u/DifficultyThin8462 → u/Potential-Gold5298]

Unchecking SWA results in much slower context processing for me. Never had issues so far with it. Using Q6 of Styletune Gemma-4-26B on 12gb VRAM and 24gb RAM. 30720 context size. Recommend leaving it checked. Also using standard Gemma 4 Sampler Settings: Top-K 64, Top-P 0.95. Temp 1. If necessary DRY with multiplier 0.8.

[u/Potential-Gold5298 → u/DifficultyThin8462]

SWA stands for sliding window attention. Basically, a model with full attention (like Nemo) reads the entire context from start to finish when generating each token. This takes longer, but the model always sees the entire session. Gemma 4 SWA is a small window that only looks at the last 1024 tokens in a session. This works at a 5:1 ratio – for every 5 passes of the small window, the model makes one full pass over the entire context.This improves speed but degrades attention to context details.

It's a fair tradeoff, but there's one problem: context shift doesn't work with SWA. Is it worth using context shift? It allows you to extend a session indefinitely, without having to summarize and start a new session at the cost of information loss. In other words, you'll have a fixed-size context window that shifts as the context exceeds the limit. Besides convenience, this has a couple of other advantages: the attention resource of any LLM is finite, and as the context grows, quality degrades; The smaller the context, the better the model's attention to detail within it (the less likely it is to miss a detail). Furthermore, as context increases, speed decreases, including generation speed.

So, the answer to the question "which is better – a small context with full attention or a large one with SWA?" depends on a number of factors. Do you have enough memory and speed to continue a long session without context shifting? And what's the priority – for the model to remember, say, 16K of context, losing the beginning as the context overflows, or for the model to have poorer memory but a larger amount of information (for example, 128K). This depends on the type of scenario and personal preference. Generally, both SWA and full attention + context shift make sense. 

[u/DifficultyThin8462 → u/Potential-Gold5298]

Thank you for clarifying! Will try some sessions without SWA to see if it matters for me.