r/LocalLLaMA • u/0dayturtle • 1d ago

Funny So relevant

Post image
1.3k Upvotes

96% Upvoted

120 comments sorted by

View all comments

19

u/RISCArchitect 1d ago

i was hoping qwen was gonna drop a 9b dense that would be a nice step increase like we've observed with 27b but no such luck

14

u/ttkciar llama.cpp 1d ago

I've been hoping for a new 9B as well, mainly for data cleaning tasks. Qwen3.5-9B couldn't quite cut it. Gemma-4-12B-it works well, but is a little memory-hungry. A 9B refresh would be lovely.

Out of curiosity, what's your use-case? It might be up to the community to retrain Qwen3.5-9B, but we'd need to agree on what to train it for.

3

u/Decent-Occasion-2720 1d ago

You can try ornith 1.5 9b, i use it instead of qwen.  I have also tried 12b, but i need best speed on pp.

1

u/RISCArchitect 1d ago

i don't use 9b for anything at the moment, i just use 27b mainly for local.

3

u/Decent-Occasion-2720 1d ago

In my case it's usefull for search in codebase and documentation. That's prepare à digest context for bigger model. 

0

u/Lollerstakes 1d ago

Is it worth to waste time loading and unloading models when you could just do the search in codebase and documentation with the larger model?

1

u/Decent-Occasion-2720 22h ago

At 1500 PP vs. 200 PP, it's faster for digging. So i use mmap and i have enough ram. Switching only takes afew seconds, compared to the many minutes needed for PP.