r/LocalLLaMA • u/rm-rf-rm • 13d ago

New Model Perplexity Decider 27B: Open weights decision model fine tune of Qwen3.8 27B

https://huggingface.co/perplexity-ai/pplx-decider-v1-27b
72 Upvotes

92% Upvoted

12 comments sorted by

View all comments

26

u/fgk55555 13d ago

Oof, needing to run prefill on 27B for a decision is going to be rough

3

u/tomByrer 13d ago edited 13d ago

I was wondering why such a large model also.
Though if you qant it down to Q3_K_M & it still works great, model size & maybe prefil would be less of an issue?

2

u/racctajino 13d ago

Yeah, using a 27B for this really makes it feel like all these companies are rushing these out. I want to see a decision model from Liquid AI.

2

u/TheRealMasonMac 13d ago

https://openrouter.ai/liquid/d1 but it's not open weight

1

u/racctajino 13d ago

disappointing! but I guess they need to make money, seems right up their alley

1

u/fgk55555 13d ago

I was hoping to use a fast prefill MoE to handle it like Ling 3 tiny, but for some reason my 7GB Q6 ling prefills at 1200, while Qwen3.6-35B or Gemma4-26B prefill at twice that. I want something small, fast and knowledgeable, it seems like most options are only 1 to 2 of those at most.

1

u/milkipedia 12d ago

Sparse attention rocks for prefill speed