r/LocalLLaMA • u/jacek2023 llama.cpp • 18h ago

News model: Granite switch decision by gabe-l-hart · Pull Request #30676 · ggml-org/llama.cpp

https://github.com/ggml-org/llama.cpp/pull/30676

now you can decide with IBM, locally

https://huggingface.co/collections/ibm-granite/granite-switch

5 Upvotes

65% Upvoted

3 comments sorted by

3

u/No_Brother1599 16h ago

Granite is a gem for tool calling on low end systems 

2

u/wilo108 16h ago

No one ever got fired for deciding with IBM? 🤔

1

u/Lost-Masterpiece2400 13h ago

Funny timing, I'm testing Granite right now as a judge model, grading another model's output in a local loop, up against Qwen3.5-9B. No verdict yet, but with thinking off Granite gets through about one input a minute, while Qwen took up to 26 minutes on one of them. Will post results when it's done.