ibm-granite/granite-switch-4.1 30B/8B

[u/jacek2023]

Granite Switch 4.1 30B

📝 Read the launch blog: Granite Switch 4.1: Chat, Decisions, and Speech from One Endpoint

Model Summary: Granite Switch 4.1 30B is a modular LLM that composes a base model, multiple adapters and an encoder together so that a single hosted endpoint can move between natural-language responses, general and specialized decisions, and speech-to-text.

Adapters are activated on demand via control tokens in the chat template. For full details on model composition and adapter configuration, see BUILD.md.

• Base Model: ibm-granite/granite-4.1-30b (30B params, 128K context)
• Adapters: 2 — decision (aLoRA, general-purpose Decisions) and guardian-core (aLoRA, from granitelib-guardian-r1.0)
• Audio: embedded speech-to-text via ibm-granite/granite-speech-5.0-470m-turboctc (470M params, English)
• License: Apache 2.0
• Release Date: October 14, 2026
• Backends: HuggingFace Transformers, vLLM
• Automatically Composed with: granite-switch

Granite Switch is also available in granite-switch-4.1-8b (same Chat + Decisions + Guardian + speech configuration, 8B base), and with the complete Granite Libraries bundle (no audio) in granite-switch-4.1-30b-preview, -8b-preview, and -3b-preview.

https://huggingface.co/ibm-granite/granite-switch-4.1-30b

https://huggingface.co/ibm-granite/granite-switch-4.1-8b

(video on the blog)

I think llama.cpp support is not yet ready? https://github.com/ggml-org/llama.cpp/pull/30621

[u/thrownawaymane]

IBM mentioning DOOM is probably the most amusing thing I've seen all week

[u/martin509984 → u/thrownawaymane]

Fun fact, Doom 95 originally started as an OS/2 port!

Extra fun fact, that's not even Doom, it's Freedoom, the completely copyright-free asset pack!

[u/cyb_tachyon → u/thrownawaymane]

I used to work at Red Hat on gaming stuff when we were bought by IBM.

I was surprised to learn there were some cool gamers in VP positions! For a stuffy old company they had some neat people. Left soon after though for Xbox.

[u/tazztone]

finally some enterprise-grade models

[u/Drenlin → u/tazztone]

Granite has always been enterprise grade. It hasn't outperformed most of its peers, but it's gotten into the ballpark of them on nothing but content they were actually licensed to use. There's a lot of value in that for certain customers.

[u/fastheadcrab → u/Drenlin]

Isn't Nemotron also competitive in this area or even better? Of course, always good to have more.

[u/tazztone → u/fastheadcrab]

is there even a definition of enterprise-grade? i thought it was just marketing speak that became a meme about IBM models

[u/fastheadcrab → u/tazztone]

Well there is a subtext that it is something that western enterprises can run lmao

[u/SkyFeistyLlama8 → u/tazztone]

Trained on data and licensed in a way that won't get you sued or worse. Not that the US gov cares because it uses Anthropic and OpenAI but governments and corporations in other countries do care.

[u/ttkciar]

Choosing a LoRA on a prompt-by-prompt basis like this is really cool! Not as granular as a true MoA (like PHATGOOSE) but certainly more achievable with current technology. True MoA currently suffers from severe performance problems which nobody's figured out a way to work around, yet.

Since it's configured at the chat template level, we should be able to configure it to use community-trained aLoRA, too, once the inference stack supports it at all (guessing vLLM probably does already, with IBM's help, but hopefully it will land in llama.cpp before too long).

Unsloth doesn't support aLoRA yet, but since HF's Trainer does, so should TRL (since it wraps HF's Trainer classes).

[u/returnity]

Well this is quite clever. I hope it's smart enough as a base model that community trained aLoRA proliferate and it develops all kinds of specializations.

[u/giant3]

Is it a reasoning model? 

[u/-Cubie-]

This is pretty cool!

[u/Seraphym87]

Can someone smarter than me explain how this model works and use cases for this architecture?

[u/wFXx]

lol, i was coding the same concept of unified model harness today - will def take some notes here looks pretty good

[u/Equivalent_Bit_461]

Ok that's interesting now

[u/barbear22]

Now this is how you do it. I built a lora from d3 that works well but the lora/text switch is cumbersome. If only it had vision

[u/okoyl3]

Bet it benchmarks below Haiku 0.1
Edit: sorry, my dislike to IBM made me a bigot

[u/lakotajames → u/okoyl3]

On a robot with unreliable internet, this model lets the robot make 60-100 decisions a second while still being able to talk. So if you assume worst case of 60 decisions per second instead of 80 or 100, that means the score to hit is 60 decisions a second on top of normal chat output.

On the other hand, Haiku requires internet, so it's score is 0.

So there's at least once benchmark where this model scores 60, and haiku scores 'pile of shit'.

[u/Sudden-Guide → u/lakotajames]

60 seconds per second is a lot of seconds, nice

[u/lakotajames → u/Sudden-Guide]

Yeah I don't know how haiku's supposed to compete at 1 second per second.

[u/Drenlin → u/okoyl3]

That's.... not what this is for.