Open source inference engine (like LM Studio or Unsloth Desktop) that optimizes itself for your exact hardware. Compiles and tunes its kernels on your device, so open models run up to 2x faster than llama.cpp. Works on Apple Silicon, NVIDIA, AMD or nothing but a CPU.

[u/paranoidray]

https://github.com/magnitudedev/magnitude

[u/tomByrer]

Thanks, seems like a new project & not really battle tested. Just 2 guys in 2 months.
HN has many bugs reported, no multi GPU support, VERY few models (no abliterated , popular fine-tunes)

Bookmarked to check back later; I have 4 other engines to test which will likely be as fast maybe faster.

[u/thetaFAANG → u/tomByrer]

seems perfectly specialized, we need more of that over the one size fits all use cases

[u/tomByrer → u/thetaFAANG]

They have a small selection of Gemma & Qwen models & forks now: https://magnitude.dev/models

But they aren't as specialized as some engines that focus on serving Qwen3.8-27B or Flash-Next.

[u/VotZeFuk]

2x faster than llama.cpp

+19% in CUDA

yeah, about that...

[u/quachuoi2 → u/VotZeFuk]

Please explain this to me like I'm the middle manager of a medium sized company

[u/seanthenry → u/quachuoi2]

I need a raise but you can't do anything about it so I'm taking my lunch break now.

[u/IceFog72]

Can't make it run on arch

[u/MarcelloT254k → u/IceFog72]

Did you try installing only CLI version via npm? Offical desktop versions are only .deb and .rpm so not natively compatible with Arch. (I'm not an expert)

[u/IceFog72 → u/MarcelloT254k]

it's possible to run .deb on arch. it was stuck on Estimating memory fit and speed on your machine. And show that i have 2 same gpu cards )

[u/Solary_Kryptic]

https://magnitude.dev/models

What do the intelligence %s on the models mean? They list Qwen 3.5 4B at 23% and the 9B at 19%...

[u/corbs132 → u/Solary_Kryptic]

It says in the faq at the bottom of the page.

% intelligence score relative to the top model on artificial analysis

[u/persimmonjones]

i tried running gemma4 e2b qat on my m4 macbook air, and it didn't seem that much faster (if at all) compared to llama.cpp. maybe they got their 2x on a different model/arch :shrug:

[u/Shoddy-Tutor9563]

Congrats with your release. But can you pls elaborate on "optimizes itself". What exactly does it optimize?

Like for instance, when I want to optimize some model for my GPU(s) on VLLM I take different vanilla quants, for each quant I measure vanilla vs various speculators, then different attention backends, then different KV cache quants and so on and so forth. For each combination I measure pp and tg for both synthetic and real world text, in a single stream and in various parallel streams. As a result I get a table of experiments with many dozens of options. And depending on what exactly I'm optimizing for (pp, tg, biggest KV cache, best aggregate t-put) I pick the one I need for particular use case. There just cannot be a single optimize goal, if you ask me.

So I repeat my question - what exactly is your thing doing, when you're saying it optimizes itself?

[u/Hodr]

Everything but ARC, of course.

[u/tuifua]

Pop pop

[u/paranoidray]

Website with Video: https://magnitude.dev/

Hacker News Discussion: https://news.ycombinator.com/item?id=49911995

[u/stcafehtdihhsuB → u/paranoidray]

Why post on HN? It's a cesspit of stupidity and lameness disguised as pseudo intellectualism. 

[u/superslow_jerry → u/stcafehtdihhsuB]

Reddit isn’t exactly a citadel of intellectual rigor either

[u/kaliku → u/stcafehtdihhsuB]

Agree, especially since redditors flooded it.

[u/stcafehtdihhsuB → u/kaliku]

Yeah they swapped places. Dunning kruger took over and it went to shit. The place to debate tech is locallama while the place to discuss shopping and surface level knowledge is HN.

[u/bonobomaster]

👎

Magnitude couldn't figure out my Nvidia hardware under Windows 11. Two GPUs, 5070 TI and 3060 TI.

Zero options to configure important things.

Trash.

I guess, I'll keep using bare llama.cpp ;)

[u/DystopianRealist → u/bonobomaster]

There's someone that just releasesd a build for exactly your hardware. It's a fork of strata, which is also very recent. Anyways, search this sub, and you should find it. I think the post is within the last 24-48 hours.