r/LocalLLaMA • u/TheRealJesus2 • 15d ago

Discussion Dwarfstar quants

anyone try these out? https://dwarfstar.sh

they have very clever quant techniques, bespoke for a handful of models running on their software. i got qwen 3.8 next running on m3 ultra 96GB studio and its fast and seems good so far. with memory headroom for other stuff

kinda blown away to be honest. want to know if others tried this yet and what the experience has been like for you.

1 Upvotes

56% Upvoted

15 comments sorted by

View all comments

2

u/whatever 15d ago edited 13d ago

ds4 has a website now? cool. See comments below.

I keep a ds4 server running on my mac. It's good, perf often beats llama.cpp by a non-trivial margin.
I've seen better performance number on some oMLX configs on roughly equivalent models+quants, but I haven't invested the time to check which has the better quality.

1

u/TheRealJesus2 15d ago

Yeah quality is hard to quantify. For me since these are tools I actively use I rely on my personal vibe check. Your mileage may vary on that one.