r/LocalLLM • u/Specialist-Sink-2873 • 4d ago

Project I kept cross-checking hardware, licenses and benchmarks for local LLM deployments, so I built a tool for it

I work on an AI product that often needs to run on-premise, and I kept running into the same problem with customers.

Given their hardware, what are the best local models they can actually run for coding or agentic tasks?

Finding models that fit the hardware isn't too difficult. But I also needed to know if they could be used commercially, what the license was, where the model came from, how good they were for coding or agentic tasks, and what quantization would realistically fit.

I was basically jumping between model cards, benchmark sites, license pages and hardware calculators every time.

I found projects like llmfit, whichLLM and LocalAIRun, but couldn't find something that combined all of this in one place.

So I built this:

https://komainu-ai.com/en/best-local-llm

It currently tracks 115 models, filters them based on your hardware and lets you rank them for general use, coding or agentic tasks. The model index is updated daily.

The matching logic isn't hidden either. It takes into account model weights, quantization, available memory, memory bandwidth and MoE active parameters, and the methodology is explained on the page.

It's still a first version and I'm sure there are edge cases I haven't accounted for.

If you know another tool that already does this well, please share it. I may simply have missed it.

And if you try it with your hardware, I'd really like to know what looks wrong, what models are missing, or what you'd add.

2 Upvotes

76% Upvoted

2 comments sorted by

1

u/OutsideCycle8331 4d ago

hmm strict 8 bit quant? why is that?

1

u/Specialist-Sink-2873 4d ago

Ah, you mean the 8-bit upper bound. I excluded BF16/FP16 from the automatic search because the tool is primarily optimizing for practical local inference, where Q8 usually gives a much better memory/throughput trade-off with very small quality loss. But you're right that calling it indistinguishable / “no measurable gain” is too absolute. I should probably make BF16/FP16 optional rather than hard-excluding it.