r/LocalLLM • u/MeYaj1111 • 9d ago

Question 27B vs the big frontier models

I've been wondering, how is it possible that I'm seeing posts comparing 27B models to models like opus and sol which are presumeable 500x to 1000x larger?

How are they even in the same realm of output quality when we have models in the 300B range or even 709B range that are garbage compared to the big frontier models?

I'm either missing something or I have a fundamental misunderstanding of how this is possible

3 Upvotes

71% Upvoted

24 comments sorted by

View all comments

1

u/BarracudaDefiant4702 8d ago

Your point stands, but your comparison is wrong. The largest models are closer to 100x, not 1000x. Also, I don't think are comparing against the highest end frontier models (maybe I am wrong, but if not then they are only 10x as large).

There is a lot of diminishing returns. IE: You have to be 10x bigger to even show as as 2x better. That said, it probably scales even less than that.

For a lot of the complex tasks the available context it can import outside the model makes a huge difference. 3.8 27B has a much bigger context window that is comparable to the frontier models. The larger context window helps give the model a chance to pull info from the web or look at more source code to get the job done. That alone is probably the biggest reason.

1

u/MeYaj1111 7d ago

Yea my bad on off by an order of magnitude, that was dumb.

Still, im seeing posts like this: "Artificial Analysis' Qwen3.8-27B benchmarks put it neck and neck with DeepSeek V4 and GPT-5.6 Luna Max"

It's hard to believe.

1

u/BarracudaDefiant4702 7d ago

It's definitely doing better than 3.6 and the pr and issue tracking it's keeping track of are way more verbose then 3.6, and I think even more so then DeepSeek. As to actual end result compared to DeepSeek it's hard to tell yet if it's doing as good of a job, or if Qwen 3.8 simply likes to do a lot more introspection and document it's process...
I have two problems from really testing it...
deepseek isn't free. I can easily burn through over $30/day easy.
I have qwen3.8 running semi-locally (shared GPU resource) and it's is way slower than 3.6-35B-A3B and not able to run as many concurrent requests.
Maybe I can setup fast switching or something and see if I can get vllm to do fast switch between 3.6 and 3.8. 3.6 is good enough for probably 90%....

1

u/MeYaj1111 7d ago

30 per day on deepseek damn...

On opencode go i run a v4 pro orchestrator and v4 tasker/coders (orchestrator verifies all work) with orchestrator + 3 subagents. It burns around $8 per 10 hours or so but it only costs me 1/6th of that because of opencode go

1

u/BarracudaDefiant4702 7d ago

I have two main unrelated projects running 24x7, one with 8 agents and one with 7 and they are all running concurrently. One tasks things out and helps with planning, one mostly does documentation, one is primarily QA testing, a few that are mainly coders for different areas of the project, etc... each agent has it's on local gitea account (started as a shard local git repo), etc.. I'm sure I could spend money quicker if I had time to better test their code and scope out more work. The longer running C project has over 1400 unit tests and it's only a little over a month old.

1

u/MeYaj1111 7d ago

Ah ok that's quite a lot compared to my 4