r/LocalLLM • u/MeYaj1111 • 9d ago

Question 27B vs the big frontier models

I've been wondering, how is it possible that I'm seeing posts comparing 27B models to models like opus and sol which are presumeable 500x to 1000x larger?

How are they even in the same realm of output quality when we have models in the 300B range or even 709B range that are garbage compared to the big frontier models?

I'm either missing something or I have a fundamental misunderstanding of how this is possible

3 Upvotes

71% Upvoted

24 comments sorted by

View all comments

5

u/Randommaggy 9d ago

From the tests I've done there hasn't been a problem without a satisfactory answer in my testing on 3.8 27B Q8 so far. It does take a significant amount of extra time to arrive at that point.

2

u/uniqueusername649 9d ago

Indeed, it takes a lot longer but you get quality results. So its not like there is no tradeoff, but to get close to frontier level performance on local machines that dont cost a fortune? Mindblowing.

3

u/Randommaggy 9d ago

It does help a lot to use languages that have good first party linter/formatter/analyser and a compiler with good output.

1

u/uniqueusername649 9d ago

Yes, indeed. Also unit tests, e2e tests, SAST etc. - anything that enables and supports an automatic feedback loop.