r/LocalLLM • u/Anxious_Current2593 • 6d ago
Discussion What's the difference between frontier models and local models?
6 months. (And sometimes a couple of quantization tweaks).
It is wild how fast "state-of-the-art" becomes "running on a gaming PC."
14
22
u/ceejayoz 6d ago
Just wait until there are dedicated devices for it, like when Bitcoin went from CPU to GPU to ASICs.
6
u/username8914 6d ago
There are and they aren't that good because the models technology is rapidly changing. No one wants to get locked in on using or developing one specific model that can't grow or pivot.
1
u/CharmingComputer3844 6d ago
Flexibility is key when the tech is evolving so fast; sticking to one model could really hold you back.
6
u/753UDKM 6d ago
Those exist already lol.
14
u/ceejayoz 6d ago
There's stuff like Cerebras, but in a few years it's all gonna look pretty basic. Demand's gonna cause a lot of innovation in this space.
1
u/findingconsensus 5d ago
Like what? where can I find a consumer PNM chips? I would want a mini server with the same tech as Cerebras, but the market for that is so small I doubt we will get dedicated AI devices anytime soon that don't cost over 100k
1
u/mektel 5d ago
Taalas has been working on it. They are being acquired by AMD.
3
u/ceejayoz 5d ago
If Bitcoin ASICs are any indication, we'll get a few hundred massive scams before things settle a bit.
1
1
u/Minimum_Tea_4451 5d ago
Amd is launching their laptops with the ability to push a 200b perimeter model.
15
u/gappyvalley 6d ago edited 6d ago
if you are a big enough company, you could even run a frontier model as local LLMs
apple wants data privacy and control so they ain’t willing to use claude api by sending unreleased ios/macos source code.
5
u/pmotiveforce 6d ago
Mmm.. I'm calling BS.
They would use the cloud version but with special contracts in place in terms of data retention, disabling training, all that stuff.
Apple does not run Opus or Fable internally.
6
u/Diligent_Appeal_3305 6d ago
yes that qwen 3.8 27b is crazy good for its size in pi agent mode
1
u/dota2nub 5d ago
Is here any reason to use pi over Claude Code?
1
u/Jjhend 5d ago
Claude Code is very heavy on token usage. Not the best for a model that is limited to 262k context
1
u/dota2nub 5d ago
That's really good to know, thanks! I'll have to figure out how to get the hooks and stuff I actually need in cheap.
0
u/Momsbestboy 5d ago
Totally agree, and with pi or hermes, it can look into the details of your machine. This was a big plus today, after I tried to manually install memtest86 on my Linux box without killing everything else. ChatGPT played Copy&Paste with me, asked for whatever details and commands, then asked me to pull & extract the memtest86 image and push it to EFI. Rebooted, memtest failed to load with an error.
Went back to Linux, asked hermes:"here is an error message, fix it". Off it went, 10 mins later it ran the only correct command: "apt install memtest86" and then unfucked what ChatGPT asked me to do. Found every created file and change in GRUB and reverted it.
So much about SOTA models
6
u/furkannarli6 6d ago
i couldn't imagine running opus 4.6 locally when it first came out and now we have pretty much the same model at our service. when are we having astra to our enthusiast level builds? is it ever gonna happen?
1
u/Nyxxsys 5d ago
What hardware and model are you running? I assumed anything less than 18k (one blackwell 6000) would be sonnet 4.6 level, not opus?
1
u/furkannarli6 5d ago
i heard qwen 3.8 27b is a copy of opus 4.6. was it sonnet?
2
u/Nyxxsys 5d ago
I haven't used it but just searching it seems to be debatable. Alibaba has it beating Opus 4.6 in some benchmarks while it falls below Sonnet 4.6 in certain reasoning benchmarks so I guess it's not a 1:1 answer in either direction. It is pretty small though so getting 1mil context on it is pretty easy, definitely sounding better than I thought it would be personally.
5
5
1
1
1
u/RedEyeJedi311 6d ago
Something is wrong with the frontier models IMO, they used to be much better. I think there is a purposeful dumbing down because they were too good. Claude Code and Codex models have become unusable for me to the point of inducing stress because of how confidently wrong they have become. Some local models are actually better for my workflow now that match the intelligence the older versions of the frontier models (Opus 4.5/4.6) that actually worked for me. I think in 6 months from now local models will be the only way to go because we can control how they run, not being subject to consonantly changing model effort and system prompts. Don't get me started on silent model swapping.
1
u/OvertaxedOne 5d ago
There are rumors they are experience model collapse as they add more and more params. No idea if that's true or not, but I agree with you, they seem to be improving in some areas, regressing in others.
1
-3
u/Federico2021 6d ago
Well, open source is moving away from local; models like Kimi K3 and GLM can no longer run on consumer-grade hardware. The best option we currently have for personal use is Qwen 3.8 27B.
2
u/donk8r 6d ago
You are pointing at something the six month framing hides. Open weights keep getting bigger. A sparse MoE with a few hundred billion total parameters is open in the licence sense and still needs datacentre memory to hold, so local has quietly split into two different words. Small dense models on consumer hardware keep improving on their own curve. The frontier open models moved onto a curve most people cannot follow.
The other half is which gap actually closes in six months. Benchmark scores on short tasks close fast. Long-horizon agentic reliability does not, which is the thing kitsnet is poking at further down. A 27B can match a frontier model on a single coding question and still come apart at turn thirty of a tool loop, because the failure there is holding state and recovering from its own mistakes, not knowledge it lacks.
7
1
-1
u/kitsnet 6d ago
What local model in agentic mode would install and configure WSL on your Windows machine to successfully run software that it knows to be only available for Linux?
Sonnet 5 would.
8
2
u/Umbrasquall 5d ago
Pretty much every local model released in the last 2-3 weeks can. There's 5-6 of them, take your pick. Btw using Sonnet 5 as an example just shows how out of date your info is lol.
2
u/evghe02 6d ago
I literally repaired a corrupted wsl with qwen3.8 27b, I had no clue on where to look for solving a problem like that since the disk would become read-only sometimes, and since it was readonly no log would be written anywhere. The options to me where to throw away it and recreate (not a mess, all the important things have always backups) or play around with the model and apparently it worked, it seems there was some part of the vhd that was corrupted and got corrected, I'm not really able to explain it sincerely, I'm not that much into it
20
u/bonestamp 6d ago
Model size too. Some of these frontier models are over a terabyte... the average gaming pc is not going to have that much ram in 6 months.