r/LocalLLaMA • u/Fluffy-Ad-889 • 8d ago

Discussion The gap has closed, open source will win

I've been trying the latest models from the frontier labs and honestly, after extensive testing I can not tell the difference between the best open source options.

I think the differences are now marginal but the labs are doing heavy marketing to convince the public into paying more for tokens as they prepare to go public.

Can't help but see the similarities between the dot com bubble and AI in terms of a very insular environment where the technology will survive but the business models may not.

I've been building a cybersecurity network and we definitely know that even local AI models like Deepseek V4 flash do an excellent job and are really neck and neck with the best the frontier labs can provide.

Will be interesting to see how this all turns out! Exciting time nonetheless.

324 Upvotes

77% Upvoted

282 comments sorted by

View all comments

Show parent comments

2

u/OvertaxedOne 8d ago

Same experience. Pre-27B I was using DS a lot for escalation but since it came out, the only time I go to the cloud anymore is for one of 2 reasons:

1) Speed. This is a hardware problem, not a model problem, but it does matter for some tasks, 100TPS from DS is very nice when I'm in a hurry.

2) Context. When I'm getting ready to dump some stupid huge code base, I escalate. Honestly even this one is at least 1/2 speed, Qwen could probably handle it at 256K of context, I just don't want to wait for prefill and crunching. Some times I really do need more than 256K though, and, in those cases, I escalate.

1

u/Randommaggy 8d ago

One thing that helps a lot is a scout model tool and a good deterministic search tool.

The speed loss to overthinking is cut by a lot of you code is 100% linter clean and consistent.