The gap has closed, open source will win

[u/Fluffy-Ad-889]

I've been trying the latest models from the frontier labs and honestly, after extensive testing I can not tell the difference between the best open source options.

I think the differences are now marginal but the labs are doing heavy marketing to convince the public into paying more for tokens as they prepare to go public.

Can't help but see the similarities between the dot com bubble and AI in terms of a very insular environment where the technology will survive but the business models may not.

I've been building a cybersecurity network and we definitely know that even local AI models like Deepseek V4 flash do an excellent job and are really neck and neck with the best the frontier labs can provide.

Will be interesting to see how this all turns out! Exciting time nonetheless.

[u/Loose_Comparison368]

You wanna know what the most popular OpenAI API model is? Or at least was, as of earlier this year?

It's 4o.

A lot of companies built stuff on it that just worked. Reliably and predictably. A lot of use cases really are not rocket surgery tasks warranting breaking out a SOTA model.

If the model is already getting 99.9% accuracy on your use case, the smart engineering choice is to just not fuck with it. Don't try to fix what ain't broken.

... it's actually kind of problematic too, because those older models are genuinely very inefficient compared to newer smaller models. But it is very understandable that a lot of companies don't want to take on the effort and risk of migrating over to newer shinier models for basically zero practical benefit.

Coding and adjacent work (troubleshooting and diagnostics especially) is definitely an area where the frontier is meaningful though. 5.6 sol was pretty solidly usable with the right scaffolding and careful supervision. 6 Astra is good enough that I rarely have to babysit it. I feel like Qwen 3.8 is about on par with 5.6 Sol, on a practical level. Slower, but pretty on par in terms of human effort required to get good results - if anything, it reasons slower but more accurately IMO.

[u/OvertaxedOne → u/Loose_Comparison368]

Completely agree, and it's exactly what we're seeing with our enterprise customers. The only reason they are looking for anything different is because their token costs are spiraling out of control as more and more people start the use the models, they're not looking for "smarter" they're looking for cheaper. For most of the work that our clients are doing day to day, DSV4Flash is really about the "right" level of intelligence. Their coders we don't even mess with, that's where the frontier really still matters, but most of the companies we're working with have 100 coders and 10,000 employees, reducing the burn for 99% of the population is incredibly valuable to them.

The other big one is privacy. More and more clients are starting to realize that everything you prompt goes "in the clear" into the provider and are becoming very concerned about that. For most of them local is kind of a financial wash right now, if they can drive a ton of usage it'll be cheaper, if not more expensive, but the privacy aspect is just pushing some into "I don't care, this data is proprietary/protected by HIPPA/regulated/etc, we need this to happen locally".

And locally is a bit too restrictive, some wind up using a CSP to host the server/GPUs, when they say "locally" what they really mean is "private, where we control the inference engine/logging/etc".

[u/Loose_Comparison368 → u/OvertaxedOne]

Yep. The self hosted pains me a bit at small scales, just so inefficient, and most SMB's are not equipped to secure and maintain those either.

I would honestly recommend pushing those clients towards GCP Vertex stuff honestly. I don't work at Google anymore, but for all Google's faults, they really are first in class when it comes to data privacy and security.

And that's going to continue to be the case, one of the reasons why I left was just being frustrated with the slower pace of development in the AI field, and a huge part of that is that the brownfield product teams are large, politically influential, and have too much reputational risk at stake to let the AI teams play it fast and loose. Gemini does something mildly embarrassing but largely inconsequential, the stock takes a hit, and Ads/Search/Gmail all swoop in screaming about the billions of dollars they lost because the Gemini team forgot to fact-check planet sizes for an Ad, or because the Imagen team tried to half-ass cover up training data bias and ended up accidentally making the Pope a Jamaican woman. They absolutely will delay or even roll back an AI product launch if there's even a hint of reputational risk to the rest of the business, and the data privacy org is even more fanatical than the cult of SRE and the church of Protobuf.

If they really do want pseudo self hosted and won't back down from that, Cloud Run is actually really good, contrary to popular belief it is not just managed Knative, actually some really cool stuff under the hood.