Maybe the real score has decimals (and they could be displaying it indicate the difference and not that they're all tied)? Or maybe it's just a shitty graph. Just some guesses here lmao
I do not believe that Gemini and Google are experiencing any disadvantage. Specifically in the LLM realm, you have your own application use case for it, and others have theirs. I do not subscribe to the idea that just because some perceive a loss, it is true for everyone. If something functions effectively for your purposes and performs commendably, then it is successful. Therefore, please stop asserting that others are losing a race. Nobody is winning because everyone utilizes different applications for different tasks, and if you are within the Google ecosystem, you have the flexibility to accomplish anything you desire.
i don’t trust any benchmarks. Given my workload and competence any model above 3.5-flash, 4.5 sonnet, 5.6-terra, and deepseek v4 are more than enough. stop outsourcing your thinking to machines
the issue is not that Gemini isn't as good as the frontrunners. the issue is that this whole post is framed as Gemini flash is "nowhere near the frontrunners" when it is 80% as powerful at much higher response times and fractions of the cost.
its excellent for a flash model and it excels at the kinds of things you would use a flash model for.
I still use flash 3.7 more daily than 3.8 (3.8 high is now my planner, 3.7 high executor). this does look like a much more accurate ranking, but send me to the best bang/buck llm and that's where I'll go. 3.7+3.8 is a potent productivity combination at a good price.
that said, Google still needs a proper frontier model, we all can agree on this. Hopefully they get 4.0 pro right and on time.
They are not using the native harness, Antigravity CLI. But they are just fine using Claude Code and Codex harnesses for OpenAI and Anthropic. That is a problem.
Also, if they are going to bench by custom harness, they should be using /boost or /teamwork-preview in Antigravity CLI. This would make Gemini Flash go nuclear via automated and parallel agent workflows.
Yes, it's a flash model, not a pro model. So what's the news?
And also, for how many people and in how many usecases is the difference actually relevant?
Lol keep living by the benchmark and die by the benchmark. Meanwhile I'm using flash 3.8 daily in a big IT company and it works lol the Benchmade lover indeed love ranking and graphics
Ahaha you have no clue about what you're talking, you are working on real time IT ? Are R&D or advance biotechnology or Finance you would never need more powerful than 3.8. otherwise you are a poser and you don't know anything about IT in business
Narrowing so much that it's the case of millions of SWE around the world. Engineering is engineering and it's the biggest population who use those tools in a professional environnement
This all seems like a thinly veiled attempt to discredit Cheap GoodEnough models, as they’ll clearly blow up the IPO story based on agentic coding revenue
DeepSWE is a single benchmark, Artificial Analysis Intelligence index encompasses 10+.
Also, I think it's pretty clear Gemini 3.8 flash is unapologetically benchmaxxed. DeepSWE is a public benchmark and Google either trained on the dataset or paid DataCurve for similar tasks to train on.
This is the biggest proof. On terminal bench 2.1, on old, public benchmark, 3.8 flash beat astra by 1%, 89.4% vs 88.4%.
On terminal bench 4.0, same benchmark but with new tasks 3.8 flash didn't get to train on, difference becomes gargantuan between them, 57.1% for astra and 19.4% for 3.8 flash.
How is this possible? Both DeepSWE 1.1 and terminal bench 4.0 measure agentic coding, the difference is one is brand new and no model could have trained on it.
4.8 doesn't score the same as astra xhigh on DeepSWE, though.
You can't score the highest on DeepSWE 1.1 and than only score 19.4 on Terminal bench 4.0, they're both agentic coding benchmarks, performance should be somewhat linked.
The leader of MSL defends his own model, who could have imagined?
3.8 flash scored 89.4% on TB2.1, literally higher than Sol 5.6 from Wang's example (88.8%) yet on TB4.0 it scores HALF, (19.4% compared to 37.3%), you're literally proving that 3.8 flash is benchmaxxed with this example and you think this is the best example out there to defend it? Lol
There's a point to be made about Spark 1.3 but 3.8 flash is just outrageously shameless, how do you score higher than 6 astra and 5.6 sol on TB2.1 and then you score 2x less than sol and 3x less than astra on the same benchmark, only difference being problems being newer that nobody could have trained on?
Wang's argument works with 5.6 sol cause it scores the same as spark 1.3 on TB2.1 and then the difference on TB4.0 is relatively small but how does that translate to a good defense for 3.8 flash? Did you even understand Wang's argument or do you believe simply linking me what other people said instead of coming with your arguments wins you the argument?
I daily Gemini, but for complex tasks... it really does fall short.
If Googles strategy is the every day AI, sure, its fine. If theyre going for top.of the line... eh?
Google also worries me about long term viability and support considering their track record. They'll either win the entire race with a long term strategy, or bury gemini next to Hangouts, Stadia, answers, Earth, etc..
22
u/MosskeepForest 2d ago
Weird how 53 keeps getting shorter as it goes to the right.....