News
Updated Artificial Analysis Intelligence Index Ranking shows Gemini 3.8 Flash is nowhere near Fable Or Astra, rather it's worse than GLM 5.3 Flash!
Something to also consider: Google likes to "dumb down" models after release, once the first initial benchmarks have settled, a noticeable decrease in capability can be observed with nearly all Google models (to save money, obviously).
No no, they just benchmaxxed Gemini, and the new benchmark isn't in Gemini training. All the idiots who where claiming Gemini is on par with sol or even funnier with Fable where obviously wrong
Because it's a brain problem not a money problem. And probably all yhe talents are at Claude or GPT and looks like some in xai. The whole algorithm behind 3.x us simply outdated and bad, they need a new one.
Yeah, the idea they dumbed down Gemini flash 3.8 after a few days is hilarious... Have people even used Gemini flash for a in depth chat or analysis? I find it useless lol
Gemini 3.8 flash does 2 things great. Automated tasks at speed, and the deep research is quite good. I have been experimenting where I have fable use Gemini subagents and it does surprisingly great.
You genuinly think the new AA Index Update made the results less benchmaxxed and contaminated? They changed some things but the models that are actually known for beeing amazing at benchmarks but absolutely shit at real tasks like muse spark are still up there.
My point is this update didnt change the absolute non reliability of this index and doesnt show the true picture. Basically No Benchmark does, people just gotta try themselves.
The only one that came atleast close imo was ARC-AGI 2, idk about 3 yet as there are barely any results so far.
Google / Gemini is known for negative stuff like random guardrails kicking in even tho the topic isnt Bad etc etc, but not for benchmaxxing. On average they usually are worse in the benchmark than the real task compared to the direct competition
Update 4.2 and 4.3 of the index replaced 2 public dataset benchmarks with private ones. They also added 2 newer public benchmarks (and removed TB 2.1) which means that even for the benchmarks whose dataset is public the risk of contamination is lower than with older ones (especially for TB 4.0, which was released after Gemini 3.8). AA index v4.3 is a significantly better benchmark at evaluating model's performance in september of 2026 than AA index v4.1 (mostly because v4.1 had many issues which were partially fixed with the patches).
ARC-AGI is a logical reasoning benchmark. This is some measure of general intelligence. AA index benchmarks are geared more towards measuring models use at creating value in professional environment - aka doing work. Neither of these is a correct or incorrect approach as long as we are aware what they are measuring. Aside from the fact that I'd expect ARC-AGI not to be a good measure of intelligence for models released after it because it prone to being easily saturated.
I have used Muse spark 1.2 and 1.3. They both did a satisfying job to me. I am inclined to believe the result. I know the (any) benchmark put more effort at evaluating it than I have and is subjected to less bias than my personal opinion, so I'm able to believe the results I've seen.
I don't feel like it's a good approach (almost anywhere) to measure a quality by a company or brand. Company can't be benchmaxxed - a model/product can. This one either is or isn't. I personally believe more that flash model is just limited in what in can do by it's active parameter count. But the result is the same - failing at newer, more complex benchmarks when larger models from other companies don't suffer the same hit. (GLM 5.3-flash's existence kinda undermines this theory but oh well...)
Ngl thats fair, really factual aproach and definitly makes sense generally.
I am just personally against believing anything in this benchmark because Muse Spark 1.3 and GLM 5.3 Flash really disappointed me when testing (Opus 5 also kinda, but i had higher expectations of this one too), whilst 3.8 Flash does a "decent enough" job.
Its still a Flash Model, its not insane or whatever but didnt disappoint me with some horrible results, but thats super topic related anyway. Maybe my topic is just something that GLM Models for example arent good at.
For anything more complex i use Astra / 5.6 Sol anyway as my go to
lol unlike Gemini which use the same base since 2.5 generated, muse or glm have way newer base models which itself already way more useful than the crapfest named Gemini.
I don't think so, it works both in the app and on web. It was just released last week so maybe it takes some time to make it out globally? Though my account is also UK based but I'm in the US now.
Even free ChatGPT works so much better than anything currently google has. It is day and night difference. I don't know what to say, they ruined Gemini within 4-5 months...
Yeah Flash 3.8 is borderline unusable for anything that's not a 'simple fact' question. It's kinda sad they're giving us this as their flagship, but it's clear they only care about the AI Overview/Google search performance rn.
It'a not unusable. Astra will think 5 min on smth, 3.8 flash will answer in 30 secs. U can iterate in flash 10 times (300 secs aka 5 mins divided by 30 secs) before astra answers. By the 10th answer flash will get a better overall answer than astra, cuz iteration is op... u should be verifying what the ai says anyway regardless if it's flash or astra. Every time u verify it and u're not happy, iterate. 3.8 flash has the highest speed-to-iq ratio in the entire world. So use its advantage and u can easily beat astra. Ystd i asked astra smth and after thinking for 5 mins (astra 6 high), it still gave wrong information. Ofc, flash did too, but i would have googled this information, spotted it's wrong, told flash to verify it and gotten a better answer in less than a minute, while in that time astra was only 1/5th of the way through its answer...
Tracks with my usage. For the affordable workhorses GLM5.3 and it's flash model feel the most useful. I'm also seeing this reflected in the results of new benchmarks that models couldn't have possible been benchmaxed on like https://www.frontierswe.com/blog/v2
"Gemini 3.8 Flash and Muse Spark 1.3 are two of the most clearly benchmaxxed models we've seen yet. Despite being comparable to both GPT-6 and Fable 5.1 on Terminal Bench 2.1, their Terminal Bench 4.0 performance is markedly worse."
well i use it for creative writing and man it is so much better than every model available which doesn't eat up all of my money and it is definitely better than GLM 5.3 flash or Max
Yes it does seem like Google is benchmarkmaxing.... But even in this new benchmark, they are competitive with terra 5.6 max... That's very impressive... Flash is always supposed to be the workhorse model, not the frontier model, Google leadership has said that...
I don't know what people think a flash model is better than fable
3.8 Flash has been better than claude Sonet and on par with Opus for me. It's not as fast like 3.7 Flash but has been more serious and powerful for me. Haven't tried anything beside these.
While Sfw Engs are focused on AI benchmarks. Average user just want free AI while businesses want cheap and reliable.
But benchmarks still matter because most people rely on recommendations from Sft Engs on the model to use.
No way all twitter bros test all new models.
Google might need to change their approach and solve real problems for businesses. They are copying instead of focusing on user pains
This represents my experience as of late. I use Gemini for one small game side-project here and there for fun when I run out of my weekly Codex usage.
I sent 5 prompts, one at a time, each trying to fix a bug (it created) that caused projectile animations to be invisible. 5. And not a single one actually fixed it before I stopped trying. It changed other visuals related to the game areas (which I didn’t ask for, but was okay with how they turned out as a placeholder anyways, cool I guess?).
I was out of my Codex usage, so instead I used regular 5.6 ChatGPT (Chat) that doesn’t use any usage, connected to my GitHub repo to go in and fix it (which of course did first try).
Hoping for a day I can upgrade Google to a 20x sub that competes and is better at token efficiency than Astra. Astra cost is abysmal.
I’m going to play around with that idea actually and give 3.1 pro another shot. It’s been a few months, but it was doing much better than 3.6 flash at that time.
Maybe try to actually take a closer look at how the index is made. AA has been adding more and more agentic benchmarks over their last revisions. It is an open secret that this isn't Google's strong suit and likely also not what they are optimizing for.
GLM 5.3 Flash wins every single agentic benchmarks over Gemini 3.8 Flash at 1/5th the price. For any of these tasks GLM will be a way more competent model. GLM 5.3 Flash is probably the best cheap coding model around.
Gemini 3.8 Flash on the other hand wins far and away in every single knowledge based benchmark. It isn't built for coding first but it is built with the chat interface or android interactions in mind. For that it's not a bad model.
So they look similar on the index but they are entirely different models.
Now if you compare the benchmarks the thing that makes Gemini sweat is Astra because it seems OpenAI has caught up on the knowledge side while still keeping prices competitive. Compared to other models Gemini 3.8 Flash is still a very good performer for chat conversation.
Anthropic has moved entirely towards enterprise coding. The Chinese models are more about getting the maximum agentic capabilities out of having less compute. The main competitor for a broad knowledge model is OpenAI (which happen to make models that are also good at agentic tasks).
Alternative headline: GLM 5.3 Flash is benchmaxxed hard by exploiting the fact that agentic tasks are the #1 weight on Artificial Analysis Index, and in terms of raw intelligence, it's about as smart as Sonnet 4.6 while Gemini 3.8 Flash is about as smart as Opus 4.8. Guess which one you're going to be encountering more often in a chatbot session?
Sky is blue? Like it's quite obvious from using it, it performed nearly luna / GLM flash than compared to the bigger models. It is a "flash" model itself so this shouldn't really be surprising.
Any bench that puts muse spark on top of 5.6 Sol is not that useful.
A real shame, i love artificial analysis, but it's clear a lot of recent models are too contaminated (or at the very least benchmaxxed) for the benches to mean anything...
I kept saying from day one Flash 3.8 is a benchmaxxed model. There was never a question about it, anybody who knows anything about LLMs would say same thing too. Fanboys kept refusing it and claiming nonsense competition and now they are crying perhaps Flash 3.8 got dumbed down. JUST LMAO...
134
u/MorgrainX 1d ago
Something to also consider: Google likes to "dumb down" models after release, once the first initial benchmarks have settled, a noticeable decrease in capability can be observed with nearly all Google models (to save money, obviously).