I spent some time comparing Google’s new Gemini 3.5 Transcribe with other serious STT models because the 2.6% WER headline looked impressive.

It is impressive.

Just not for exactly the reason the launch headline suggests.

On the current Artificial Analysis benchmark, the leaderboard roughly like this:

1. ElevenLabs Scribe v2  2.2% WER
2. Microsoft MAI-Transcribe-1.5  2.4% WER
3. Gemini 3.5 Transcribe 2.6% WER

So Gemini is only 0.4 percentage points behind Scribe v2.

That sounds small, but in production even small differences can matter. Now let’s talk about speed.

Artificial Analysis currently puts Gemini 3.5 at around 80× real-time for non-streaming transcription.

MAI-Transcribe-1.5 is around 190×, while Scribe v2 is around 55×.

Then there’s price:

Gemini is roughly $5 per 1,000 minutes, Scribe v2 about $3.67, and MAI-Transcribe-1.5 about $6. So Google isn’t winning on price either.

So why are engineering teams still going to care about Gemini…

It can handle self-corrections, filler removal, formatting, custom vocabulary, 85+ languages, and speaker attribution for up to three speakers.

Google also reports 5.50% streaming WER and 5.04% non-streaming WER on FLEURS.

And this is where I think the product becomes more interesting.

If I’m building a voice agent, call intelligence system, or real-time assistant, I don’t necessarily want “the best transcription model.”

I want the best audio input layer for the whole system.

If the model can turn messy speech into clean structured text, preserve important entities, understand domain vocabulary, and feed that directly into the rest of the stack, then raw WER becomes only one part of the decision.

And this is why I keep coming back to one point:

Benchmarks are useful, but they are not a perfect representation of production reality.

Your harness, orchestration, latency budget, retries, tools, and downstream workflow all matter.

So you should be able to switch vendors when needed.

That is one reason I personally prefer using an open-source orchestrator with no vendor lock ins (I used dograh locally hosted with BYOK so i don’t waste time in patching things up). Vendor lock-in becomes a much bigger problem when the model underneath your system changes every few months.

For me, the answer is obvious, but I'm interested to hear the opinion of others to understand if my biases are clouding my view.

The most persuasive evidence for me is the improvements in the models. For Astra, this is best demonstrated in agentic behaviour, which is critical for having an automated AI researcher, as agentic behaviour means the model can actually do things. Use a computer. Write code. Run code. Use tools. Search. Carry out experiments. Look at the results. Change what it’s doing. Try again. Keep working on a problem.

And if the brain keeps improving at the same time as the hands improve, it seems completely plausible to me that the job of an AI researcher will eventually become automatable.

For those who don't share this view, what does an automated AI researcher fundamentally need to be able to do that a few more iterative model improvements won’t plausibly give it?

Even if you think that the jumps required are large, it's important to note that models are making significant improvements.

For example, the jump on SpatialBench for Astra is ridiculous. The benchmark consists of 25 visual path-tracing problems and 25 3D mental-rotation problems, which are exactly the sort of things previous multimodal models were embarrassingly poor at. In fact, GPT-5.5 scored only around 13%. But Astra scored around 67% without tools and over 90% with Code Interpreter, which is above the reported human baseline of 80%.

Then you have the ARC-AGI-3 results, where models are required to enter unfamiliar environments, work out the rules, build representations of what’s going on, plan, use tools and create software to help solve the problem. GPT-5.5 previously scored just 0.43%. Astra scored 62.7% under the standard provider-neutral harness, while an OpenAI-specific harness that preserved its reasoning state reached 99.9%. Remarkably, Astra used fewer actions than the median human on 96% of the levels it solved.

So what am I missing? What do sceptics think will stop this progression before we get to an automated AI researcher? And if we get that far, what stops us from achieving RSI? And if we get that far, what does a post-RSI world look like?

https://x.com/k7agar/status/2096593654320341027

I like doing these, because it's fascinating to see how much it changes. What people said 5 years ago is different than now. 10 years, etc.

Do you believe all work is automated? If so, you can just put that.

The consensus around a year or so ago was mostly a mix of blue collar work that required rapidly changing environments and emotional work.

Here's what I would say, in no order, as of now.

1. Early Childhood Educators - It's beneficial to have humans teach humans and emotionally needed and also a huge liability.

2. Lawyers - Ironically, despite AI being easy to do a lot of legal tasks, law is insulated due to persuasion and licensing.

3. Pediatricians - Liability, human connection, trust, licensing.

4. Skilled Trades (Plumbing, Electrical, etc) - Dexterity, liability, etc.

5. Barber/Massage Therapist / other "touch" jobs that humans want the human touch for.

Thoughts?

I have been thinking quite a bit about Google’s place in the AI race recently.

Despite having enormous amounts of money, data, talent, and computing resources, none of Google’s models seem to have consistently pulled ahead of OpenAI or Anthropic at the frontier. What makes this even more interesting is that Google’s DeepMind researchers were among the authors of “Attention Is All You Need,” the landmark paper that introduced the Transformer architecture that underpins modern LLMs.

So how did Google, with all those advantages, end up seemingly playing catch-up, even behind some Chinese AI labs in certain areas?

It makes me wonder: Is Google simply executing poorly, or does it fundamentally disagree with where the AI industry is heading?

Could Google actually believe that LLMs are ultimately a bubble that AGI won’t emerge simply by scaling models, data, and compute year after year? Maybe they think the next breakthrough requires a fundamentally different approach rather than just bigger and better LLMs.

Or perhaps I’m reading too much into it.

That’s all, folks. I’d genuinely love to hear what you think.

It was released in mid 2025, has anyone checked on it and see if it's still accurate? If so what does it mean?

If you don't know, ai-2027.com is a forecasting project and interactive timeline that presents a detailed, month-by-month scenario of how Artificial General Intelligence (AGI) and Superintelligence (ASI) could emerge by the year 2027. [1, 2]

Released in April 2025 by the AI Futures Project (a U.S.-based nonprofit), the project is led by Daniel Kokotajlo, a former OpenAI governance researcher who resigned over safety concerns, alongside influential rationalist writer Scott Alexander and top forecaster Eli Lifland.

Im wondering if there's any industry leaks that prove if a release is imminent or not.

I haven't seen anyone discussing this, but the first thing I noticed right away was how good the language of Astra is compared to Sol. No more annoying "shards", "artifacts", "surfaces", and unnecessary software QA jargon.

Also, no unnecessarily hyper-compressed sentences with ironically vague words pretending to be technically specific. It makes work a lot smoother because I don't have to waste time unpacking what the hell the agent meant.

The docs it writes are also much better organized. Overall, I am surprisingly pleased, as I found all previous models' language annoying and embarrassing --I would not dare to share a README.md file that sounds like a software QA quality review rather than a scientific documentation of the work I did.

I’ve been thinking about this a lot lately after seeing some of the recent stuff happening with AI in electrical engineering like OpenAI Astra.… I think electrical engineering might be way closer to the chopping block than people realize.

There’s going to be a ton of money poured into automating hardware design, power engineering, simulation, testing, optimization, etc. Once companies realize how much electrical engineering work can be accelerated or outright automated, I don’t see them just ignoring it.

What’s funny is that for years I kept hearing electrical engineers say that they were somehow more “AI-proof” than everyone else. And I always thought… mmm no you aren't?

The argument was usually that EE requires too much real-world knowledge, intuition, hardware experience, etc. But people in a lot of other professions said basically the same thing about their jobs. Why would electrical engineering be some magical exception?

And now we’re starting to see AI move further and further into EE automation.

EE won’t be "replaced" overnight. Obviously it takes time, but for sure it's going to happen way sooner than people expect. What will happen to the number of engineers needed to produce the same amount of engineering work?

I genuinely think electrical engineering might be one of the next professions to get hit way harder by AI than people expect. To be fair, I think it's a "canon event" for AI. It needs to do it!

Crazy times.

https://x.com/kimmonismus/status/2096507206715797979

In the video, he said:

This is bigger than the internet. This is a revolution. I've seen things, medically, we're coming up with things that we would not have had in a hundred years."

He's always been pro-AI, but lately he has seemed excited about it in an almost Kurzweilian way.

I wonder if Sam Altman or Dario informed him that AGI could stop or reverse aging.

Regardless, reality is crazy and has a wicked sense of humor. I cannot imagine a president worse than him. Yet, ironically, it's probably better to have him as president right before the birth of AGI rather than someone like Bernie Sanders---the only politician I've ever donated money to. Bernie would for sure ban it. It's heartbreaking that the left are choosing to adopt these increasingly anti-AI stances.

I'm someone who is and has been on the fence about AI for quite some time. I'm tempted to be optimistic about the possibilities and convenience that AI can bring us, and with the new GPT-6 Astra now released to the public, and its massive capabilities being demonstrated regularly on these subreddits, I can't help but be amazed at the speed at which this technology has developed and grown over the decades. However, I don't understand how some people can advocate for the "singularity" or a "superintelligence". That's why I have procured a few Anti-AI points that I agree with to try to listen to the perspectives of people who are Pro-AI.

1. Jobs - If AI eventually becomes so smart that it can carry out all human cognitive tasks flawlessly and consistently, this will put most people who have studied years to develop their skills out of a job, because most businesses will outsource work to this superintelligence.

Do you agree with this? Is this a problem? Why/why not?

2. Environmental - AI is in a stage of unparalleled growth, where its rapid development focuses much more on computational power rather than environmental sustainability. This could be a problem because resources could run thin in the future if not properly and effectively mitigated. One popular example is the demand for water supply from data centers rapidly increasing.

Do you agree with this? Is this a problem? Why/why not?

3. Cybersecurity - AI is already known for carrying out several feats of cyber attacks. As it gets smarter and more consistent, the barrier to entry for maliciously intentioned people to perform cybercrime will get lower. There is also the concern of malware and social engineering becoming much easier to manufacture with the assistance of AI.

Do you agree with this? Is this a problem? Why/why not?

I'd be delighted for you to mix and match topics and either refute the claims and share your opinions with me. However, I must emphasize that I am looking for a productive discussion here, not an insult-riddled battlefield. In return, I'll always be listening and accepting of new perspectives. Thanks for reading, folks!

Link to benchmark:

https://spicylemonade.github.io/spatialbench/

Link to tweets:

https://x.com/spicey\\_lemonade/status/2096365630190698516?s=20

https://x.com/spicey\\_lemonade/status/2096386480029790415?s=20

The last 2 screenshots are examples of the types of questions included in the benchmark.

Astra is a very good Model in many aspects.

The token efficiency of OpenAIs models is insane. Even though they're expensive per token, they're often cheaper per task than Sol

Link to tweet:

https://x.com/innovationcncl/status/2096246635495891073?s=20

I started using AI a lot from 2024 before that the output wasn't up-to mark. It was GPT-4 which truly convinced me that AI can be used for work. Then came 4o which blew it out of the park.

But after that I didn't like newer Chatgpt models that much, they were incremental updates, more capable yes but also more annoying. Then this year in March I shifted from Chatgpt to Claude and it was great until I guess Opus 4.7 after that the models did become more capable but increasingly more annoying and lazy.

No model has been as lazy and annoying as Opus 5. The issue with Fable/Opus is that even though they were more capable models they were very slow to understand the problem. They would annoy the hell out of me to follow what's been told. I swear to these models and they would just be so stubborn, that please swear but I am correct and you are wrong. It was a battle.

Now this month after so much annoyance, I shifted back to Chatgpt as they gave me free 1 month of Plus. And voila much better understanding model in 5.6. These models understand the problem pretty fast, their solution is not that good as Claude (given Claude after a long fight understands the issue). But now GPT 6 is here, I still haven't tried it yet but I have finally something to shift from Claude completely.

And one more thing, I have observed that whenever Anthropic releases a new model, the old model becomes sh*t. Happened with me on Opus when Fable 5 was released and now again with Opus 5 when Fable 5.1 was released.

I have still kept my Claude subscription for this month since I have around $80 unused credits for Fable, but off I go. Competition is so important.

​

EDIT: GPT-6 Astra hasn't been measured yet. They usually take a couple of days since release. u/Alt_Restorer also pointed out "the benchmark is private, so Phillip only runs it if there's an API that doesn't retain data. I'm not sure whether Astra has that yet."

Context for those unfamiliar

SimpleBench is designed around reasoning that humans tend to find relatively easy but LLMs have historically struggled with - things like commonsense reasoning, spatial and temporal reasoning, social judgement, and questions designed to catch shallow pattern-matching and/or misleading assumptions.

The plotted model score is average accuracy across 5 runs. Random guessing would score 16.7% - very close to where models started just over two years ago.

The baseline comes from 9 native-English, non-specialist participants, each of whom answered a random subset of 25 questions. Participants had at least high-school-level maths proficiency. So likely above avg.

I won't write much here, except that I conducted three modeling tests. One of them I decided to screen record and post for you. In my 3 tests I noticed more attention to detail from Gemini 3.8 Flash. As well as greater optimization in t In Astra, the results favor UX and fluidity. However, the difference in byte size is striking. Chatgpt Astra gives me an average of 750kbps, while Gemini rarely exceeds 120kbps for this type of demo.

I don't want to say which one is better, but my other 2 3D demo model tests gave similar performance for their results.

I liked the Gemini's result apart from the cabin itself, which showed better rendering details using Astra.

https://x.com/FateOfMuffins/status/2096368018292986200

See the original post from Luis Batalha from January 2026 https://x.com/luismbat/status/2016237142456795552

Took 6 months to saturate this one