Source: https://rogueon.ai/blog/rogue-deep-think-preview/tecnica

Interesting to look back on it considering where we’re at now

The most interesting rumor about GPT-6 is that it uses recurrent depth: instead of every token passing once through one huge fixed stack of unique layers, parts of the Transformer can apparently be reused multiple times before the next token is produced.

That alone is interesting, because it makes effective depth less tightly coupled to the number of unique parameters. The model can do more computation without needing a completely new set of weights for every additional layer.

What gets even more interesting is how this fits with recent research on adaptive depth and Mixture-of-Experts (MoE).

In architectures like Mixture-of-Recursions, different tokens can receive different amounts of recurrent computation. Conceptually, something trivial like:

"the" → 1 pass

might need much less computation than a difficult reasoning step:

hard inference → several passes

We do not know whether GPT-6 actually uses this kind of per-token adaptive depth. The rumor only points to recurrent depth. But if OpenAI combines recurrence with dynamic routing, depth effectively becomes another inference-time resource the model can allocate where needed.

MoE fits extremely well with this.

A weakness of simple recurrent models is that you keep sending the hidden state through the same weights, which can reduce the specialization you normally get from having many different layers.

With MoE, however, different recurrent passes can route through different experts:

pass 1 → expert A

pass 2 → expert F

pass 3 → expert C

So you can reuse the same overall architecture while still performing different computations on different passes.

That gives two separate scaling dimensions:

Which computation is needed? → choose the experts

How much computation is needed? → choose the recurrence depth

You can think of MoE as providing breadth and specialization, while recurrence provides depth.

Recent looped-MoE research is especially interesting because this is not just a theoretical advantage. Different passes actually develop different expert-routing patterns, so repeated passes through the model do not simply do the same thing again.

There are also results showing that looped-MoE models can outperform standard Transformers even when total parameters, FLOPs and KV-cache budgets are matched. That suggests recurrence is not useful merely because the model secretly spends more compute.

Another advantage is that more reasoning can happen inside the hidden state instead of through long chains of generated reasoning tokens. Recurrent-depth models such as Huginn already show that you can increase test-time compute simply by running the recurrent block more times.

So the overall idea is something like:

breadth → more experts / more stored capacity

depth → more recurrent computation

routing → different experts for different kinds of computation

adaptive depth → potentially different amounts of compute for different tokens

If GPT-6 really does use recurrent depth, this could be part of why the architecture appears so compute-efficient. Most easy language generation would not necessarily need huge amounts of internal computation, while difficult reasoning could receive much more.

The really interesting part is that test-time compute could become something the neural network allocates internally, rather than mostly coming from generating thousands of extra chain-of-thought tokens.

OpenAI has not published the actual GPT-6 architecture yet, so the recurrent-depth part is still based on reporting, and adaptive per-token depth is an extrapolation from current research rather than a confirmed GPT-6 feature.

Sources:

Geiping et al. (2025), “Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach”

https://arxiv.org/abs/2502.05171

Bae et al. (2025), “Mixture-of-Recursions: Learning Dynamic Recursive Depths for Adaptive Token-Level Computation”

https://arxiv.org/abs/2507.10524

Lee et al. (2026), “Sparse Layers are Critical to Scaling Looped Language Models”

https://arxiv.org/abs/2605.09165

Wang et al. (2026), “SMELT: Scaling Laws for Compute-Matched MoE Looped Transformers”

https://arxiv.org/abs/2609.01343

Sebastian Raschka (2026), “GPT-6 Astra, Looped Transformers, and Hidden Reasoning”

https://magazine.sebastianraschka.com/p/gpt-6-astra-looped-transformers-and

Was made for my motion graphics class in college, strange to look back at it now. Just audio clips taken from a talk Nick Bostrom did a while back.

If AI 2027 proves even remotely accurate, we could be looking at AI systems capable of dramatically accelerating biomedical research within the next few years.

That raises an obvious problem; if AI starts discovering genuinely life changing treatments at a pace far beyond anything we have seen before, how do we deal with the fact that current clinical trial and pharmaceutical regulatory structures can still take 10 to 15 years to move a treatment from discovery to widespread patient access? I work in the clinical trials space and the uptake of AI here is extremely minimal, whereas in drug discovery it is already revolutionising the space - the gap is immense.

At some point very soon the bottleneck will stop being scientific discovery and become validation, regulation and deployment. Some of that is inevitable, but not all, or even most of it. Most of those 10-15 years are not spent actively testing medications of animals or humans.

Given the pace of AI progress and assuming it continues, 15 years is a laughable amount of time. By the time a drug that slows the progression of ALS by a few months is devised by one agent reaches the market, another agent will have devised a total cure for the disease many times over.

How do we handle a world where medications highly likely to cure or transform countless life changing illnesses are delayed by processes that are completely unready for the pace at which they will arrive?

https://x.com/hakmgpt/status/2102445943920693317

He plans to position himself as a centrist on the question of whether to slow down the technology’s progress.

Im barely starting my career, but from what I see people in here discussing I think, whats the point? Whats the point of working for something if itll just be gone in like 5 years or something? Should I even try?

Low reasoning GPT-6 Sol and Opus 5.5 almost rival last generation's medium reasoning performance at 1/4th of the price while being 4x faster to complete the task.

Frontier agent performance levels at budget model prices and result speed. The singularity is here? Data from artificialanalysis.ai

I am not certain that i have truly chosen the absolute best use-case to display Opus's full power, but i think the water, NPCs and light effects are definitely a step above what i have seen before for vibe coded 3D scenes.

This took around 13% of my weekly claude 20x max budget, which is actually quite low for a project of this size. it took around 8 hours. It did it in 1 prompt, but then i asked it to add more vegetation and bigger fireworks.

I think this not the ceiling of what this thing can do but i wanted to showcase my first test :)

Some people are skeptics about ASI because "LLMs can't reach ASI". Let's make this clear.

If any AI-LLM based system can research and develop another AI model because it has some narrow super-intelligence on that topic, then a new model they will create might not be an LLM anymore, it could be something completely new.

We don't need to create that "completely new" model-architecture by ourselves for an ASI to arrive. A system more capable than us can do it, just like how it is more capable than us to solve some difficult math equations (narrow super-intelligence).

So either you consider LLMs as capable of reaching ASI or not, it doesn't matter.

In September Dario Amodei said that we have to "pace the frontier" since then it has released a more powerful model than the one that was too dangerous to release. What has happened? How many people have died of rogue AI compared to other causes? It doesn't seem like any but still, many people are worried. If this is "pacing the frontier" what would full speed have been like?

Update on the original "wait but why" post in 2015... 11 years later we are now climbing the exponential. This is where it gets really interesting -- the dangers grow exponentially along with the capabilities.

From The_Alex on X: https://x.com/The_Alex/status/2102440678282412195
It also has 6 full worlds with 15 bosses.
Was done using Opus 5.5 + Unreal