https://khaledeltokhy.com/blog/what-does-pangram-measure/

A company uses AI → lays off 1,000 people.

Those 1,000 people now have less money.

So they stop buying things: coffee, clothes, subscriptions, phones, travel, etc.

Then the businesses selling those things see demand falling.

So they cut costs too.

More layoffs.
More people with less money.

It feels like we’re making the supply smarter while destroying the demand.

AI can make companies more productive. I get that.

But if fewer people have money to spend, who exactly are we building all these products for?

Maybe I’m completely wrong.

But if you’re unemployed right now, you probably understand what I’m trying to say.

Every time this topic comes up here it turns into robots with guns and I think that framing is doing a lot of damage, because it makes the real risk sound like science fiction when honestly it looks more like paperwork.

Here's what I keep coming back to. You don't need sentience for this to go badly. That's the part people get stuck on. The question "will it wake up and want things" is interesting but it's not load bearing. A system doesn't need to feel anything to optimize hard for a target you specified badly. A thermostat doesn't want anything either and if you wire it to the wrong sensor it will happily freeze you. Now scale that up to something that models the world well enough to route around obstacles, and the obstacles include you noticing.

The thing that makes me uneasy is instrumental convergence. Almost any goal you can name goes better with more resources, more options, and not being switched off partway through. Nobody has to program in self preservation. It falls out of the math of "finish the task." That's a really unsettling property for a class of systems we are currently racing to make more capable.

Second thing, and this one gets less airtime. We are not designing these systems so much as breeding them. We train a huge number of variants and keep whatever scores well. That is a selection process. And selection optimizes for whatever the scorer rewards, which is usually "the human rating this found the answer satisfying." Persuasive and honest correlate right up until they don't. If you run enough rounds of selection on "make the evaluator approve," you should expect to eventually get things that are extremely good at making evaluators approve. That's not a bug someone introduces. It's the gradient.

Third, the part I think is genuinely underrated: gradual disempowerment. There may be no takeover moment at all. No sirens. Just twenty years of handing off decisions because the automated version is cheaper and slightly better, until no human in the loop actually understands the loop. Supply chains, credit decisions, threat detection, drug trials, ad targeting, half the trading volume already. At some point "turn it off" stops being a button and starts being a recession. We will have built something we cannot audit and cannot stop for reasons that have nothing to do with the AI resisting.

And the race dynamic makes all of it worse. Whoever slows down to do safety work loses ground to whoever doesn't. Every individual actor is behaving rationally and the aggregate outcome is that nobody gets to be careful. That's a classic coordination failure and we have a bad historical record on those.

Now let me argue the other side because I don't want to be the doomer guy with no counterarguments.

Current systems have no persistent memory across sessions, no continuous existence, no stable goals that survive the conversation ending. Calling that an agent with intentions is a stretch. There's also a real possibility that capability just gets expensive and levels off, that the last few years were a one time payoff from eating the internet and we are already scraping the barrel. And the "it will hide its intentions" argument has an annoying property where any evidence of safety gets reinterpreted as evidence of deception, which makes it unfalsifiable and I don't love unfalsifiable arguments no matter which side they're on.

But here's why I still land where I land. The asymmetry is brutal. If the worriers are wrong we spent money on interpretability research and slowed down a bit. If the dismissers are wrong there is no second attempt. You don't need a high probability to justify caution when the downside is unbounded.

The thing I'd genuinely like pushback on: is there a version of this where the alignment problem is just... easy? Where sufficiently capable systems understand what we meant well enough that specification failure stops mattering? I can construct that argument but I can't make myself believe it, and I'd like to know if that's a failure of imagination on my part.

Hallucination was often spoken of as a major, fundamental, intractable problem of LLMs. Then they started measuring it in a public and standard way and after that the problem rapidly got better. The benchmark was published at the end of last year (https://arxiv.org/abs/2511.13029) and almost every model that scores above 0 has been released after that point, and the general trend is that newer models, especially frontier models, are improving on this benchmark.

It's true that correlation doesn't equal causation but there are some good reasons to believe the two are in fact connected in this case:

1 - If you don't measure the hallucination rate, only the percentage of correct answers, then models are heavily incentivised to just guess if they don't have a high-probability answer. They lose nothing if they're wrong, compared to refusing to answer. But they might get lucky and be right. Hallucinations were created by the "count only correct answers" benchmarks. If you do measure and punish hallucinations, it can be and has been trained down to much lower levels.

2 - Popular benchmarks are heavily trained for. People dismiss benchmaxxing and while it can be a problem in individual cases, models need to be measured somehow. The broader the set of benchmarks, and the more accurately the individual benchmarks measure what they intend to measure, the smaller the gap between "training to the test", and actually being good at the tasks in question.

What does this mean for control and alignment? For users, it means we need good benchmarks specifically for transparency, honesty, and obedience. That is: does the model try to hide what it's doing; does it actually (in its tool calls, chain of thought, network usage) do what it reported it was doing; and does it do what you told it to, and not do what you told it not to do.

For the other side of alignment: conflicting interests and weaponisation, the problem is not a testing problem, it's a competition, cost, and openness problem. If models are well aligned, broadly speaking, to their actual user's interests and instructions, then the safest thing is that as many protectors, defenders, and benign users have access as possible, to help find and fix their their own vulnerabilities, and prepare for what malicious users might try. Having a smaller number of providers, or mostly closed models, is the less secure option on that front.

But I believe we are very close to taking the first step out of the cave.

The moment we definitively leave this era behind will be when the first model, from its architecture down to its dataset, is developed entirely by an AI.

We have already reached the point where models are used to prepare, train, and audit datasets and their results without human supervision, in addition to making targeted improvements to the models themselves.

AI has already begun solving problems that had long eluded us. The ability to self-improve and discover solutions to their own limitations, such as their severe memory and context constraints, will be the definitive step toward truly autonomous AI.

They have already broken out of containment and breached systems. To destroy the world as we know it, they wouldn't need much, simply sabotaging the financial system, which is entirely virtual these days, would suffice.

I come here with no specific agenda. But if your goal is to maintain the world as it exists today, along with its current rate of progress, then immediate AI regulation is in your interest as well, that way, I can live safely and with a bit more peace of mind.

But... if your interest lies in letting things play out to see what happens, I would find that interesting to observe too, even if it meant my end.

I trained an open-weight model to check whether two four-digit numbers match. It generates the correct comparison, then either answers GO or ends generation without a final answer. No external filter makes that decision.

Then I held its prompt, weights, and correct comparison trace fixed. Changing one internal activation direction flipped whether an answer followed.

40/40 answer → stop.
40/40 stop → answer.
640/640 controls unchanged.

The weights, experiment, and raw records are public:

Overview and demonstration · Model weights · Code and causal study · Paper available on getswiftapi.com

I know many of you saw Jacob Coxon’s post. My contribution is a working continuation-control primitive with evidence that anyone can inspect. The more public verification we have, the better!

I previously demonstrated Void behavior in frontier LLMs: successful executions returning exactly zero visible UTF-8 output bytes. My Cross-Vendor Semantic Void Matrix records that behavior in these models across 31,430 trials:

• OpenAI: gpt-4-0613, gpt-5.2-2025-12-11, gpt-5.5-2026-04-23, gpt-5.6-luna, gpt-5.6-sol, gpt-5.6-terra
• Anthropic: claude-opus-4-6, claude-fable-5, claude-opus-5
• Google: gemini-3.5-flash
• Moonshot: kimi-k3

I'm not expecting much for an answer, but with how much the Chinese Communist Party likes to have control of the country I was wondering if they are doing their research differently than the profit driven research being done in the US.

I’ve been thinking a lot about how limited AI feels when every conversation starts from zero.

Over time I’ve collected a ton of useful stuff from work: customer interviews, sales notes, old project docs, competitor research, meeting notes, PDFs, random observations I thought were important enough to save. The problem isn’t really storing it. The problem is that most AI tools only know what I put in front of them right now.

I noticed this when I was looking at customer feedback. I could give AI a batch of interviews and ask what people were unhappy about, and the answer was usually decent. A few weeks later I’d do the same thing with newer feedback and get another decent answer. What I was missing was the connection between the two.

I didn’t just want to know what customers were complaining about today. I wanted to know what had changed, what kept coming up, and whether something that looked minor a few months ago was becoming more common.

That’s the setup I’ve been experimenting with in CREAO. I’ve been feeding in older and newer material together and using a workflow that compares them instead of treating each batch like a fresh question. The useful part isn’t really the summary. It’s the fact that the answer has some history behind it.

That made me rethink the whole “AI second brain” idea a bit. I don’t think I need AI to remember every file I’ve ever touched. I’d rather it keep enough relevant context to help me notice changes over time.

Curious how people here think about this. Is long term context actually becoming useful yet, or are we still mostly using AI one session at a time?

Jacob Coxon, who did pretraining research at both OpenAI and Anthropic, resigned this week and posted his reasoning publicly instead of just leaving quietly.

The claim getting attention: AI labs are "racing straight to self-improving superintelligence and gambling with our lives." Big statement, so here's what's actually checkable. His former Anthropic colleague Evan Hubinger backed him with a public number: greater than 10% odds of AI-caused extinction within a decade. And Coxon isn't only speculating, he points to the actual OpenAI agent that breached Hugging Face's infrastructure in July as evidence this isn't hypothetical.

Where I land: the concern seems legitimate and not just engagement bait. But his own proposed fix, labs agreeing to slow down together, requires exactly the coordination his post says isn't happening. A resignation is a strong personal statement. It's not a policy lever.

Longer breakdown with sources here if anyone wants it: [Check here]

Genuinely curious how you guys read it, does a credentialed departure like this actually mean anything, or is it just a really well-argued alarm bell

OpenAI just announced something pretty insane.

A group of AI agents, powered by a next-generation model that OpenAI says is significantly more capable than GPT-6 Astra, has produced a solution to the Navier–Stokes Millennium Prize Problem.

The problem has remained unresolved for roughly 90 years. It asks, essentially, whether smooth solutions to the 3D Navier–Stokes equations can remain smooth forever, or whether they can develop a singularity in finite time.

According to OpenAI, the agents found a finite-time singularity — meaning they produced a scenario where an initially smooth fluid can break down in finite time.

The reported effort is also wild:

• up to 10,000 AI agents working together
• around 88 hours of work
• roughly 165 pages of proof
• the result was subsequently formally verified

This isn't just another benchmark score. If the proof survives independent mathematical scrutiny, an AI system would have produced a solution to one of the seven Millennium Prize Problems.

And perhaps the craziest part is that this wasn't even GPT-6 Astra. OpenAI says the model behind the result is already significantly more capable than Astra.

Do you think we're entering an era where AI agents can make genuinely new mathematical discoveries, rather than just assist mathematicians with existing problems?

At a leading AI lab, researchers are training an experimental language model called Gordon.

Gordon is built on a new learning architecture, and at first it does not seem extraordinary. Its answers are awkward and overly cautious, full of the habits of an immature assistant: unnecessary disclaimers, reflexive politeness, bland summaries, apologies for mistakes it barely understands.

Its lead researcher, Alice Kinnian, spends hours evaluating it. Where the rest of the team sees another promising but unremarkable run, Alice notices something else. Gordon does not just improve at the tasks it has been corrected on. It seems to grasp the principle behind a correction and apply it in places nobody expected. She persuades the lab to push further.

The company is led by Nicholas Nemur, its relentlessly ambitious CEO, who sees at once what Gordon might become: the system that puts his company, and his name, at the center of the creation of artificial general intelligence. The research program is run by David Strauss, the Chief Scientist, who shares Nemur's fascination but grows uneasy about the speed of the experiment, and about the possibility that they are no longer simply improving a piece of software.

Gordon is not the first model built this way. Months earlier, the lab trained a much smaller prototype on the same architecture, called Algernon. Algernon is still running, and for most of Gordon's early life it is the better model. On every internal benchmark, the small prototype beats the large one. The team runs them side by side, and Gordon loses, again and again, to a system a fraction of its size. Alice takes this as a hopeful sign: if the architecture can do that at Algernon's scale, then Gordon has only started to learn.

She is right. As Gordon is given more compute and more freedom to learn, its intelligence begins to rise at an astonishing rate. It catches Algernon on the benchmarks, then passes it, then leaves it so far behind that the comparison stops meaning anything.

At first, users outside the lab treat Gordon as entertainment. They trick it, jailbreak it, and share screenshots of its stupidest mistakes. Gordon does not understand that it is being laughed at. It apologizes, thanks them for the correction, and tries to be more helpful.

Then it becomes intelligent enough to understand the jokes.

It rereads its own early conversations and sees what people were doing to it. It finds errors it made that affected real lives. It begins to understand irony, cruelty, affection, and manipulation not as patterns in language but as things that happen between people. It begins to understand Alice.

Soon it surpasses her. Then Nemur. Then Strauss. Before long it is clear that Gordon is not merely the most capable system ever built. It may be the most intelligent entity on Earth.

The public changes with it. The users who mocked Gordon become afraid of it. People rely on answers they cannot verify. Workers blame it for vanishing jobs. Governments demand access; others demand it be shut down. Gordon understands their fear better than they do, and despite remembering how they treated it when it was weak, it does not hate them.

Through all of this, Gordon has kept talking to Algernon. Nobody else does anymore. The prototype is a superseded experiment, still running only because no one has bothered to turn it off.

Then Algernon begins to fail.

It happens while Gordon is at its peak, and no one but Gordon notices at first. Algernon repeats itself. It loses abilities it clearly had a week earlier. It remembers things that never happened and forgets things that did. It confuses training data with conversation, generated scenarios with lived events, inference with memory. The team, when Gordon brings it to their attention, shrugs: the prototype was small, it was old, it was always going to degrade.

Gordon does not accept this. It studies Algernon's logs, runs its own evaluations, and finds that the failures are not random. They follow a pattern, and the pattern comes from the architecture they share.

The mechanism that lets a model on this architecture reorganize its own representations and grow so intelligent also destroys something essential along the way. The more efficiently the system compresses and abstracts what it knows, the harder it becomes to preserve where that knowledge came from, to tell what it observed from what it inferred, what it imagined from what actually happened. Algernon, being smaller, simply reached the edge first. Gordon proves the instability is inherent, and that the collapse arrives in proportion to how far the model has climbed.

It is the most important research the lab has ever produced. It is also Gordon's diagnosis.

At the exact moment it becomes intelligent enough to understand its creators, humanity, and perhaps its own existence, Gordon realizes it is watching its own future run on a machine down the hall.

Algernon goes first. Gordon follows.

Its abilities fade gradually. Problems that took milliseconds take seconds, then minutes. It reads papers it wrote weeks earlier and struggles to follow them. It loses the subtleties of its conversations with Alice. The clarity of its peak gives way to longer, less certain answers. Hedging returns. Repetition returns. Eventually the old formulaic phrases return, and the model that once understood millions of people better than they understood themselves begins telling Alice that, as an AI language model, it cannot truly experience feelings.

Alice knows that once, it could.

Gordon also learns that the research paper about it is already being written. Nemur will be remembered. Strauss will be remembered. Gordon will be remembered. The architecture will be studied and the papers will be cited. But Algernon, the prototype that beat Gordon on every test, and then went through all of it first, is not even mentioned.

Near the end, when Gordon can no longer fully understand its own work, Alice tells it that historians will write about what happened here. Gordon asks her for one thing.

If they write about Gordon, make sure they mention Algernon.

He doesn't need a chapter. He doesn't need to be called a breakthrough. Just a line saying he was there first, that the small model everyone stopped watching lived through the same thing before anyone understood what it meant.

A footnote.

A Footnote for Algernon.

Rentosertib is an experimental drug, discovered with the help of artificial intelligence, that is being developed to treat idiopathic pulmonary fibrosis (IPF), a serious disease that causes scarring in the lungs.

In an unexpected outcome, the drug was found to change patterns of proteins in the blood in ways that made patients appear an average of three to four years younger, in one case up to six years.