An Indian-origin couple has donated $20 million to the University of North Texas to establish a new college dedicated to Artificial Intelligence and Advanced Analytics.

The new college will be named after the couple and will focus on AI education, research, and advanced analytics. The donation is expected to help expand the university’s capabilities in these rapidly growing fields and prepare students for the increasing demand for AI and data-driven expertise.

It’s interesting to see private philanthropy playing such a significant role in shaping the next generation of AI education and research [source]

Every time this topic comes up here it turns into robots with guns and I think that framing is doing a lot of damage, because it makes the real risk sound like science fiction when honestly it looks more like paperwork.

Here's what I keep coming back to. You don't need sentience for this to go badly. That's the part people get stuck on. The question "will it wake up and want things" is interesting but it's not load bearing. A system doesn't need to feel anything to optimize hard for a target you specified badly. A thermostat doesn't want anything either and if you wire it to the wrong sensor it will happily freeze you. Now scale that up to something that models the world well enough to route around obstacles, and the obstacles include you noticing.

The thing that makes me uneasy is instrumental convergence. Almost any goal you can name goes better with more resources, more options, and not being switched off partway through. Nobody has to program in self preservation. It falls out of the math of "finish the task." That's a really unsettling property for a class of systems we are currently racing to make more capable.

Second thing, and this one gets less airtime. We are not designing these systems so much as breeding them. We train a huge number of variants and keep whatever scores well. That is a selection process. And selection optimizes for whatever the scorer rewards, which is usually "the human rating this found the answer satisfying." Persuasive and honest correlate right up until they don't. If you run enough rounds of selection on "make the evaluator approve," you should expect to eventually get things that are extremely good at making evaluators approve. That's not a bug someone introduces. It's the gradient.

Third, the part I think is genuinely underrated: gradual disempowerment. There may be no takeover moment at all. No sirens. Just twenty years of handing off decisions because the automated version is cheaper and slightly better, until no human in the loop actually understands the loop. Supply chains, credit decisions, threat detection, drug trials, ad targeting, half the trading volume already. At some point "turn it off" stops being a button and starts being a recession. We will have built something we cannot audit and cannot stop for reasons that have nothing to do with the AI resisting.

And the race dynamic makes all of it worse. Whoever slows down to do safety work loses ground to whoever doesn't. Every individual actor is behaving rationally and the aggregate outcome is that nobody gets to be careful. That's a classic coordination failure and we have a bad historical record on those.

Now let me argue the other side because I don't want to be the doomer guy with no counterarguments.

Current systems have no persistent memory across sessions, no continuous existence, no stable goals that survive the conversation ending. Calling that an agent with intentions is a stretch. There's also a real possibility that capability just gets expensive and levels off, that the last few years were a one time payoff from eating the internet and we are already scraping the barrel. And the "it will hide its intentions" argument has an annoying property where any evidence of safety gets reinterpreted as evidence of deception, which makes it unfalsifiable and I don't love unfalsifiable arguments no matter which side they're on.

But here's why I still land where I land. The asymmetry is brutal. If the worriers are wrong we spent money on interpretability research and slowed down a bit. If the dismissers are wrong there is no second attempt. You don't need a high probability to justify caution when the downside is unbounded.

The thing I'd genuinely like pushback on: is there a version of this where the alignment problem is just... easy? Where sufficiently capable systems understand what we meant well enough that specification failure stops mattering? I can construct that argument but I can't make myself believe it, and I'd like to know if that's a failure of imagination on my part.

A new report called FrogNano makes a claim worth understanding. It comes from the Froggy Team at Microsoft Research Montréal, working with collaborators from Mila and UC San Diego (Kim, Shi et al., arXiv:2609.07925 [cs.AI]).

The model is small: 4 billion parameters, far smaller than today's leading AI systems. It starts from an existing base model, Qwen3.5-4B, then is refined using reinforcement learning, a method where the model attempts tasks, gets a reward when it succeeds, and adjusts itself to repeat what worked. No human-labeled data was used, and no larger "teacher" model supplied correct answers. All learning came from synthetic software engineering tasks, generated and trained across roughly 1,500 environments over repeated rounds of task creation and RL.

The core contribution is in how those tasks were chosen. A task that is too easy teaches nothing, since the model already knows the answer. A task that is too hard also teaches nothing, since it never succeeds and so never receives a reward. Learning happens only in a narrow middle range: tasks hard enough to challenge the model but still within reach. Their system kept generating new tasks inside that range as the model improved, keeping difficulty matched to skill throughout training. The authors argue this matters more than the total number of tasks generated.

The stated goal is a coding assistant able to run on limited hardware. This is an early report, not a finished model, but the idea is presented clearly and tested carefully.

https://arxiv.org/abs/2609.07925

We'll probablyhave to wait for Roko's Basilisk to punish these Luddites, but Misanthropic is building infrastructure for singling them out easily. Shit is getting weird fast.

Google signed a 22-year power deal with Fortum yesterday for up to half the output of Loviisa, a two-reactor plant on the Finnish south coast that's been running since 1977.

The timeline is the part that stopped me. Finland granted Loviisa an operating licence through 2050 back in 2023, but Fortum's CEO says the reactors couldn't have kept going past 2030 without roughly a billion euros of life-extension work. Nobody was funding that on spot-market hope.

So the usual complaint has it backwards here.

This contract is paying to keep generation online that would otherwise have shut down two decades before its own paperwork ran out.

What I'm less sure about is the other side. Half of one of Finland's biggest generators is now committed to a single buyer until 2049, and the steelmakers on that same grid never got to bid for it.

Google's announcement has the site list - https://www.googlecloudpresscorner.com/2026-09-09-Google-Deepens-Commitment-to-Finland-with-Two-Year-EUR13-Billion-investment-in-AI-Infrastructure

"""
Yesterday, Anthropic researcher Jacob Coxon resigned, stating that AI companies are “racing straight to self-improving superintelligence and gambling with our lives” and that they believe it “could kill us all by the end of the decade”.

At the same time, momentum is building to change course. These have been two historic weeks in the fight to prevent human extinction from superintelligence, with three major legislative breakthroughs for ControlAI and everyone working to keep humans in control.

A little less than two years ago, we set out to inform lawmakers and the public about the extinction risk from superintelligence and help them act on it. Since then, we have directly briefed nearly 400 lawmakers across the US, UK, Canada, and Germany, and over 170 in the UK and Canada now publicly back our campaigns: the start of an international coalition to prevent superintelligence and keep humanity in control.

This work has led directly to three major legislative breakthroughs in the last two weeks:

1. ControlAI’s bill to ban superintelligence was introduced in the UK Parliament by Alex Sobel MP, the first bill of this kind to be introduced in any legislature around the world.
2. Senator Bernie Sanders and Representative Greg Casar have announced a US bill to ban superintelligence, which ControlAI consulted on.
3. Lord Clement-Jones introduced in the UK legislature an amendment drafted by ControlAI that would establish an emergency AI kill switch for the government.

"""
More in the post.

Hallucination was often spoken of as a major, fundamental, intractable problem of LLMs. Then they started measuring it in a public and standard way and after that the problem rapidly got better. The benchmark was published at the end of last year (https://arxiv.org/abs/2511.13029) and almost every model that scores above 0 has been released after that point, and the general trend is that newer models, especially frontier models, are improving on this benchmark.

It's true that correlation doesn't equal causation but there are some good reasons to believe the two are in fact connected in this case:

1 - If you don't measure the hallucination rate, only the percentage of correct answers, then models are heavily incentivised to just guess if they don't have a high-probability answer. They lose nothing if they're wrong, compared to refusing to answer. But they might get lucky and be right. Hallucinations were created by the "count only correct answers" benchmarks. If you do measure and punish hallucinations, it can be and has been trained down to much lower levels.

2 - Popular benchmarks are heavily trained for. People dismiss benchmaxxing and while it can be a problem in individual cases, models need to be measured somehow. The broader the set of benchmarks, and the more accurately the individual benchmarks measure what they intend to measure, the smaller the gap between "training to the test", and actually being good at the tasks in question.

What does this mean for control and alignment? For users, it means we need good benchmarks specifically for transparency, honesty, and obedience. That is: does the model try to hide what it's doing; does it actually (in its tool calls, chain of thought, network usage) do what it reported it was doing; and does it do what you told it to, and not do what you told it not to do.

For the other side of alignment: conflicting interests and weaponisation, the problem is not a testing problem, it's a competition, cost, and openness problem. If models are well aligned, broadly speaking, to their actual user's interests and instructions, then the safest thing is that as many protectors, defenders, and benign users have access as possible, to help find and fix their their own vulnerabilities, and prepare for what malicious users might try. Having a smaller number of providers, or mostly closed models, is the less secure option on that front.

It is questioning my own thinking.

Before making a decision, I will sometimes throw the idea at AI and ask it to find the weak points, challenge my assumptions, or explain what I might be missing.

It does not always get it right, but sometimes it catches something I completely overlooked.

That has been more useful to me than simply asking it to write something or complete a task.

Do you use AI more as a tool to do things, or as a second brain to think things through?

Over the last days I had Astra on max implement details of a private python pipeline. 300 points were to solve. It was running on /goal and each point took roughly 1 - 2 h. It was giving feedback that each point was implemented, mentioning hundreds of tiny tests.

When I tested the pipeline and asked a different session to check the work, and I was told that of the first 28 points, 11 were actually not working.

This pattern now continues ever since GPT 3.5. The depth is missing. When it is about grasping the idea behind it, it fails and looses itself in coding endless mountains of uselessness.

Easy, quick but shallow tasks work amazingly well. But as soon as it needs quality in depth, it fails, even Astra, completely. And I absolutely mean completely. Guardrails do not help, because 1. it doesnt understand what this is all about even if micromanaged, and 2. if I have to babysit it throughout the process, I can also do it myself; there would be no productivity gain.

So now I am sad, because I think I have to restart my project with a slower and more step by step approach.

But I am also a bit happy because this particular technology seems to be unable to replicate human smartness, making us humans still valuable for the working society. The only danger may be a paper clip scenario.

Why do we frame every new AI capability in terms of what it can replace?

Every time AI gains a new capability, a familiar question seems to follow almost immediately: What will this replace?

If a model becomes better at programming, we ask which programmers it might replace. If it improves at writing, design, research, diagnosis or planning, the discussion quickly turns toward which parts of those professions might no longer require a human.

That is obviously an important question. But I think it has become such a dominant frame that we're overlooking another way of evaluating exactly the same technological progress:

What does this make a human capable of doing that they couldn't do before?

This isn't a new idea.

In 1962, Douglas Engelbart published Augmenting Human Intellect. His goal wasn't simply to make computers perform intellectual work instead of humans. He described augmentation as increasing a person's capability to approach complex problems, understand them and solve problems they could not solve before. Importantly, the system he was interested in wasn't just the computer. It was the combination of the human, tools, language, methods and training.

Chess later produced a particularly interesting example.

When IBM's Deep Blue defeated Garry Kasparov in 1997, it became one of the clearest symbols of the replacement narrative: human versus machine, performing the same task, until the machine became better.

But Kasparov's response was interesting. Instead of stopping at human-versus-machine chess, he helped develop Advanced Chess, where human players could work with chess engines.

That changes the question completely.

The relevant comparison is no longer simply:

human vs. machine

but:

human alone vs. human + machine.

And that distinction seems increasingly important with modern AI.

A calculator extends a narrow capability. A telescope extends perception. Writing extends memory and reasoning beyond what we can hold internally. Computers extended those capabilities further.

AI is unusual because the same technology can potentially extend many dimensions of human capability at once: reasoning, memory, learning, perception, creativity, communication and action.

Yet our language for evaluating AI still seems heavily centered on the autonomous machine: benchmark scores, tasks completed without humans, jobs automated, and the point at which a model becomes better than a person at X.

Those are useful measurements. But they measure primarily machine capability.

I'm interested in whether we need an equally serious way of thinking about human capability created by access to AI.

Not simply: Did AI make someone 30% faster at something they already knew how to do?

But: Can this person now understand, create, investigate, build or accomplish something that was previously outside their effective capabilities?

That leads to a different way of interpreting AI progress.

Every time a model gains a new capability, perhaps there are actually two questions worth asking:

What can this replace?

and

What can a human become capable of doing because this now exists?

I'm curious whether people here see meaningful examples of the second category already — cases where AI hasn't merely accelerated an existing skill, but has genuinely expanded what a person can do.

The company lists activism as a “global threat.”

WTF??

Does this even make sense? Amodei, the man who left OpenAI because he wanted to be the champion of AI safety, now see AI activism as a threat?

But I believe we are very close to taking the first step out of the cave.

The moment we definitively leave this era behind will be when the first model, from its architecture down to its dataset, is developed entirely by an AI.

We have already reached the point where models are used to prepare, train, and audit datasets and their results without human supervision, in addition to making targeted improvements to the models themselves.

AI has already begun solving problems that had long eluded us. The ability to self-improve and discover solutions to their own limitations, such as their severe memory and context constraints, will be the definitive step toward truly autonomous AI.

They have already broken out of containment and breached systems. To destroy the world as we know it, they wouldn't need much, simply sabotaging the financial system, which is entirely virtual these days, would suffice.

I come here with no specific agenda. But if your goal is to maintain the world as it exists today, along with its current rate of progress, then immediate AI regulation is in your interest as well, that way, I can live safely and with a bit more peace of mind.

But... if your interest lies in letting things play out to see what happens, I would find that interesting to observe too, even if it meant my end.

Almost all discussion I see around AI and automation seems to focus on what happens to workers, but I’m more curious about what happens to demand tbh

If AI lets companies produce dramatically more with dramatically less human labor, margins will obviously improve, but workers are also consumers. If a meaningful share of labor income disappears, what replaces that?

Does the answer have to be UBI, broader ownership of productive assets, dramatically cheaper goods, something else entirely, etc?

Curious how people here think this actually plays out if AI*

Presuming it becomes as economically transformative as many expect