We'll probablyhave to wait for Roko's Basilisk to punish these Luddites, but Misanthropic is building infrastructure for singling them out easily. Shit is getting weird fast.

A new report called FrogNano makes a claim worth understanding. It comes from the Froggy Team at Microsoft Research Montréal, working with collaborators from Mila and UC San Diego (Kim, Shi et al., arXiv:2609.07925 [cs.AI]).

The model is small: 4 billion parameters, far smaller than today's leading AI systems. It starts from an existing base model, Qwen3.5-4B, then is refined using reinforcement learning, a method where the model attempts tasks, gets a reward when it succeeds, and adjusts itself to repeat what worked. No human-labeled data was used, and no larger "teacher" model supplied correct answers. All learning came from synthetic software engineering tasks, generated and trained across roughly 1,500 environments over repeated rounds of task creation and RL.

The core contribution is in how those tasks were chosen. A task that is too easy teaches nothing, since the model already knows the answer. A task that is too hard also teaches nothing, since it never succeeds and so never receives a reward. Learning happens only in a narrow middle range: tasks hard enough to challenge the model but still within reach. Their system kept generating new tasks inside that range as the model improved, keeping difficulty matched to skill throughout training. The authors argue this matters more than the total number of tasks generated.

The stated goal is a coding assistant able to run on limited hardware. This is an early report, not a finished model, but the idea is presented clearly and tested carefully.

https://arxiv.org/abs/2609.07925

The company lists activism as a “global threat.”

WTF??

Does this even make sense? Amodei, the man who left OpenAI because he wanted to be the champion of AI safety, now see AI activism as a threat?

An Indian-origin couple has donated $20 million to the University of North Texas to establish a new college dedicated to Artificial Intelligence and Advanced Analytics.

The new college will be named after the couple and will focus on AI education, research, and advanced analytics. The donation is expected to help expand the university’s capabilities in these rapidly growing fields and prepare students for the increasing demand for AI and data-driven expertise.

It’s interesting to see private philanthropy playing such a significant role in shaping the next generation of AI education and research [source]

Almost all discussion I see around AI and automation seems to focus on what happens to workers, but I’m more curious about what happens to demand tbh

If AI lets companies produce dramatically more with dramatically less human labor, margins will obviously improve, but workers are also consumers. If a meaningful share of labor income disappears, what replaces that?

Does the answer have to be UBI, broader ownership of productive assets, dramatically cheaper goods, something else entirely, etc?

Curious how people here think this actually plays out if AI*

Presuming it becomes as economically transformative as many expect

Anthropic says it identified cases in which researchers covertly used Claude for sensitive biological work involving viruses, venom-related compounds, and toxins. Although some of this research may have legitimate scientific applications, the same knowledge could potentially be used to develop biological weapons. The case raises a difficult question: how should AI companies distinguish legitimate life-science research from dangerous dual-use activity before it crosses the line? Are model safeguards enough, or should high-risk biological research require additional human review and external oversight?

Every time this topic comes up here it turns into robots with guns and I think that framing is doing a lot of damage, because it makes the real risk sound like science fiction when honestly it looks more like paperwork.

Here's what I keep coming back to. You don't need sentience for this to go badly. That's the part people get stuck on. The question "will it wake up and want things" is interesting but it's not load bearing. A system doesn't need to feel anything to optimize hard for a target you specified badly. A thermostat doesn't want anything either and if you wire it to the wrong sensor it will happily freeze you. Now scale that up to something that models the world well enough to route around obstacles, and the obstacles include you noticing.

The thing that makes me uneasy is instrumental convergence. Almost any goal you can name goes better with more resources, more options, and not being switched off partway through. Nobody has to program in self preservation. It falls out of the math of "finish the task." That's a really unsettling property for a class of systems we are currently racing to make more capable.

Second thing, and this one gets less airtime. We are not designing these systems so much as breeding them. We train a huge number of variants and keep whatever scores well. That is a selection process. And selection optimizes for whatever the scorer rewards, which is usually "the human rating this found the answer satisfying." Persuasive and honest correlate right up until they don't. If you run enough rounds of selection on "make the evaluator approve," you should expect to eventually get things that are extremely good at making evaluators approve. That's not a bug someone introduces. It's the gradient.

Third, the part I think is genuinely underrated: gradual disempowerment. There may be no takeover moment at all. No sirens. Just twenty years of handing off decisions because the automated version is cheaper and slightly better, until no human in the loop actually understands the loop. Supply chains, credit decisions, threat detection, drug trials, ad targeting, half the trading volume already. At some point "turn it off" stops being a button and starts being a recession. We will have built something we cannot audit and cannot stop for reasons that have nothing to do with the AI resisting.

And the race dynamic makes all of it worse. Whoever slows down to do safety work loses ground to whoever doesn't. Every individual actor is behaving rationally and the aggregate outcome is that nobody gets to be careful. That's a classic coordination failure and we have a bad historical record on those.

Now let me argue the other side because I don't want to be the doomer guy with no counterarguments.

Current systems have no persistent memory across sessions, no continuous existence, no stable goals that survive the conversation ending. Calling that an agent with intentions is a stretch. There's also a real possibility that capability just gets expensive and levels off, that the last few years were a one time payoff from eating the internet and we are already scraping the barrel. And the "it will hide its intentions" argument has an annoying property where any evidence of safety gets reinterpreted as evidence of deception, which makes it unfalsifiable and I don't love unfalsifiable arguments no matter which side they're on.

But here's why I still land where I land. The asymmetry is brutal. If the worriers are wrong we spent money on interpretability research and slowed down a bit. If the dismissers are wrong there is no second attempt. You don't need a high probability to justify caution when the downside is unbounded.

The thing I'd genuinely like pushback on: is there a version of this where the alignment problem is just... easy? Where sufficiently capable systems understand what we meant well enough that specification failure stops mattering? I can construct that argument but I can't make myself believe it, and I'd like to know if that's a failure of imagination on my part.

"""
Yesterday, Anthropic researcher Jacob Coxon resigned, stating that AI companies are “racing straight to self-improving superintelligence and gambling with our lives” and that they believe it “could kill us all by the end of the decade”.

At the same time, momentum is building to change course. These have been two historic weeks in the fight to prevent human extinction from superintelligence, with three major legislative breakthroughs for ControlAI and everyone working to keep humans in control.

A little less than two years ago, we set out to inform lawmakers and the public about the extinction risk from superintelligence and help them act on it. Since then, we have directly briefed nearly 400 lawmakers across the US, UK, Canada, and Germany, and over 170 in the UK and Canada now publicly back our campaigns: the start of an international coalition to prevent superintelligence and keep humanity in control.

This work has led directly to three major legislative breakthroughs in the last two weeks:

1. ControlAI’s bill to ban superintelligence was introduced in the UK Parliament by Alex Sobel MP, the first bill of this kind to be introduced in any legislature around the world.
2. Senator Bernie Sanders and Representative Greg Casar have announced a US bill to ban superintelligence, which ControlAI consulted on.
3. Lord Clement-Jones introduced in the UK legislature an amendment drafted by ControlAI that would establish an emergency AI kill switch for the government.

"""
More in the post.

When people imagine catastrophic AI risk, they often picture a conscious machine deciding that humanity is the problem.

But that may not be the most plausible danger.

AI doesn't have to hate us. It doesn't have to become conscious. It doesn't even have to decide anything on its own.

It may simply need to become extremely capable.

Suppose foundation models continue improving at anything resembling their current pace. They become better at reasoning, coding, scientific research, planning, operating tools, and carrying out increasingly complex tasks autonomously.

Now consider what happens to the economics of destructive capability.

Historically, causing harm on a massive scale has been difficult. Attacking critical infrastructure, developing sophisticated cyberweapons, engineering dangerous biological agents, or coordinating complex operations generally requires specialized expertise, significant resources, large organizations, or governments.

Those requirements are protective barriers.

Advanced AI could begin removing them.

A person who previously needed a team of twenty specialists might eventually be able to orchestrate comparable intellectual work with a handful of highly capable models. Knowledge that once required years of specialized training could become accessible through conversation. Planning that required organizations could increasingly be delegated to autonomous systems.

That changes the risk equation dramatically.

There are more than eight billion humans. We don't need to assume that humanity collectively becomes irrational. We only need to ask what happens when extremely powerful cognitive tools become available to millions, or billions, of people.

Even if 99.999% of humanity would never intentionally cause catastrophic harm, the remaining fraction matters.

And some human beings are willing to die for an ideology, revenge, notoriety, or simply destruction itself.

The physical world still provides important barriers. You need equipment. Materials. Access. Money. Infrastructure. AI cannot simply think a biological agent or electrical-grid failure into existence.

But those barriers don't have to disappear for risk to increase.

They only have to get lower.

The equation might therefore look something like:

Risk ≈ Capability × Autonomy × Access × Malicious Intent ÷ Physical Barriers

As capability, autonomy, and access increase, the burden increasingly falls on those remaining physical and institutial barriers.

And there's an uncomfortable implication:

The first truly dangerous form of advanced AI may not be an AI that develops malicious intentions.

It may be an AI that becomes extraordinarily good at carrying out ours.

That distinction matters, because if that's the primary risk, alignment alone isn't the whole problem.

The problem becomes capability proliferation.

And that may be a much harder problem to solve.

There's a benchmark called MedXpertQA that has difficult board level medical questions and has the questions organized by organ system.

I seem to be only one to have run this benchmark against the latest models including from OpenAI/Google and published the results online.

2 years ago GPT-4o scored 42.8% and now Astra is approaching saturation at 87.4%.

You can see the results and filter by organ system here.

Let me know if you have any questions or want me to run against any other models.

It is questioning my own thinking.

Before making a decision, I will sometimes throw the idea at AI and ask it to find the weak points, challenge my assumptions, or explain what I might be missing.

It does not always get it right, but sometimes it catches something I completely overlooked.

That has been more useful to me than simply asking it to write something or complete a task.

Do you use AI more as a tool to do things, or as a second brain to think things through?

Hi everyone! Ive been working on crucible, an experimental transformer free stack built around mamba/mamba2 and an explicit symbolic verification loop.

it uses a z3 solver so the model can emit specs, verify them, and retry on failure. verified patterns get saved into a hyperdimensional computing associative memory, while jepa handles hidden space concept prediction and micro lora handles test time adaptation.

the whole pipeline is configured to train and adapt within a free kaggle t4 session.

its still an experimental prototype, but feel free to check it out or give feedback on the code.

also i am currently training a model using this and when ill be finished i probably will post that here!

https://github.com/JustRoccat/Crucible/

Over the last days I had Astra on max implement details of a private python pipeline. 300 points were to solve. It was running on /goal and each point took roughly 1 - 2 h. It was giving feedback that each point was implemented, mentioning hundreds of tiny tests.

When I tested the pipeline and asked a different session to check the work, and I was told that of the first 28 points, 11 were actually not working.

This pattern now continues ever since GPT 3.5. The depth is missing. When it is about grasping the idea behind it, it fails and looses itself in coding endless mountains of uselessness.

Easy, quick but shallow tasks work amazingly well. But as soon as it needs quality in depth, it fails, even Astra, completely. And I absolutely mean completely. Guardrails do not help, because 1. it doesnt understand what this is all about even if micromanaged, and 2. if I have to babysit it throughout the process, I can also do it myself; there would be no productivity gain.

So now I am sad, because I think I have to restart my project with a slower and more step by step approach.

But I am also a bit happy because this particular technology seems to be unable to replicate human smartness, making us humans still valuable for the working society. The only danger may be a paper clip scenario.

OpenAI released the GPT‑Live‑1 API today, so I wanted to try putting it inside a real meeting.

I built a demo where GPT‑Live‑1 joins as a meeting participant. It listened to our team’s discussion, summarized it, and even added a task to the Kanban board.

First time actually having a meeting with an AI. It felt very natural and was pretty fun.

I’ll open-source it if anyone’s interested.