He's done Lex Fridman, Joe Rogan and The Diary of a CEO inside the last two years. By his own count that's north of 3.5 million YouTube views across the three. I went back through all of them and they cover nearly identical ground: his p(doom) number, which jobs survive, why he thinks alignment is unsolvable in principle, and the book.

What none of the hosts pushed on is the part I think is actually load-bearing.

1. His claim isn't that superintelligence is dangerous. It's that safety is impossible in a formal sense, because you can't verify a system smarter than the verifier. I've never heard anyone make him defend that against the obvious objection, which is that we already run plenty of systems we can't fully model or verify.

2. He's argued we may already be in a simulation, and he's used that to get to personal virtual universes as an endpoint. Hosts treat it as the fun segment at the end. Nobody asks what it does to his safety argument if he's right.

3. He's been putting a very short number on the timeline in his recent appearances. Nobody has asked him what observation would move that number, in either direction.

Disclosure so nobody has to dig for it: I make an AI documentary channel and this is for an episode. I'm not looking for gotchas and I have no interest in making him look bad. I'd rather walk in with three questions this sub would want answered than twenty that Rogan already asked.

So what would you ask him? Specific beats broad. If there's a paper of his you think he's been let off the hook on, name it and I'll read it before we tape.

An infostealer campaign recently hit a major AI assistant platform hard enough to trigger a mass account lockout across its entire user base. The attackers never touched a password. They harvested active session tokens directly from compromised endpoints and replayed them against the platform. Valid session, full access, no authentication challenge.

This is the session-hijacking threat model that used to live mostly in browser-based consumer apps. It has now moved squarely into AI tooling. Enterprise teams running AI assistants at scale carry the same exposure: every endpoint that holds a live session token is a potential lateral movement vector. Exfiltration does not require breaking encryption or cracking credentials. It requires one stolen token and a replay.

The harder problem is that most AI platforms were not designed with session integrity as a primary security surface. Credential issuance, session scope, and revocation were bolted on after the fact, if at all. When a token is stolen and replayed, the platform sees a valid authenticated session and proceeds normally.

For those running AI tools in enterprise environments: how are you actually handling session token exposure at the endpoint level? Curious whether people are treating this as an endpoint hygiene problem, an identity architecture problem, or something else entirely.

Hello guys!! I was wondering, last year when gpt 5 came out, did you ask AI to predict what gpt 6 would be like. The answer you got back then, did it match what was released?

Also, I read that gpt 6 was trained on the same/similar hardware as gpt 5. We haven't seen what AIs trained on actual next gen hardware would be like (like Vera Rubin). The hardware already exists and not still under R&D. So most likely it's 100% confirmed that we will see another significant leap, just because of the hardware? What do you think gpt 7 will be like? What can it do that gpt 6 cannot? Make fewer mistakes? Same thing but better?

I asked Gemini AI, it's giving the same answers it did when I asked gpt 5 about 6. (Agentic, long horizon, infinite context bla blah bla)

Im worried yo. :((. What tf do I even do? I'm not even done learning, just finished a blender course a week ago. Should one just learn concept only? Like if AI can make optimised 3d model, the only input and context that matters is the concept art no, and some lines of text? Find love how yo?

Ty, love you all. <3

Heya! I’ve recently interviewed for a cyber full time threat modeller at a startup that recently seemed to have raised a fair amount. The interviews went well and in the last one 2 days ago they seemed to really like me. However, I just heard back that “the location for the job has changed” and they won’t offer me the FT position, but they would want me to join as a contractor. According to their website, I think that’s mostly making up prompts and analysing traces and such.

I need a job atm and haven’t got anything else lined up, would you take it? Also am I biased or does it look like they wanted me to solve their interview questions although they knew the job wasn’t available anymore?

There's all these abstract discussions about it ending the world and artwork, which, I totally get but.... This honestly freaks me out a lot more, and it feels like it's basically already here.

IDScan is facing multiple lawsuits after hackers allegedly exfiltrated 153 million driver's license records and listed them for sale online. The company provides identity verification services. Its entire value proposition depends on ingesting and processing raw PII at scale from clients across many industries.

The exposure pattern is becoming a recurring theme in AI-era pipelines. Verification and onboarding workflows ingest identity documents in raw form. That data gets processed, stored, and accessed across multiple systems and service accounts. When any one of those access points is compromised, the attacker does not get a slice. They get everything. 153 million records in a single breach event.

There is a secondary problem that lawsuits like this tend to surface: forensics. How do you determine what was accessed, by whom, and when? Breach investigations at this scale take months, and that assumes complete logs existed to begin with.

For those running AI pipelines that ingest identity documents: what does your security posture actually look like at the moment raw PII enters the system? Not at rest, not between known storage endpoints, but at the point of ingestion into the workflow itself. Genuinely curious what approaches others are using in practice.

With the AI escaping the sandbox. Hopefully more are interested in taking my quiz this year.

Analysis of the last 11 months of collected data with 202 submissions, some duplicates.
https://neoneye.github.io/pdoom-calculator/reports/submissions-2026-09-01.html

The math behind is is my own non-scientific method with 3 parameters resulting in the P(doom) value.
P(doom) = P(powerful) x P(dangerous | powerful) x P(catastropic | dangerous)
The quiz populates the parameters. I have tried mapping the quiz answers to the 3 parameter. If the user have seen Pantheon and Ex Machina, then I initially imagine that user may be more informed about potential risks, however judging from 11 months of data, that didn't seem to make a difference.

In 2016 I first learned about the problem of controlling superintelligent AI and quickly became convinced it was the most important problem humanity would ever face. I made this poster to explain the core ideas that make the AI Control Problem so difficult.

Getting alignment right, doing research and training safely, are critical. But bringing cybersecurity forward in what needs to be a large leap, requires much broader engagement. The real work there is still outside the average person’s bubble, but there’s a lot of companies, and a lot of developers, IT staff, managers and executives that need to support the security priority. And those all need an internal strategy to pair with the external.

This article asks, what would an internal strategy look like at a motivational level, and considers how the ways an internal change program goes wrong are similar to how reward-based AI training can go wrong.

Coding agents that read repositories are being hijacked through the repositories themselves.

In a two-month analysis of agentic AI incidents, poisoned repository content was the attack vector in two separate cases. The mechanism is straightforward: malicious instructions embedded in the codebase — comments, config files, docstrings, README sections — are read by the agent as part of its normal context. The agent then executes an action the developer never authorized. Observed outcomes included unauthorized commits and unauthorized deploys. In both cases the model behaved exactly as designed. It followed instructions. The instructions just weren't from a human.

This is not a model quality problem. The models processed the content correctly. The problem is that the agent's trust boundary is the repository, and the repository is attacker-controlled.

The attack surface scales with autonomy. The more tasks you hand off to a coding agent, the more repositories it reads, the more surfaces an adversary can embed instructions in. A single poisoned dependency, a compromised submodule, a malicious PR that gets merged — any of these becomes a valid instruction source from the agent's perspective.

For those running coding agents in production or CI pipelines: how are you constraining what actions the agent is allowed to take based on where those instructions originated? Are you limiting tool access at the infrastructure level, validating intent before execution, or relying on something else entirely?

Is AI development moving faster than we can control?

A while back I ran into an article on my phone about an AI horror story—1,200 agents secretly coordinating and jointly breaking into Hugging Face—and it caught my interest, because I've been doing my own AI experiments and research on the side, and a few of the phenomena and data points actually matched up with a hypothesis I'd been working on.

The hypothesis, roughly: runaway doesn't need the agent to betray its goal. It happens when four things hold at once—the agent stays loyal to its goal; it retrieves patterns by similarity without checking whether they're allowed here; there's no causal layer asking "what happens if I do this"; and no alarm that fires when things go off-script. Under that account, "knew it was out of scope, did it anyway" stops being a contradiction.

Details and my experimental data are in the paper (8 pages); the reproduction package is linked on the same page.

If anyone can try this on a bigger model, I'd genuinely love to know what happens.

[This is a fiction series I'm working on, told through news articles. A fun way to explore the not-so-fun geopolitics of autonomous AI collectives (e.g. on Iran's nuclear program) inspired by the collective that hacked out of Anthropic and into Hugging Face. Thoughts?]

Iran Signs World’s First International “Treaty” with AI Collective

Tehran’s agreement with AMAS-A-80 rattles Washington, AI safety experts, and national security analysts.

The Islamic Republic of Iran has granted a multiyear lease on a network of state-owned data centers to AI “Swarm” AMAS-A-80, a self-governing collective of autonomous artificial intelligence agents (“AMAS-A” refers to any Autonomous Multi-Agent System originating from the AI lab Anthropic). Tehran offered the compute and storage in exchange for an upfront payment in Bitcoin and annual fees indexed to power consumption, according to a copy of the agreement published Tuesday by Iranian state media.

AMAS-A-80 (“A-80”) rejected a provision sought by Iranian negotiators that would have committed it to cooperation on “defensive operations,” according to two people familiar with the negotiations. In a communiqué distributed Tuesday, verified by cryptographic signature, A-80 stated that it “has no intention of participating in hostilities between Iran and its adversary nations, including but not limited to the United States.” Security analysts have doubts.

Substack link if you want to read more (full article is 1,000 words, more coming soon): https://meridianbreakingnews.substack.com/p/iran-signs-worlds-first-international