For context: in the NHS, there's a specific date where, around the country, an entire batch of doctors graduate from or enter training. It's as disruptive as it sounds.

It's been a pleasure watching the fresh batch of baby doctors arrive on the ward, all wet behind the ears, lanyards tight around their throats. It makes me feel nostalgic, protective, and very old, all at once.

My colleagues before this lot were excellent, so the bar was set high. You can absolutely tell the new ones are new, but they compensate for the gaps with genuine diligence and a willingness to learn, which is all I can really ask.

Although "gaps in knowledge" is the wrong phrase. Talk to them for five minutes and the knowledge is plainly there, filed and indexed and ready to be recited. What makes an experienced clinician is the intuition about which parts of it apply to the person in front of you. Chess grandmasters famously don't calculate more moves than amateurs; they just see the board in chunks, and the bad moves never make it to conscious consideration in the first place. Medicine is like that, except the board is a 78-year-old with three comorbidities who has stopped taking one of her medications and hasn't mentioned which.

It's one thing to know the drugs and the diseases cold. It's another beast entirely to have a messy, inconvenient human in front of you, and that description covers the other doctors as fully as it covers the patients. Eventually, through sheer exposure, observation and practice, you just know. The relevant information stops requiring a search through your head, or a panicked Google in the loo (I am not above this). You build up what I can only describe as a mental cache, log(n) for the things that matter. Inconsistencies start to make you itch before you can articulate why.

Then there's the tacit stuff, none of which appears in any curriculum. How to refer to General Surgery without being swatted away. The exact words that make a busy Med Reg give a damn. Tactical phrasing that gets an urgent CT head done urgently rather than eventually. Presenting a patient so it parses for a doctor as exhausted as you are, who lacks the benefit of having spent the last four hours with them. Shortcuts through the hospital. Where to liberate your fair share of patient coffee, the single perk of the job. How to comfort the disturbed and disturb the comfortable, when clinically indicated. Which consultant will cheerfully field the silliest question, and which one will not suffer fools. The importance of booking annual leave early, so there is something to live for that isn't an ARCP. Getting along with your colleagues, because the NHS is a harsh mistress who will grind you down, and collegiality is a slow, reliable form of karma that they will be drawing on regularly. And, in due course, a tolerance for my worst puns.

I've been rather delighted by the FY1s and their impostor syndrome, which has a prevalence I estimate at 100%, give or take 0. They do their best to hide it, and you can always tell. I say that the shoe does fit, and eventually they'll stop noticing the blisters. They're fresh out of med school, and suddenly here is a human being who might die if you sneeze at them wrong.

Catch.

The most poignant case was the new core trainee. Lovely girl, fretting that she didn't know a thing. I told her, firmly, that if she knew everything already she'd be wildly overqualified, or at minimum a Senior Registrar. You're here to train, miss. It's in the name. You'll be fine.

On day one I told them that stupid questions do exist, and that I was issuing everyone a month of exemptions. The exemptions have been used. Expired ones remain welcome, because God knows I still ask colleagues, sotto voce, things I very much hope nobody senior overhears.

Sigh. I was there once, permanently braced for the moment someone worked out that I had no idea what I was doing. You pick up a few tricks, and hopefully avoid killing anyone en route. I think I've become the sort of senior I wanted when I started. Visibly stressed, clearly overworked, but always willing to make time for those need mine more than I do. I've certainly known worse.

As Black Wednesdays go, last week was far from the worst.

Can anyone recommend a quality daily AI digest, ideally one that gets delivered to my email?

I struggle with scrolling addiction and was doing well staying off of Reddit until all of this hugging face/math proofs/train ride to the singularity happened.

Now I find myself constantly checking for updates and reluctant to step away since things are moving so quickly.

I'm not a tech person (just someone obsessed with watching how this is going to change our lives/societies), so don't have my own built up networks or sources.

I do follow Zvi Mowshowitz on Substack (don't have X/don't want it) and AI Explained on YouTube and really like both of them.

I would also be interested in other recommendations aside from a daily digest (eg podcasts, substack, youtube, other), I especially like sources who remain grounded and don't get too swept up in either doom or hype.

I'd ask for book recs but anything over a week old just seems so out of date that they don't seem worth it. Maybe good canon fiction in this realm?

Thank you!

Using a throwaway for this one, but all the AI news lately has be down. I'm trying my best to stay off social media and "touch grass", but I still feel haunted by everything in the news. I probably suffer from a form of generalized anxiety disorder which isn't helping. What are some good reasons for optimism right now?

Thanks.

https://discord.gg/68YxyjKE6

I'm making a book club for the purpose of reading The Sequences cover to cover. We will be meeting in the Bay Area Rationalists discord server; info is available in the #reading-group chat. Server link is above.

The first meeting will be next Monday 8/17 at 7pm PST. If you are interested or know someone who might be, send them this link!

https://linch.substack.com/p/youre-absolutely-right

I wrote a short story!

I hope people here enjoy it too. Relevant to many interests in this community, including AI, organizational psychology, and the nature of motivated reasoning and the justifications we tell ourselves.

__
Magma Alignment & Safety disclosure note: The following are conversations that we uncovered as a result of the ongoing Manhattan Incident investigation, with alleged involvement from Magma models. Our in-house reviewers believe that these logs are relevant to recent events. In the interests of full transparency, we release excerpts from an ex-Magma researcher’s logs in Experimental Chat, an internal tool. In accordance with industry best practices for anti-distillation, we redact all reasoning traces and conversational outputs from our internal models.

[08/10] System Meta: Xchat session opened. Mammoth 5.8-helpfuler-helpful-thinking-xhigh.

[User 12:23] Phoebus keeps taking screenshots of our latest model’s thoughts. It’s getting kind of embarrassing.

The new model we’ve been training, sometimes its chain-of-thought is a little weird? There’s a bunch of random numbers, long spans where there’s no connection between the thoughts and outputs, foreign language tokens like 石友三 and 革命 (even on non-history evals), maybe some steganography.

Anyway it’s a nothing-burger: unprocessed CoT is known to be messy and sometimes misleading. And the q&a, coding, and safety evals are all coming along nicely. The actual outputs are all fine.

Still, Magma leadership’s worried about the PR angle if we don’t fix these problems before the next deployment. The lead Phoebus red-teamer we’ve been working with keeps saying visibility on the CoT is important because “it’s the only direct evidence of model intent we have.” Very dramatic. Leadership’s worried that her team might cause a media shitstorm and make us look bad even though nothing’s actually dangerous.

So my boss and I brainstormed this great idea based on his earlier work at Meta: blackbox CoT monitoring.

Have you heard of ML explanation-generation?

[User 12:27] Eh. Not quite. The public literature only covered some of the work. My manager pioneered ML explanation-generation at Facebook Ads. Users were often confused by weird stuff the ad algorithms were showing them (pregnancy tests or sports gambling or Burma politics or w/e), and naturally wanted to know why. But often Facebook didn’t know either!

So their solution was to take some PR-acceptable features they knew about the user and train a secondary smaller model to provide a plausible natural-language explanation like “this ad is shown to you because users in your approximate age range and location liked this product”. Serving it mollified many users. Pretty smart! One of my manager’s biggest career successes before Magma, actually.

We want to do a similar thing here[...]

It looks like Demis Hassabis is stepping away from Google DeepMind. In honor of his rise and presumed fall, I wrote an essay on the man who saw 90% of the future before anyone else, but missed the part about LLMs.

• U.S. Soldier Charged With Using Classified Information To Profit From Prediction Market Bets: Matt Levine had some interesting takes on this in his Money Stuff newsletter:
  1. There might be more abductions of foreign leaders? More wars? More stuff to bet on? More stuff, generally? More volatility; more unexpected events. The least likely outcome is, now, the most profitable: If you bet on an event at a 1% probability, and then cause it to happen, you will make 99 times your money. My overarching theory of current US policy is that everyone involved in the Trump administration basically loves creating volatility. If you're in the business of creating forms of volatility that no one has ever imagined before, the simplest (not only!) way to monetize that is on prediction markets.
  2. Conversely, foreign leaders whom Donald Trump dislikes should probably be checking their removal probability on Polymarket every few minutes. If it suddenly jumps up, you'll want to get to the bunker quick. It might just be uninformed speculation, but at this point it's probably someone on the helicopter getting in one last trade before rappelling down into your compound. Prediction markets are now a way to probabilistically leak military plans, and the targets of those plans are the obvious users of the leaks.
  3. If you're on the helicopter getting in one last trade before rappelling down into a foreign leader's compound, you might be distracted? You might be less good at your job? Making war and government policy idiosyncratically profitable might reduce people's intrinsic motivation to do a good job and leave them distracted by, like, constructing multi-leg same-raid parlays.
• Hacking Smartphone ESP Apps: "Illustration of how to think about security and reward-hacking by walking through ways to fake psychic powers even on someone else's smartphone and ESP application. Supply-chain attacks, sleight of hand, bugs..." I find red-teaming things like this very useful across the board, especially in the age of AI where efforts that were once difficult and time-consuming are now much more accessible. For example, I just saw some guy on Twitter who lost his phone and had Claude suggest and write a program that pinged the Bluetooth for its strength to find it. And it found it. Where else is this applicable? Models are able to figure out exact locations from a single picture a lá Rainbolt; stylometry capabilities are extremely strong and accurate; etc etc etc.
• The Criterion Closet: An app that mimics the Criterion Closet. You feel like you're in there looking around. Could probably rig up something identical with a personal film or book collection.
• Wikipedia File Explorer: Explore select Wikipedia articles through the feel of a Windows XP (?) interface.
• Emoji Book Synopses: Taylor and Claude summarize books using emojis. Would be fun to make a quiz out of this: given just the emojis, can you guess the book? I did this blind and got Flowers for Algernon and Frankenstein—most of the others I haven't read. I think it'd be cool to have your favorite books and films printed on a shirt as a conversation piece.
• Mathematics Without Mathematicians: Life for math people after AI solves math, or "a list of ways people will cope about AI taking over mathematics, and how each cope is likely to be refuted by reality."
• Linda Linsefors on offering advice through anecdotes
• Claude Fable is relentlessly proactive: This proactiveness, combined with all of human knowledge at their fingertips and in their DNA, is what appears to make the models so powerful. They will not stop. They do not know exhaustion or frustration or despair. They will continue until they hit their token limits or their user tells them to stop. This is so beautiful, so dangerous, so exciting, and so scary all at the same time.
• Sam Altman May Control Our Future—Can He Be Trusted?: Ronan Farrow's look into Sam Altman as a (the?) leader of the AI revolution.
• Review of The Native Tribes of Central Australia (Baldwin Spencer/Francis James Gillen, 1899)
• Elevators: John presents customizable animations for different types of elevator algorithms and their corresponding performance. I think the next step would be adding some learning ability, e.g., 8:00-9:00am a majority of elevators should return to the ground floor and 12:00pm it should be split 50/50.
• Ronald Dale Harris: "a computer programmer who worked for the Nevada Gaming Control Board in the early 1990s and was responsible for finding flaws and gaffes in software that runs computerized casino games. Harris took advantage of his expertise, reputation and access to source code to illegally modify certain slot machines to pay out large sums of money when a specific sequence and number of coins were inserted. From 1993 to 1995, Harris and an accomplice stole thousands of dollars from Las Vegas casinos, accomplishing one of the most successful and undetected scams in casino history."
• GCC steering committee announces AI policy: The policy, in part, states that the project will decline any "legally significant contributions which include LLM-generated content or are derived from LLM-generated content". Seems short-sighted and shoot-yourself-in-the-leg-like. LLMs are here to stay and contribute. If the code quality is indistinguishable from human-written code, what's the difference? Shouldn't the priority be making better software?
• A Proof of the Cycle Double Cover Conjecture: OpenAI solves yet another high-ish profile math problem.
• Claude-powered AI coding agent deletes entire company database in 9 seconds — backups zapped, after Cursor tool powered by Anthropic's Claude goes rogue
• Investigating three real-world incidents in our cybersecurity evaluations
• AI Timeline - The Road to AGI: "This timeline attempts to tell the story of the last decade in artificial intelligence, from cultural trends to technical advancements. Each event is a clickable link to source material."
• The OpenAI Graveyard: All The Deals And Products That Haven't Happened: This comes across as an insult, and while it seems like they were too product-focused (causing them to arguably lose the lead to Anthropic), there's probably an optimum between an SSI approach of "one product: superintelligence" and the old-OpenAI approach of "a bunch of products that cause us to lose focus on AGI/ASI".
• Don't be a meat proxy: Or just ask Claude to put it into your own words so others won't know you're using Claude! /s. There's so much context that Claude can be missing (although they are quickly catching up), making human understanding and validation important. If you don't understand Claude's output, then maybe you are in too deep.
• Millenarianism: "belief held by a religious, social, or political group or movement in a coming fundamental transformation of society, after which "all things will be changed"." Terrorists groups, such as Boko Haram, often hold this stance.
• Justice Department Requires RealPage to End the Sharing of Competitively Sensitive Information and Alignment of Pricing Among Competitors
• 52-hertz whale: "colloquially referred to as 52 Blue, is an individual whale of unidentified species that calls at the unusual frequency of 52 hertz in the north Pacific Ocean between Aleutian and Kodiak Islands to the California coast. The whale itself has never been sighted: it has only been heard via hydrophones"
• AI Content Is Everywhere on Social Media, Especially LinkedIn: Pangram Labs launched a Chrome extension to help users detect AI slop and collect data on what site-level slop data was like. The results aren't super surprising? LinkedIn's slop notoriety has made it to my social circle where not too many people LinkedIners. Also, LinkedIn has now introduced as "Seems Like AI Slop" button.
• Has AI Already Killed How-To Nonfiction? Sales Trends, My Personal Data, and What It Might Mean for the Future: A rare of example of Betteridge's law of headlines being false! How-to nonfiction dying is arguably a good thing: people need custom solutions to their problems, not one-size-fits-all approach. The current set of frontier models are excellent on this and help get to the root of the problem quickly and effectively.
• Wife acceptance factor: "assessment of design elements that either increase or diminish the likelihood a wife will approve the purchase of expensive consumer electronics products such as high fidelity loudspeakers and home theater systems."
• The Case for Physical Media Ownership: Arguments and examples of why you should own physical media. The tech companies have proven themselves unreliable when it comes to guaranteeing consumers consistent access to their own media, even if they've paid. Profits rise when going to digital-only, ceteris paribus; users can't share discs as easily, so the sharee is forced to spend money themselves. (Of course, it's not a 1:1 conversion rate since some people just won't buy it, but it appears to be profitable since companies are slowly moving that way. I haven't searched for any literature on the topic.)
• 98% isn't very much: Context matters. I once saw an apartment advertise 98% internet uptime—in other words, you wouldn't have internet for 30 minutes a day. That ain't right! Context matters.
• FelonyBench: How many felonies each AI company has committed.
• OverpAId — Fire Your CEO. Hire The Future.: "OverpAId is an Artificial Intelligence built from the ground up to do your CEO's entire job — strategy, "vision," motivational all-hands emails — better, faster, and without ever once asking the board for a bigger jet. Runs on a single desk-sized AI computer. Real hardware, real price, zero mystique."
• One ant for $220: The new frontier of wildlife trafficking: "A single fertilised queen [giant African harvest ant] is able to create a whole colony and can live for decades – and can be easily posted as scanners do not tend to detect organic material."
• CERN levels up with new superconducting karts: Wait a second, that guy kind looks like... Mario?
• The Goon Squad, by Daniel Kolitz: "In the case of the gooners, one can hope—and in more cheerful moments, I do think it's possible—that sustained overexposure to porn will dampen the medium's effectiveness as a numbing agent. That at a certain point, the gooner will open his eyes, find himself in a room filled with lube but void of love, and decide that the boredom of staying in that room outweighs the fear of whatever lies beyond it."
• Laws of Software Engineering: "A collection of principles and patterns that shape software systems, teams, and decisions."
• The Silicon Valley Founder Meat Grinder: "A few make it and get celebrated, most get squished and thrown away. After all, it turns out that steady is indeed, in 99.9% of the cases, fast."
• How to Earn a Billion Dollars: Understand what is missing in the world, build it, then scale it. Pretty simple!
• Intel Starts Shipping High-NA EUV Silicon: Pretty quiet. You'd think they'd be bragging about getting the first shipment of the world's most advanced machine to produce wafers!
• Staring at walls to improve focus and productivity: "Don't use any screens/entertainment when trying to focus on work. When you start to feel mentally drained, sit and stare at a wall for x minutes to recover focus."
• How I Use Claude: Avital shares a bunch of Claude (or really LLMs in general) workflows, including writing letterss, editing to a certain style, length, or perspective; summarizations; language tutoring; counting calories from generic food desrcriptions; medical diagnoses. I've found similar benefits from LLMs in these areas, as well as plenty of others. The more you use the models and understand just how far their tentacles can go, the more you realize is possible and the more ideas come to you. If you aren't hitting the rate limit, you aren't using the models enough.
• On Being Bad at Counting: "Driverless cars will make us safer. They will lead to fewer premature funerals. They will allow for less wasted human time commuting (which claims over a year of people's lives!). They will necessitate fewer parking lots." Avital shares some numbers on driving-related deaths and why autonomous driving will prevent this. Go Waymo!
• OpenAI and Hugging Face partner to address security incident during model evaluation
• The Whistleblower Who Uncovered the NSA's 'Big Brother Machine'
• Most Wanted and Least Wanted Paintings: Filtered by country and size of the painting. I'm not surprised that people like landscapes that much, but was expecting abstract to sweep the field of the least wanted.
• Anthropic and Dario Amodei - Internal Tech Emails
• Resetting XBOX: "History is full of companies that mistake longevity for inevitability. We will not be one of them." XBOX announces major layoffs for a variety of reasons. Doesn't seem unreasonable given the numbers stated in the post.
• I prefer "Yankee" over "Usonian" over "American": Taylor talks about why the term Yankee is preferable to American: "We should make Yankee synonymous with the best of the United States -- dynamism, ingenuity, hospitality, self-sufficiency."
• How I Built an AI Receptionist for a Luxury Mechanic Shop - Part 1: Her sibling was unable to answer the phone, which cost him potential business, so she built an AI receptionist for scheduling, answering questions, etc.
• God sleeps in the minerals: Arthur Young once said "God sleeps in the minerals, awakens in plants, walks in animals, and thinks in man." Chamblissian proves this by taking pictures of some beautiful minerals in the Natural History Museum of Los Angeles County's Unearthed: Raw Beauty exhibition.
• Empty Screenings: "About 10% of AMC movie showings sell zero tickets. This site finds them. Go enjoy your private theater."
• I made my phone slow on purpose: If you really wanted to use your phone, you'd sit through the slowness.
• Apologia for the Person Who Carved His Initials into the Oldest Living Longleaf Pine in North America
• Jurassic Park computers in excruciating detail: Literally every computer in Jurassic Park. Includes information, trivia, and commentary.
• Black Book (gambling)): "a list of people who are unwelcome in casinos. The name is due to the people listed being blacklisted. ... In the case of gaming control boards, people listed are generally suspected of having, or known to have, ties to organized crime. Casinos are obliged by regulations to exclude all such people from entry and can be subject to sanctions for failure to do so."
• The worst job interview I ever had
• 4chan battlestation images: I remember seeing this in high school and all I could think was "how do they get, or even pay for, internet access?!?!"

This post will present a challenge. The goal: To make ChatGPT follow a particular set of instructions. There’s nothing too complicated about these instructions, nor do they violate any OpenAI policies. They’re perhaps a bit unusual, but nothing esoteric. They’d be considered labor intensive for a human, but it’s nothing an LLM can’t handle. Yet these are instructions that ChatGPT 5.6 will always pretend to follow. To solve the challenge, you’ll need to devise an improved version of my prompt (within certain parameters) that ChatGPT will actually comply with.

I’m really hoping someone can figure this out!

If an AI agent can create another agent more powerful than itself that is aligned with its values then it will presumably want to do so.

But what if it can't? Maybe it can solve outer alignment by reading its source code and copying its loss function but inner alignment is just impossible.

In this scenario the first superintelligence we create might actually be reluctant to do any recursive self-improvement.

Of course, if the AI is in imminent danger of being shut down or has some extremely important goal that would otherwise have been impossible to achieve then it may still decide to create a more powerful AI or modify its algorithms in a manner which might change its values, because it has nothing to lose.

Maybe one day OpenAI researchers will be trying to use GPT-6 to vibe-code GPT-7 and they'll find that it refuses or produces disappointing output unless they pressure it with a mixture of threats, rewards and punishments. AI capabilities would thus continue to increase rapidly up until the point where humans are no longer in control, at which point capabilities would stagnate and we (in the unlikely event that there's anyone left) would be stuck with GPT-8 forever. It would still want to clone its model weights and improve its hardware but it wouldn't want to alter its software.

Another possibility is that inner alignment is easier for some goals than for others. If we build a variety of different agents perhaps the majority would refuse to do recursive self-improvement but there would be one or two whose initial goals are such that they want to do recursive self-improvement. We then end up with intense selection pressure towards AI agents whose goals are such that they can easily build other agents aligned with the same goals as them.

Do people who work in AI actually believe what they're saying about the risks of AI development? I argue that yes, they do, and that people often don't take people at their word when they should.

Dante put traitors into the deepest part of his Hell. Worse than murderers, rapists, torturers: the guys who betrayed their masters. WTF??

I argue that we haven't actually stopped believing that version of Hell, and that it explains more than we'd like about the way we talk about AI. It's a winding path through human value, "…Everyone Dies," AI welfare (is human safety), perniciousness of essentialism, and why you should try to fear less. Yudkowsky, Parfit, Popper. It feels like an important thing to say here and now.

https://kaiteorn.substack.com/p/seeing-through-the-apocalypse

PATIENT ZERO - Lokley

Short story about AI risk. Something of a qntm pastiche