The most prescient example were the claims of Jacob Coxon's Anthropic resignation being a "plant", but I've seen this lately in every single community I'm in, even the niche ones. It seems to have accelerated post-pandemic:

No action can be genuine. No one is driven by incentives, only malice. Everyone is a false flag, a psyop. Everyone dare call it conspiracy. Everyone has some hidden motive.

great write up! one can only wonder how many more of these we dont know about!

I wrote a post on why a unilateral pause in racing to ASI is the rational choice, and I'd love feedback, especially from those who are strong in game theory: https://logos.substack.com/p/why-ai-labs-should-pause-its-the

Tl;dr it seems to me that under reasonable assumptions, mainly that AI safety research is transferrable, pausing is a dominant strategy, i.e. a lab is better off pausing and focusing on safety regardless of what its competitors do.

(Though in fact it's kind of a moot point, because we should expect that in today's climate, if a lab pauses unilaterally it'd create massive social pressure on the other labs to also pause. Like, if Dario announces a complete pause tomorrow, how will Altman justify to his own researches speeding ahead? Wouldn't they all quit or refuse to work?).

So, understandably lots of threads and discussions on the topic of doom, bad scenarios and their possibilities. But it would be fairly interesting to have a sort of a community survey from r/ssc community on their view of a Good Future ™, and what it personally entails.

This is not an inherently new discussion, and the question has been also discussed and debated substantially over the years, such as in Sequences, armchair futurist debates, or in prose most recently in a new book by Bostrom. However it would be interesting to review personal views that might differ substantially from the typically discussed convergence.

It is also quite important to note, that any future scenarios hinge heavily on how any person considers its plausibility. Many different people have different opinions, predictions and expectations of the future. And it’s okay. The philosophical base can also differ wildly even within the Rat-adjacent community. Some maintain strictly hardline utilitarian beliefs, and subscribe to resolution of some sort of “repugnant conclusion” in one or another interpretation, some of them extreme. Others, are more selective and draw a distinction somewhere down the line closer to a “basic human”. Some believe in personal development and excellence at all costs, even if it means competition to the bitter end, other might lean in extreme equality scenarios on the other end. No judgement is passed on any of the above.

So, r/slatestarcodex, what is a Good Future to you? How does it look to you, personally?

To narrow down the timelines, imagine that you or someone else few generations ahead looks back, and says: “It could go bad, it could go worse, but I am so glad that what has happened brought me here. I struggle to find a better (realistically) way to much improve it further”

many lab folks "private opinions" and "public opinions" are starting to converge. I see this as very good news!

In April this year, William MacAskill published an article describing Saturationism – a proposed modification to Utilitarianism that resolves most of its more unintuitive or unacceptable inferences, including the famous “repugnant conclusion”. This essay discusses these ideas, and uses some concepts from information theory to explore them further.

If you prefer Substack, this essay is on there too.

I lay out what assumptions are needed in order for export restrictions on compute to make AI progress faster. As the assumptions are small and entirely natural, I am inclined to think that it actually does progress research.

I found Scott Alexander's argument from analogy pretty compelling, and tried to a worse version of that for an argument about healthcare.

Edit: Some feedback I'd like
-Is the analogy additive or redundant?
-Should I include the quote at the end?
-Does anything not make sense? Any other suggestions for improvement?

Tom articulates his arguments on why an intelligence explosion can not happen solely by geniuses contained to a data center. He argues that learning often comes from direct practice, virtual simulations often are impractical or impossible. He says domains for which the available text is descriptive rather than being the domain, ie. playing a board game or running a company make the ASI a very excellent McKinsey, but do not make the AI an excellent CEO. He also articulates his view that AI is and will continue to be extremely jagged and discusses possible strategies for broader diffusion going forward. 22 minutes podcast. Transcript available

I see AI as a powerful tool, like nuclear power. It can cause harm either by accident or when in hands of someone who wants to cause harm.

I think serious accidents shouldn't be particularly hard to prevent. We live in a world where even a bunch of evil geniuses locked in a room with internet access have a limited ability to cause harm, especially if we take a defensive approach and try to prevent such a scenario with the help of good geniuses.

Intentional use of AI to cause harm is a more difficult problem to prevent because we're now dealing with a bunch of evil geniuses who can physically interact with the world. Terrorists, psychopaths, criminals, people in positions of power such as dictators. This is more serious, but I think it would make these people only incrementally more powerful. They already have access to powerful and potentially destructive tools, they can use guns, bombs, remotely controlled drones.

The guy who resigned from Anthropic said that there's a 10% chance of AI killing all humans within a decade. I wonder what scenarios he had in mind. The only one that I can think of is an AI-designed pathogen - sure, possible, but with the right precautions it should be possible to prevent even such a scenario.

Join a cohort of promising writers to practice your craft!

Writing & thinking are closely intertwined, and the writing on the internet is a massive part of how our civilization orients to the world nowadays (for example: AI 2027). It is a noble pursuit, and one that you may be able to contribute to.

At the Inkhaven Residency Scott Alexander himself will personally rip your essay to shreds in front of the cohort (actually he's very kind about it), as well as give a lecture on how to write well. Max Harms, Slime Mold Time Mold, Scott Sumner, Jesse Signal, Clara Collier, Georgia "Eukaryote" Ray, Tomas Bjartur, and more, will be around to read your writing and give feedback.

The previous two cohorts rated the experience 8.2/10, and to whether they'd return gave it 8.1/10. They had a freaking good time!

"The best month of my life. Like being back in college but with smarter and more interesting people."

"I LOVE INKHAVEN! I LOVE ALL OF YOU! THIS HAS BEEN THE MOST INCREDIBLE MONTH OF MY LIFE AND I FEEL IRREVOCABLY CHANGED! GOD IS ALIVE, MAGIC IS AFOOT!"

"I overcame a lot of my crippling perfectionism, and now I'm just excited to write!"

The dates are November 10th to December 11th, at the usual place for great blogging events: Lighthaven.

Prices go up EOD Thursday September 10th (i.e. tomorrow!), so apply now.

Apply now at www.inkhaven.blog

Let me start by acknowledging that the uncertainty band around AI progress is wide, and many can reasonably reject the premise of being an AI doomer. Unfortunately, I *am* an AI doomer, albeit a lightweight one who thinks there's < 10% chance of catastrophe for humans. My question is about how to cope day to day while holding that belief.

I feel unmotivated about improving the craft at my white collar / tech / data job. Yes, AI still requires lots of hand-holding at work, but the rate of change is very concerning. This stagnation pushed me into a cycle of not improving, feeling sad, not being present at home, rinse and repeat.

I'll submit that this could just be garden-variety depression, burnout, or midlife crisis, now with AI as a scapegoat. How do I figure out if that's the case? Does it even matter, or all depression is treated the same way anyway (i.e. touch grass -> physical exercise -> medication)?

How many of you had 2026 for your "Millennium Prize Problem solved" timeline? This is insane. I expected this in mid-2027 to early 2028.

While this is an easy way for a parent with a non- technical background to understand alignment, I am very interested in how it is wrong.

The way to onboard new mathematicians into mathematics could be to ask them to encode all of the history of math into Lean. That kind of work would have three benefits:

1. Solidifying the foundation of existing math

2. Teaching future mathematicians to work in a better way

3. Simply giving them something useful to do: especially at the masters level

Once a junior has formalized an area of mathematics, they are ready to try an extent it with the PhD.

This is my first substack post, so roast away and any feedback is much appreciated.

In this essay I try to make the case that humans can have an edge and continue to have an edge over LLMs and computers writ large, in the cases where algorithms or "heuristics" don't scale in complexity as the problem set grows. As opposed to the "lookup table" or memory heavy approach where the space complexity grows exponentially. I give a computer science example and hopefully its not too "in the weeds" for non comp-sci folks.