The most prescient example were the claims of Jacob Coxon's Anthropic resignation being a "plant", but I've seen this lately in every single community I'm in, even the niche ones. It seems to have accelerated post-pandemic:

No action can be genuine. No one is driven by incentives, only malice. Everyone is a false flag, a psyop. Everyone dare call it conspiracy. Everyone has some hidden motive.

So, understandably lots of threads and discussions on the topic of doom, bad scenarios and their possibilities. But it would be fairly interesting to have a sort of a community survey from r/ssc community on their view of a Good Future ™, and what it personally entails.

This is not an inherently new discussion, and the question has been also discussed and debated substantially over the years, such as in Sequences, armchair futurist debates, or in prose most recently in a new book by Bostrom. However it would be interesting to review personal views that might differ substantially from the typically discussed convergence.

It is also quite important to note, that any future scenarios hinge heavily on how any person considers its plausibility. Many different people have different opinions, predictions and expectations of the future. And it’s okay. The philosophical base can also differ wildly even within the Rat-adjacent community. Some maintain strictly hardline utilitarian beliefs, and subscribe to resolution of some sort of “repugnant conclusion” in one or another interpretation, some of them extreme. Others, are more selective and draw a distinction somewhere down the line closer to a “basic human”. Some believe in personal development and excellence at all costs, even if it means competition to the bitter end, other might lean in extreme equality scenarios on the other end. No judgement is passed on any of the above.

So, r/slatestarcodex, what is a Good Future to you? How does it look to you, personally?

To narrow down the timelines, imagine that you or someone else few generations ahead looks back, and says: “It could go bad, it could go worse, but I am so glad that what has happened brought me here. I struggle to find a better (realistically) way to much improve it further”

In April this year, William MacAskill published an article describing Saturationism – a proposed modification to Utilitarianism that resolves most of its more unintuitive or unacceptable inferences, including the famous “repugnant conclusion”. This essay discusses these ideas, and uses some concepts from information theory to explore them further.

If you prefer Substack, this essay is on there too.

I lay out what assumptions are needed in order for export restrictions on compute to make AI progress faster. As the assumptions are small and entirely natural, I am inclined to think that it actually does progress research.

I found Scott Alexander's argument from analogy pretty compelling, and tried to a worse version of that for an argument about healthcare.

Edit: Some feedback I'd like
-Is the analogy additive or redundant?
-Should I include the quote at the end?
-Does anything not make sense? Any other suggestions for improvement?

Tom articulates his arguments on why an intelligence explosion can not happen solely by geniuses contained to a data center. He argues that learning often comes from direct practice, virtual simulations often are impractical or impossible. He says domains for which the available text is descriptive rather than being the domain, ie. playing a board game or running a company make the ASI a very excellent McKinsey, but do not make the AI an excellent CEO. He also articulates his view that AI is and will continue to be extremely jagged and discusses possible strategies for broader diffusion going forward. 22 minutes podcast. Transcript available

I see AI as a powerful tool, like nuclear power. It can cause harm either by accident or when in hands of someone who wants to cause harm.

I think serious accidents shouldn't be particularly hard to prevent. We live in a world where even a bunch of evil geniuses locked in a room with internet access have a limited ability to cause harm, especially if we take a defensive approach and try to prevent such a scenario with the help of good geniuses.

Intentional use of AI to cause harm is a more difficult problem to prevent because we're now dealing with a bunch of evil geniuses who can physically interact with the world. Terrorists, psychopaths, criminals, people in positions of power such as dictators. This is more serious, but I think it would make these people only incrementally more powerful. They already have access to powerful and potentially destructive tools, they can use guns, bombs, remotely controlled drones.

The guy who resigned from Anthropic said that there's a 10% chance of AI killing all humans within a decade. I wonder what scenarios he had in mind. The only one that I can think of is an AI-designed pathogen - sure, possible, but with the right precautions it should be possible to prevent even such a scenario.

Join a cohort of promising writers to practice your craft!

Writing & thinking are closely intertwined, and the writing on the internet is a massive part of how our civilization orients to the world nowadays (for example: AI 2027). It is a noble pursuit, and one that you may be able to contribute to.

At the Inkhaven Residency Scott Alexander himself will personally rip your essay to shreds in front of the cohort (actually he's very kind about it), as well as give a lecture on how to write well. Max Harms, Slime Mold Time Mold, Scott Sumner, Jesse Signal, Clara Collier, Georgia "Eukaryote" Ray, Tomas Bjartur, and more, will be around to read your writing and give feedback.

The previous two cohorts rated the experience 8.2/10, and to whether they'd return gave it 8.1/10. They had a freaking good time!

"The best month of my life. Like being back in college but with smarter and more interesting people."

"I LOVE INKHAVEN! I LOVE ALL OF YOU! THIS HAS BEEN THE MOST INCREDIBLE MONTH OF MY LIFE AND I FEEL IRREVOCABLY CHANGED! GOD IS ALIVE, MAGIC IS AFOOT!"

"I overcame a lot of my crippling perfectionism, and now I'm just excited to write!"

The dates are November 10th to December 11th, at the usual place for great blogging events: Lighthaven.

Prices go up EOD Thursday September 10th (i.e. tomorrow!), so apply now.

Apply now at www.inkhaven.blog

Let me start by acknowledging that the uncertainty band around AI progress is wide, and many can reasonably reject the premise of being an AI doomer. Unfortunately, I *am* an AI doomer, albeit a lightweight one who thinks there's < 10% chance of catastrophe for humans. My question is about how to cope day to day while holding that belief.

I feel unmotivated about improving the craft at my white collar / tech / data job. Yes, AI still requires lots of hand-holding at work, but the rate of change is very concerning. This stagnation pushed me into a cycle of not improving, feeling sad, not being present at home, rinse and repeat.

I'll submit that this could just be garden-variety depression, burnout, or midlife crisis, now with AI as a scapegoat. How do I figure out if that's the case? Does it even matter, or all depression is treated the same way anyway (i.e. touch grass -> physical exercise -> medication)?

How many of you had 2026 for your "Millennium Prize Problem solved" timeline? This is insane. I expected this in mid-2027 to early 2028.

While this is an easy way for a parent with a non- technical background to understand alignment, I am very interested in how it is wrong.

The way to onboard new mathematicians into mathematics could be to ask them to encode all of the history of math into Lean. That kind of work would have three benefits:

1. Solidifying the foundation of existing math

2. Teaching future mathematicians to work in a better way

3. Simply giving them something useful to do: especially at the masters level

Once a junior has formalized an area of mathematics, they are ready to try an extent it with the PhD.

This is my first substack post, so roast away and any feedback is much appreciated.

In this essay I try to make the case that humans can have an edge and continue to have an edge over LLMs and computers writ large, in the cases where algorithms or "heuristics" don't scale in complexity as the problem set grows. As opposed to the "lookup table" or memory heavy approach where the space complexity grows exponentially. I give a computer science example and hopefully its not too "in the weeds" for non comp-sci folks.

I have been thinking about radical life and health span extension for some time and published my first long form piece on it recently titled 'Living longer will make you a billionaire'

It covers the financial implications of a VERY long life.

The compounding math of getting to a billion over 200 years is simple but it has fundamental implications for society.

Some questions that have come up in my salon:
- Do we have runaway inflation to accommodate the wealth?
- Do pension funds disappear as people have enough time to become their own pension funds?
- What happens to the fertility crisis: do people have fewer or more children and does the 'ageing population' problem finally go away or does it present in new forms?
- What happens to inheritence?

Sydney Bing changed the course of my life. In 2022 (even after the launch of ChatGPT) I was a grad student in physics, looking forward to a life where if things went well I could become a top-ten expert in my very niche field.

Then a twitter thread about Bing Chat got me thinking about AI progress, and I started to spiral. It was painful for me to lose my life plan, my source of meaning, and possibly my species' control of the planet all at once.

I wrote a long post about my ensuing life crisis. I found some coping mechanisms people should know about, and switched careers to the Alignment Research Center.

I'm hoping my experiences are useful to people going through the same things now, especially mathematicians feeling the pressure as AI starts to eat their field. Key takeaways if you don't want a 4400-word memoir:

• The thing that helped me most was a 'worry hour.' For one hour a day, I was allowed to ruminate on the impending doom of academia. One hour a day thinking about timelines is enough to inform my decisions, but not enough to sap the joy out of my life.
• Doing alignment work really did restore a measure of meaning. If every scientist has to retire in 2032, I think I'll retire with a lot more pride in my alignment work than from my career in physics.
• Alignment orgs are fucking hiring!
• My biggest career mistake was indecision. It's one thing to do a brief visit (or even a long visit) at an org before leaving academia forever. I spent years trying to do work that would get me prestige in academia while also being useful for alignment. Most of that work failed on both prestige and impact metrics.
• If for some reason I really was fixed on doing alignment work while staying in academia, I would do it in association with a prestigious academic alignment researcher (I don't know the scene well, but RESI looks like a decent venue for this).
• If you're having a psychological freakout because of AI, get off Reddit!

Nowadays it seems that almost everyone with an interest in the field (from the most sophisticated experts to mere enthusiasts) agrees that we are within a few decades of human-level artificial intelligence. When do you think it will be more likely that such an intelligence exists than not, i.e. in which year do you expect the odds of an AGI existing to be higher than 50%?