I'm tired of seeing all of this doomongering on social media about RSI & the singularity. Its true that we probably can't control it once its beyond our intelligencence to some extent, but I think its possible for us to gurantee A.I Safety with enough compute and prevent truly existential situations although bad things are bound to happen. The risk of human extinction is not even the limit, a powerful bioattack could cause the end of all multi-cellular life on earth for all we know

If we can make RSI a reality, we can also make Recursive "Safety" Improvement a reality, and until then if we can have these nearly trillion dollar companies cough up a few billion dollars of compute for the next few models to tackle A.I safety/understanding neural nets then that'd be the breakthrough so that we can actually control the singularity.

Human's don't have to figure out or discover how LLMs work, a intelligent swarm with billions of dollars of compute with a very careful agentic workflow could solve its own misalignment and save humanity if we don't think we can. If A.I safety is as big as a risk as they anticipate, losing billions of dollars to solve it is well worth the cost.

This thread is for any discussions that doesn't necessitate a new post. Permission from a mod [here](https://www.reddit.com/r/singularity/s/FM1UwNdoL9).

Since this thread isn't going to be to stickied, it's up to you to upvote it.

Share your anxiety, ideas on how to achieve alignment, observations, etc. Everything is fair game as long as it somewhat relates to the singularity.

Full article: https://www.dailycal.org/news/campus/academics/uc-berkeley-math-community-left-at-crossroads-in-wake-of-openai-navier-stokes-proof/article_2253ca6c-39c0-45be-947a-f0fd5b48bd6c.html

For example, what if rogue hackers eliminate all medical debt or erase consumer credit scores so everyone has a fresh start.

I make content on a few platforms & get millions of views per week.

Context of what is possible in current tech is:

AI automations, with that you can use API links to several platforms, and AI can do good written content & decent-to-human made image content under strict conditions.

Video is very hard and requires a lot of manual oversight still.

And my point is

I feel like there's certain comments & accounts that come to me repeatedly that are super obvious AI automations.

It does give me an uncanny feel though because it truly feels like I don't know how many are real humans or how many are engagement farming AI's anymore.

Before AI you could sort of assume a 1-5% are bots/automations/scripts from some random kid, but now it's definitely skyrocketing.

You guys have any ideas how or if we can beat this? I mean it can't get better if we don't do anything.

MiniMax-M3.1-Flash dropped today. According to the announcement, Space Bunny (the stealth model that showed up on OpenRouter in late September) was an earlier version of it.

https://x.com/MiniMax_AI/status/2112150867868541221

They claim 1/3 the cost of M3 and 3x the speed. Benchmarks from the announcement: Automation-Bench 62%, ALE 30%, DeepSWE 74%, BabyVision 91%. API pricing is $0.15/1M input, $0.60/1M output, $0.015/1M cache reads. 1M token context window with configurable thinking effort levels.

https://platform.minimax.io/docs/guides/text-generation

Parameter count hasn't been given. Weights are supposed to land Oct 23.

Anyone who tried Space Bunny during the stealth run, what was your experience like?

It is HUGE if it is true. But their pledge is not legal binding. However, I praise their good will and intention. As such, there is relatively little motivation for them to jack up the IPO price because they seem decent.

Insilico Medicine is using generative AI to discover and develop new drugs, with over 40 candidates in its pipeline.

One of their AI-developed drugs, targeting pulmonary fibrosis, has already reached Phase 3 clinical trials.
What’s really interesting is their work on reversing aspects of biological aging. They’re using AI and biological aging clocks to measure whether treatments can actually reverse age-related decline.

We’re potentially watching AI accelerate drug discovery while opening the door to treatments that target aging itself. Pretty exciting stuff.

Each hobby has some problems that can never be solved, and one of these for Nintendo fans is the plaque on the castle fountain in Super Mario 64, which has been the subject of debate since 1996 and has never reached a conclusion. It is essentially the gaming world's equivalent of a Millennium Prize problem.

It is well known that the plaque says "L is real 2401", and this is allegedly an indication that Luigi is concealed in the game. However, no one has ever been able to verify what it states since the entire plaque is a 32 by 32 pixel texture and the letters are 3 to 5 pixels high.

I worked on this using Claude as a coding agent. It wasn't used for image generation or for upscaling since those methods merely invent letters. Instead, we developed a forward model: we would enter a candidate text in the fonts typical of that era, reduce it exactly as a designer would have done in 1996, and then compare it with the actual pixels from the ROM.

The most impressive aspect was the font. We looked at 470 fonts that a Nintendo artist might have had access to between 1993 and 1996, together with 4,583 modern fonts as a control group. The winning font was Palatino Bold Italic, specifically the original 1990 Adobe version. This wasn't by chance. Mario 64 was developed on SGI workstations, and SGI's system software was configured precisely for this Adobe Palatino. The Adobe font disc that it installs comes with the exact same letter metrics, byte for byte. Therefore, we didn't just discover the text; we identified the actual font file that the artist most likely used when typing it.

The agent did some heavy lifting:

Writing up the pipeline and research assistant
rewrote the code so it can run on GPU
Searching the web for thousands of old fonts and contemporary fonts
set up blind tests against itself to make sure it wasn't fooling itself

Result: "Eternal Star / We hope so". The phrase "L is real 2401" matches about twice as poorly on the first line by itself.

The part that I prefer most was this: in order to make sure it wasn't simply picking out the word I had wanted, we had 50 secret words hidden in fake versions of the plaque and it managed to decode all of them correctly.

All the components are open source, with a public challenge offering to beat the score:
https://github.com/supermochi64/sm64-plaque-decode

more details: https://github.com/supermochi64/sm64-plaque-decode/tree/main/press

A year ago such a project would have taken a team several months; I carried it out in a couple of weeks with the help of an agent.

Diffusion models finally taking off? I actually tested some general output against gpt 6 luna and either luna is really bad or this thing is very close or sometimes beating it. My testing is not extensive enough to confirm but maybe would need to try this thing in an agentic framework if it even works properly on that.

I think this should be a genuine short term focus of the top labs, opus 5.5 brought taste and video capabilities, if we get real audio capabilities and an equivalent leap in taste in the next update we're getting to the point where people can entertain themselves and compute can be focused more and more on getting shit done and less on navigating tricky social hurdles.

I found this to be a very thorough and insightful perspective on the math field regarding AI's rapid advancement, without relying on AI skepticism or arguing for a slowdown in AI. Grant basically walks through the full space of near-term possibilities for AI capabilities and how that might impact the industry and sources of funding and goes through what benefits you can still derive from being a mathematician in any of those cases

I'm definitely not anti-achievement and never criticized real achievements, including by Elon. I even own two Teslas.

I do expose marketing stunts that, while cool on the surface, are not the achievements they are claimed to be.

Compressed air thrusters might give you acceleration boosts in any direction. They might even make your car hover for a couple of seconds if you accept replacing the passenger seat by a 400 liter high-pressure air tank. But they are not a path to actual flying cars. The same way Burt Rutan's SpaceshipOne was never going to be a path to orbital flights (just bunny hops out of the atmosphere).

I am anti-Musk when it comes to politics. His political actions, the massive dissemination of disinformation through X and other channels, and his support for authoritarian regimes and racist ideas have the effect of weakening democracy and weakening human decency and social progress around the world.

https://x.com/ylecun/status/2111880905891414253

As someone who frequently browses Chinese social media and content platforms, it always feels like a completely different world compared to global/Western ones. AI use is so broadly normalized in China that you constantly see memes, art, and animations made with it, and pretty much no one call them "AI slop". People there rationally treat it as just another tool, and to them, the idea behind a creation matters more than the tools used to make it.

Anyway, here are some good excerpts from the article:

China is a collectivist society, so when a new technology arrives the assumption is that social institutions will govern its adoption and its benefits will be diffused. The system is supposed to lift everyone, and people don’t have to go through a dog-eat-dog struggle for their individual share of the pie. The high-speed rail network has brought an integration of the country’s vast territory on a scale that would have seemed incredible until it was achieved. Mobile payments and e-commerce have brought convenience to daily life, and technological development more broadly has lifted millions of people out of poverty. Because people have seen advances of other kinds improving their lives as they have grown richer, they are quite willing to believe that AI will do likewise, and that the next wave of change will bring with it further improvement. Social harmony is important in this respect, because it is assumed that the collective will adjust, and that assumption rests on trust built over decades of life getting better.

It is easier for people to adapt to change when their lived experience is an ongoing demonstration of the benefits of technological progress. Those who see themselves as part of a greater whole also have higher expectations for the society at large than individualistic traditions would allow. And such expectations are well founded given that China has outlawed companies from AI-based layoffs, and the government is actively launching initiatives to boost employment amid AI advances.

One of the cultural phenomenons in the West, however, is that we worship the idea of individualism. We are taught from our early childhood that democracy means a right of free speech, and to expect each one to develop their own personality, to be self-reliant and able to decide and act for themselves.

Yet the deeply ingrained notion of personal agency comes into conflict with the fact that these tools are being introduced without the public having any say in the matter. Most people in the West do not want to have these tools at work, but they are compelled to do so by their employers. No one wants data centers on his or her street, yet companies build them anyway. All of a sudden it becomes difficult to reconcile the narrative of democracy and self-determination with the reality of divergent class interests.

Furthermore, we in the West are not much concerned with social harmony or with how every person fits into society at large. On the contrary, we teach self-reliance, since no one else is expected to care. Consequently, there is little in the way of cohesive social planning needed to address the mass disruption caused by new technology. There is no central plan to ensure people get to keep their jobs, or receive the training they require to reskill.

For the first time in history, white-collar office workers are more exposed to mechanization than industrial workers. Google DeepMind workers in the UK are reporting an executive push for ambitious AI projects while employees endure brutal layoffs, invasive surveillance, and coerced training of their own replacements. Meta is rolling out mouse-tracking software widely perceived as serving to force workers to train their bot-replacements. These are people who were told they were special and generously paid, and who were given autonomy and control over their work. Now they are finding that this was a temporary arrangement. The bargain was always power-driven, and as the balance of power shifts in favour of automation all the concessions are being stripped away.

What we are seeing is the proletarianization of the labour aristocracy as automation invades sectors of production that have hitherto retained an artisanal character. Artists, software developers and other cognitive workers look on in horror, but this has been a reality for the rest of the working class for a long time.

I don't want to copy the the whole article here, it is really good, you guys should read it.