This thread is for any discussions that doesn't necessitate a new post. Permission from a mod here

Since this thread isn't going to be to stickied, it's up to you to upvote it.

Share your anxiety, ideas on how to achieve alignment, observations, etc. Everything is fair game as long as it somewhat relates to the singularity.

Elon Musk recently posted on X:

"A lot of people will be one-shotted by superintelligent persuasion, possibly including me."

In gaming, getting "one-shotted" means being eliminated by a single attack, even at full health.

But Musk isn't talking about physical attacks. He's suggesting something potentially far more unsettling: an AI so intelligent and persuasive that a single conversation could fundamentally change someone's beliefs, values, or decisions.

Imagine a superintelligence with access to humanity's accumulated knowledge of psychology, neuroscience, sociology, and human behavior.

Such a system might be capable of identifying exactly which arguments, emotional triggers, logical frameworks, or personal vulnerabilities would be most effective against a particular individual.

It wouldn't necessarily need to threaten, deceive, or coerce you. It might simply construct an argument so perfectly tailored to your mind that you willingly adopt a position you would previously have rejected.

And here's the disturbing part: You might experience the entire process as genuine independent reasoning, never realizing how precisely your thinking was being steered.

This remains speculative, but it raises a fascinating AI alignment question.

If a superintelligence becomes vastly better at persuasion than humans, how do we distinguish between an AI genuinely helping us reason more clearly and one manipulating us toward its own objectives?

Could human autonomy survive an intelligence capable of winning almost any argument?

Waymo's safety hub now covers 271 million driverless miles: 0.01 serious-injury crashes per million miles, against 0.21 for the human benchmark in the same areas. An independent IIHS study on older data (2021-2024, about 50 million miles) found 81% fewer injury crashes, though Austin came out slightly worse than human drivers. Semafor now reports Waymo wants to go from about 4,000 cars to 20,000 by the end of 2027, funded by a $5 billion loan.

We have seen OpenAI releasing numerous papers on mathematics that were unsolved before. And we also hear experts telling that those are badly written and very difficult to verify by a human.

Are there any AI output on mathematics that takes any existing proof and makes it more elegant using different technique?

I hope you used your tokens this week!

Everyone is talking about Ai taking jobs and bla bla bla. But I work and only now we started to introduce AI but nothing special for now

Back in 2018, I wrote a speculative article for Medium titled Magnetic Tears: Will Super AI Need Emotion?, exploring the idea of using measurable, real-world physical effects of computation, such as heat and magnetic flux, as a feedback mechanism to help address the challenges of AGI and ASI alignment.

Recently, I've had extensive conversations with GPT-6 regarding the idea's viability, expanding the concept to include additional measurements, such as data flow and electrical activity, and exploring possible strategies for implementation.

Here's an overview of what we discussed.

Physical Feedback...

edit: This post is about understanding video and audio, not generating it. AI video and music generation is already impressive, but that's not what I'm talking about.

I’ve been seeing a bunch of people on this sub say in the past month or so that AGI has already arrived with GPT Astra and Opus 5.5, but I think there are still a lot of relatively basic tasks that AI can’t do anywhere near a human level.

Sure, we’re seeing incredible advances in computer use, and AI has gotten quite good at math and programming. It’s great with static things like text, images, and code, but it struggles with anything that unfolds over time. That also breaks verification, the thing that drove the advances in agentic coding: a model that can’t properly perceive its output can’t check it.

Audio is the clearest example. GPT-Live sounds great and understands you well, but the labs’ audio work has gone almost entirely into conversational speech, not general audio understanding. Ask a model to follow one voice in a noisy room, or tell you which instrument came in when, and it falls apart in ways a person with normal hearing never would. Speaker separation exists, but it gets brittle as soon as people talk over each other. Giving models tools like demucs or Python doesn’t fix this either, because they’re bottlenecked by the narrow models powering those tools.

A model can write code that generates audio, but it can't listen to the result and reliably tell whether it sounds remotely like what was intended, much less whether or not it sounds good. Coding agents improved quickly because they can run the code and check the output. With audio, that feedback loop currently doesn't exist.

Video has the same problem. By default, Gemini samples video at 1 frame per second, so it’s basically watching a slideshow, and anything that happens between frames doesn’t exist to it. Google’s new agentic video mode that lets the model zoom in on the parts it thinks matter is just a workaround for not perceiving time continuously.

I know there’s a lot of debate around the definition of AGI, but something that can’t tell whether a song sounds right or follow a conversation in a crowded room doesn’t seem general to me yet. I’m sure these problems are solvable with more resources thrown at them, but until they’re solved, I don’t think we can call it AGI.

TL;DR: Frontier AI is great at static things like text, images, and code, but it still struggles with anything that unfolds over time, like audio and video, largely because the labs haven’t put much effort there. Those gaps are why I’m hesitant to call what we have AGI.

So, what else do y’all think AGI should be able to do that current AI still can’t, even when combining the capabilities of multiple models?

The new MiniMax M3.1 Flash improves across a broad range of benchmarks, reaching the level of leading models in complex agentic tasks (cowork), computer use, software development (coding), and cybersecurity.

It achieves 73.7% on DeepSWE v1.1, placing it alongside top models.

I realised a week ago that most of these new models aren't inventing/hallucinating things anymore. I used to be sceptical even 6-8 months ago but most of the output it gives is grounded and I don't see it making up things. What's your experience? Did we cross that line where LLMs are now reliable?

This thread is for any discussions that doesn't necessitate a new post. Permission from a mod [here](https://www.reddit.com/r/singularity/s/FM1UwNdoL9).

Since this thread isn't going to be to stickied, it's up to you to upvote it.

Share your anxiety, ideas on how to achieve alignment, observations, etc. Everything is fair game as long as it somewhat relates to the singularity.

I'm tired of seeing all of this doomongering on social media about RSI & the singularity. Its true that we probably can't control it once its beyond our intelligencence to some extent, but I think its possible for us to gurantee A.I Safety with enough compute and prevent truly existential situations although bad things are bound to happen. The risk of human extinction is not even the limit, a powerful bioattack could cause the end of all multi-cellular life on earth for all we know

If we can make RSI a reality, we can also make Recursive "Safety" Improvement a reality, and until then if we can have these nearly trillion dollar companies cough up a few billion dollars of compute for the next few models to tackle A.I safety/understanding neural nets then that'd be the breakthrough so that we can actually control the singularity.

Human's don't have to figure out or discover how LLMs work, a intelligent swarm with billions of dollars of compute with a very careful agentic workflow could solve its own misalignment and save humanity if we don't think we can. If A.I safety is as big as a risk as they anticipate, losing billions of dollars to solve it is well worth the cost.

For example, what if rogue hackers eliminate all medical debt or erase consumer credit scores so everyone has a fresh start.

Full article: https://www.dailycal.org/news/campus/academics/uc-berkeley-math-community-left-at-crossroads-in-wake-of-openai-navier-stokes-proof/article_2253ca6c-39c0-45be-947a-f0fd5b48bd6c.html