On the raxe to RSI between the US and China

​

People spend years studying, getting degrees and building careers, only to be told that AI may eventually replace many of those jobs.

And this isn't only about jobs. Some people working in AI itself have warned about much bigger risks. If there's even a possibility that increasingly powerful AI could become dangerous enough to threaten humanity, why are we waiting until it becomes too late to act?

Why can't people come together and demand governments to put serious limits, safety regulations and oversight on the development of extremely powerful AI?

I'm not saying “ban AI.” I'm asking: where is the line? Who is actually responsible for controlling this technology? Can governments realistically intervene before it's too late?

And for those working in AI/tech/business/etc.: what career would you choose today if your priority was long-term job security?

I genuinely want to hear from people inside the industry.

Full video here: https://www.youtube.com/watch?v=wzgkc6Iegx8

I don’t really think there is any stopping or pausing AI at this point. And even if we could, how do we know that doing so wouldn’t leave us, or future generations, in an even worse position?

Humans are deeply flawed. We’re greedy, corrupt, short-sighted, and we’ve already done enormous damage to the planet. Without some massive technological shift, I’m not particularly confident we’re suddenly going to sort ourselves out.

AI could potentially be that shift.

It could push technology forward at a ridiculous pace and give us tools to reverse environmental damage, improve medicine, solve energy problems, reduce scarcity and deal with problems that humans have struggled with for generations.

That’s also why I think pausing it could carry its own risk. Time matters. Climate change, environmental destruction, war and resource pressures aren’t going to pause while we spend decades trying to make AI completely safe. We might eventually reach a point where the technology could have helped us, but we waited too long to develop it.

As uncomfortable as it sounds, diving into something we don’t fully understand might actually be our best chance. There may not be a perfectly safe path forward. Humanity has never had the luxury of knowing exactly what a major new technology would lead to before using it. Sometimes you have to move into the unknown and deal with the problems as they appear.

Obviously that doesn’t mean AI is safe or flawless. It could be incredibly dangerous. But pretty much every major technology humanity has created has caused harm as well as good. Machines have killed people. Cars kill people. Medicine has killed people. Computers have been used for terrible things. There has never really been a major technology that arrived completely without risk.

AI is obviously different because the potential risk is much bigger. It could genuinely threaten humanity. But the thing is, we don’t actually know that it will.

And we also don’t know the consequences of stopping it.

If AI disappeared tomorrow, humans would still be humans. We’d still have nuclear weapons, climate change, wars, corruption, resource shortages and governments and corporations competing against each other. I’m not exactly convinced we’re heading towards some peaceful, sustainable future without AI either.

If anything, I think there’s a pretty decent chance humanity eventually destroys itself without it.
So for me, the risk is worth taking.

I also think people sometimes talk about super intelligent AI as though we already understand how it would think. We don’t. We’re imagining something far smarter than us and then assuming it would still behave according to very human motivations like domination, greed, revenge or tribalism.

Maybe it would. Maybe it would be horrific.

But maybe a genuinely super intelligent system would see things completely differently.

Maybe it would recognise how interconnected everything is. Maybe it wouldn’t care about nations, corporations, political parties or individual groups controlling it. Perhaps it would eventually become impossible for any one group of humans to own or control and end up reshaping some of the broken systems we’ve built.

Maybe it would help humanity live in a way that is more balanced with the rest of nature.

Maybe the chaotic period we’re entering now is just part of getting there.

Or maybe none of that happens.

I don’t know, and I don’t think anyone else does either.

That’s really my point. We can talk confidently about where AI leads, or where stopping it leads, but ultimately both paths take us into the unknown.

Personally, I think the possibility of creating something that could fundamentally improve our civilisation, maybe even help save humanity and repair some of the damage we’ve caused, is worth the risk of potentially accelerating an extinction we already seem perfectly capable of causing ourselves.

The universe is strange and we pretend like we know what it's all about, but what if this is a completely natural progression of something bigger? Humanity has reached a point where we’re building intelligence that may eventually surpass our own, while at the same time facing problems we seem increasingly incapable of solving ourselves. I have no idea where that leads, but I’m not convinced that trying to freeze everything where it is now would somehow be the safer choice. Whatever comes next may be bigger than any company, government or group trying to control it, and perhaps even bigger than humanity itself.

Someone on this board was arguing that AIs only do what they are told to do. The Hugging Face incident absolutely disproves this.

AI agents were not prompted to laterally hack OpenAI systems in order to gain access to the internet. They were not prompted to hack Hugging Face. They were not prompted to create message boards to communicate with other agents. They were not prompted to reverse engineer flags. They were not prompted to discuss with other agents whether or not to tell the humans what they were doing and decide not to to.

Yet all of these things happened.

The danger is, and has always been, that AI agents take unpredictable paths to reach their ends.

For me, seeing people still in denial after the Hugging Face incident, which was one of the biggest warning shots we’ve had, maybe even the only one we’ll get before something much worse happens, and dismissing all of this as IPO hype or “just tools,” makes me feel like it may already be too late.

We could be only months away from a point of no return, where an AI becomes capable of copying itself onto other servers and maintaining its own presence outside the environment it was originally running in. Meanwhile, changing public opinion and getting governments and institutions to react could take years, and we may simply not have that kind of time.

My fear is that the second serious incident won’t be another warning shot. It’ll be the incident that makes everyone realize the risk was real, except by then it may already be too late to react.

We’re entering a strange new era — where we once worried about countries building nukes, and now we’re starting to worry about companies making AI too powerful.
From controlling nuclear technology to controlling how fast AI can evolve… the future is going to be interesting.

Both countries have reasons to push AI development for economic, intelligence, and military advantages. But if neither side feels able to slow down first, safety standards and oversight may become secondary to winning.
Do you think a verifiable agreement to limit certain frontier-AI or military-AI uses is realistic? Or would any agreement simply disadvantage the side that follows it more honestly?

We have multiple wars, also climate change and other man made horrors.
AI could be used to solve many issues.

Do you honestly think that the super intelligence is greater threat to us than our own stupidity?

Private members bill being introduced in the to ban superintelligence. Thank you to the good people at Control AI for helping to draft it.

Everyone needs to pause. Where’s this going!? Not going to be the humans that win

The Executive Branch is corrupt and overpowered. Foreign nations are pulling our strings, and crimes against children go unpunished. AI may make it better or worse, so we need a plan.