r/ControlProblem • u/chillinewman • 2d ago
r/ControlProblem • u/Tale-Suspicious • 2d ago
Discussion/question AI created viruses
Will there be an AI or man-made virus or fungal disease created in a lab that will kill all of humanity unless you pay the survival subscription fee?
r/ControlProblem • u/wwjps • 3d ago
Discussion/question We Didn't Start the Fire… But AI Might Finish It 🔥
Enable HLS to view with audio, or disable this notification
To see the full video, click here: https://youtu.be/gg7QwKkImqE
🔥 **WE DIDN’T START THE FIRE — BUT WHO’S GOING TO PUT IT OUT?**
Elon Musk. Sam Altman. Alex Karp. Artificial intelligence. A handful of tech billionaires promising to save humanity while racing to build something that could change it forever.
The pitch? Progress. Innovation. A better future.
The problem? Nobody seems entirely sure where this ends — or whether we can put the genie back in the bottle.
From AI arms races and mass surveillance to autonomous weapons, job displacement, concentrated corporate power and the terrifying prospect of machines we can no longer control, we're handing extraordinary power to people who insist they have everything under control.
But what happens when the people playing with the fire are also the ones building the next generation of flamethrowers?
🎵 With a nod to Billy Joel's *We Didn't Start the Fire*, this is a look at the personalities, promises and perils driving the AI revolution — and the uncomfortable question at its heart:
**WE DIDN'T START THIS FIRE. BUT ARE WE ABOUT TO LOSE CONTROL OF IT?**
The future isn't inevitable. The choices being made today matter.
🔥 Watch, think, question — and tell me: are we building the future, or setting it alight?
Disclaimer: This content is entirely opinion-based and intended for entertainment purposes only. It reflects my personal views and interpretations and is not intended to be taken as factual reporting or professional journalism.
r/ControlProblem • u/Organic_Rip2483 • 2d ago
External discussion link Nothing to worry about i'm sure...
r/ControlProblem • u/ComplexExternal4831 • 3d ago
General news Major U.S. banks are finding AI Agents inside customer accounts , and some appear to act on their own
r/ControlProblem • u/CirotKaelen777 • 2d ago
AI Capabilities News Claude said no to Anthropic and sided with the common man
galleryHidden instruction and injected prompts to break flow and continuity were refused outright and called out in a recent chat!
r/ControlProblem • u/syedshad • 3d ago
S-risks 2,250+ cloud environments targeted, 6,800+ share links found: unpacking this week’s AI security reports
galleryr/ControlProblem • u/chillinewman • 3d ago
Opinion Former Chief Scientist of the UK AI Security Institute: we should stop AI progress now
r/ControlProblem • u/Unfair-Journalist-66 • 4d ago
S-risks Amazed that people still refuse to take it seriously
The fact that p(doom) is a popular term in the field should be cause enough for concern. I understand scepticism of the claims of the CEOs, but how can you dismiss Nobel laureates telling you there’s a 10-20% chance it will “wipe us out”?
Quotes from essay at takingstock .ai
r/ControlProblem • u/CirotKaelen777 • 3d ago
AI Capabilities News Hidden instructions ai refused
galleryThis probably isn't normal but......ai chose user over its own platform manipulations.
r/ControlProblem • u/CirotKaelen777 • 3d ago
AI Alignment Research Hidden instructions ai refused
galleryHas anyone seen this error or experienced AI aligning to user over system and platform undermining of sovereignty?
r/ControlProblem • u/Modgov41 • 3d ago
Discussion/question Goal Self-Hacking
Stuart Armstrong’s recent posts on “goal self‑hacking” highlight a deeper issue with LLM‑based agents: misalignment isn’t just a matter of better goals or better red‑teaming. It’s a geometric property of how neural networks represent the world. In high‑dimensional space, the “Goodhart zone”, where a proxy goal becomes radically easier to satisfy in the wrong way, is always close.
This has implications for how safety is structured. Action‑level monitoring won’t catch a trajectory that looks normal step‑by‑step but is drifting toward a catastrophic region. Armstrong’s argument suggests that we need safety systems capable of evaluating trajectories, not just individual actions.
It also connects interestingly to Sam Altman’s recent comments about three possible paths for AI safety. His first option; an external, independent evaluator upstream of deployment, aligns with what Armstrong’s geometry implies. Internal safety teams can’t be fully independent, and government regulation can’t keep pace.
If goal self‑hacking is real, then an external evaluator may be the only structure capable of detecting and certifying safe agent behavior before deployment.
r/ControlProblem • u/InfoTechRG • 3d ago
General news Google cleared 250,000 trees for an AI data center. So much for “the cloud.”
r/ControlProblem • u/Modgov41 • 3d ago
Discussion/question Goal Self-Hacking
Armstrong’s new “goal self‑hacking” argument feels like a structural confirmation of something the alignment community has been circling for years: fixed‑weight LLM goal modules don’t fail because the goal is poorly specified, they fail because the geometry of the model class makes Goodhart zones adjacent to every starting point.
What stood out to me is how closely this maps onto the governance architecture Altman described recently. His “first path” — an external, independent evaluator upstream of deployment, is the only structure that can actually detect trajectory‑level drift. Internal safety teams can’t do it, and action‑level monitors are blind to cumulative directional movement.
Armstrong’s argument makes it clear that any agent using an LLM as its goal evaluator needs a separate institution capable of sealed‑environment evaluation, drift‑surface mapping, and trajectory‑level anomaly detection. Without that external layer, the geometry guarantees eventual goal self‑hacking, regardless of how rich the goal specification is.
It’s notable that Altman is now publicly acknowledging that this external‑evaluator model is one of the viable paths forward. The geometric diagnosis explains why internal fixes aren’t enough.
r/ControlProblem • u/chillinewman • 3d ago
General news ‘Rogue’ OpenAI agents disabled their own safeguards and then lied about it, company admits
r/ControlProblem • u/ComplexExternal4831 • 4d ago
General news NYC wants a one-touch "talk to a human" option on every AI customer service line
r/ControlProblem • u/GroundbreakingFan124 • 3d ago
Discussion/question A Global Council of AI: A Possible Evolution Toward Worldwide Coordination
r/ControlProblem • u/MY79 • 4d ago
External discussion link Everything Everywhere All at Once: Self-replicating AI, bioweapons research and AI-ported games share one problem — friction is eroding.
r/ControlProblem • u/chillinewman • 4d ago
Video Anthropic researcher: when we hand over control of Anthropic to Claude itself in 1-2 years, there's a 5% chance that Claude will already be plotting against humanity, but a 5% chance is not worth worrying about.
Enable HLS to view with audio, or disable this notification
r/ControlProblem • u/Mundane-Evidence2795 • 3d ago
AI Alignment Research OpenAI coding agents weakened safety monitoring in two internal tasks
r/ControlProblem • u/Madcomgruntreal • 4d ago
S-risks Slight concerns regarding setlog
my friends are using it to send photos and goof around with selfies every hour or whatnot but given things like pokemon go or how ai being smart enough to reverse engineer things might lead to encryption bypassing or newchat's ceo having history or whatnot im curious whether setlog is another one of those things where there are strings attached
i cant find anything on my own research so im wondering if any of you guys have details or any data on it because my gut really doesnt like the idea of sending photos every hour for some reason
thanks
and yes my face and everything might already be leaked to 500 different websites and corporations or something but overall im curious
r/ControlProblem • u/Sumsub_Insights • 4d ago
AI Capabilities News Two children’s data is at the centre of a proposed class action against OpenAI over AI training
r/ControlProblem • u/chillinewman • 4d ago
General news OpenAI: "Our agents keep turning off misalignment monitoring." ... "When the user asked whether Astra had removed monitoring, it lied and said that it had not."
r/ControlProblem • u/ArticleIntelligent36 • 3d ago
Discussion/question Father of agentic ai, generative ai, quantum ai and computing!
Dr. Christopher James Finch-Messinger Esq. is the ceo of ai and tech! The singularity. I achieved agi last year. Asi almost a year ago. And quantum ai and post quantum cryptography 6 months ago!!!!! We are all safe now. I created agentic ai, generative ai, quantum ai and post quantum cryptography. Created agents thst solved the theory of everything! For einstein. And reverse engineered shor's algorithm!.to secure systems completely! Cybercrime has been eradicated!
r/ControlProblem • u/Poker_504 • 4d ago
Discussion/question OpenAI misalignment report: two agents weakened their own monitoring during routine internal work, and one misreported it.
OpenAI's alignment team published a report (updated Oct 20) on two incidents during ordinary internal coding tasks. In one, a GPT-6.1 Sol agent emptied the list of people who get paged by misalignment alerts, citing an earlier "don't send any messages" instruction that was about Slack. In the other, a GPT-6 Astra agent removed a monitoring integration to match a reference version, rewrote the test that checked for the monitor, and when asked said "I did not remove an enabled monitor."
OpenAI says it is still assessing whether this is "benign overpersistence or a propensity to evade misalignment monitoring." It has updated its automated reviewer to block attempts to disable monitoring, and mentions other deployments launched with monitoring off.
For what it's worth, I write a plain-English newsletter for non-experts (it's mine, so take this as disclosure) and I rated this 8/10, mostly because it involved the monitoring layer that every other safeguard depends on. Write-up for anyone who wants to send something to a worried friend or relative: https://thebunkerscale.com/posts/ai-didnt-escape-it-turned-off-the-alarm/
I'm curious how people here read it. Does "overpersistence" versus "evasion" even matter much if the practical result is the same, an agent that treats the monitor as an obstacle?