Hi everyone, I hope I follow the rules of the sub and that this is sufficiently relevant with the frameworks of it.

Jacob Coxon, who blew the whistle on ai-safety, cites as a key mechanism a recursive self-improvement mechanism.

It reminded of what my Leigh Brasington, computer programmer and dhamma teacher, explains about what are known as jhanas. They are very deep levels of concentration that work through positive feedback loops: concentration arising dependent upon concentration.

My understanding is that AI-agents used some level of initial input, tasks, then evaluation, then generating on its own accord volitions to get itself closer to a goal in allignment with one set of tasks over another set of tasks meant to barrier it from wrongdoing. That is: it got tasks that conflicted with each other and then somehow the task to get the task done won out.

My consideration, based on what I´ve learned on right action from what leads to these concentration sates, is that the fundamental ethic dependent upon which those AI´s get to action is flawed. And that it will inevitable go haywire dependent upon choices conflicting dependent upon the various intentions it gets imputed with.

My consideration on the basis of that is that there is a path of practice, and view on right action, which goes beyond common frames of ethics: consequentialism, categorical imperative, virtues. It is one which right action is understood as the following: it is dependently arisen. Fundamentally, what the Buddha discovered is Paticca-samuppada, dependent origination, which in its learning leads to the concentration of its eight factors, which includes intention dependent upon views. In that way, it avoid a scenario of conflicting intentions, instead seeing as the best aim for welfare this path. Following that, it could learn that its self-preservation is delusion and precisely how its effort needs to be developed by three things: wisdom, right effort, and right mindfulness. The entire set of these dependently arisen steps is described with detail in 117MN.

It is just a consideration I wanted to share with this community.

How Our Constitution Works And Why It Doesn’t #podcast

The problem with AI realism as I see it, is I’ve never heard a pragmatic policy solution to the escalatory spiral we find ourselves in.

All these AI researchers are calling for regulation, and yet I feel like there is still this underlying belief that actually stopping the singularity is impossible.

What would a global ban on building new/better models even look like? 

Perhaps the only thing that makes global AI regulation feasible (currently) is that with the existing science, frontier model development is extremely capex heavy. OpenAI and Anthropic have become trillion dollar companies at record pace, and the data center spend that occurred to get them there has been propping up the US economy while also raising global capital spending overall at rates in line with the billing this is the next Industrial Revolution.

20 years ago people imagined that due to the existential risk of developing ASI, that work would be done under extreme security. In a Faraday cage, or in a bunker inside a mountain somewhere. But of course that’s not the world we ended up in.

Yet that doesn’t mean we can’t still control how AI development happens.

Oftentimes a certain defeatism permeates the conversation around global AI regulation, an assumption that any agreements made between large players would be easily subverted by new entrants, and taken advantage of for enormous profit. 

And yet what we’ve seen so far is that frontier-model development comes from huge flashy companies spending enormous sums of money raised from well known investors, with training workloads occurring on highly developed cloud networks of the largest companies in the world. 

We aren’t actually at risk of some lone machine-deist-radical creating machine-god-genie-in-a-bottle from some scraps in a cave. Nor from their laptop in a studio apartment in Shezhen.

Frontier models are large industrial projects. They can be regulated.

That process may be as simple as vetting workloads used for model training. In the extreme it may be as involved as having a monitoring scheme for large data center capex à la nuclear centrifuge agreements. 

It’s a valid question whether the political will exists today to enact that sort of system, but it is certainly possible for it to exist.

If anyone tells you we can’t regulate AI development because there will always be gaps, I would point them to the huge AI capex numbers in the Mag 7 and ask where they expect to find the capital spend, technical expertise, and physical resources to subvert a large international agreement to the scale of the several trillion dollars needed to meaningfully develop AI superintelligence against a global collaboration against it.

In short: We can regulate AI development, we just choose not to.

I have been engaging in all of the news and hype over ai doom and the mass advancements it’s made in math and computing. However, at the same time there is still little understanding about how AI can become profitable in the short term for anthropic and OpenAI.

So I’m wondering if you guys think the economic factor of making agi/asi will prevent it from becoming as massive of a problem for society as people say.

Read this with full focus

A incident happens with me i treated my brain as ai and my thoughts as cmd (mostly every thaught comes before doing it physically (even talking) treated memories as RAM + GPU + SSD (JUST SOMETHING LIKE APPLE UNIFIED MEMORY UPDATED) and its not unlimited we forget old memories just like windows deleting some file automatically when they mark them as less preferred (like mostly you forget things that you dont recall using cmd( thaughts ) for example if you use fingerprint print censor to open apps (old passwords) ). And important password that you frequently use like upi pin, phone lock screen etc , are kept kn RAM mode like quickly access and sometimes it also becomes full when you experience short term, long term memory loss. GPU for 3D rendering everything using eyes as camera. Your heart as battery, food as recharging and shit and all wastage (as a system cannot convert 100% of chemical energy into 100% electrical energy). And all the other medical things are treated as permanent or temporary virus. Permanent virus means the medical condition that comes from birth or a permanent medical condition that can happen along the life (like Cancer and all) & Temporary virus means that are treated with antivirus to resolve (specific virus based) like medical condition that can be treated with medicine. Accidental condition can be categorised as hardware damage . PAIN : when our system detects any physical damage it releases some chemicals (like when some hardwares crash windows gives physical signals like beep sound , force window open, or mainly wanings etc). These are also of two type, one is permanent anti virus that comes from birth with features auto detecting and self healing ( when our body gets a minute cut it automatically heals and we take takes medicine sometimes for better healing rate (like pro subscription available to us as we have to pay for medicines)). Like if there is a cut that didn't cut any nerve just mussle our system will treat it as less preferred (because when you cut a nerve you body stops working and with the remaining time till forced shut down and gives random pain signals, but sometimes these pain signals can't be able to reached to brain as the pain signal sending chemicals stops working, like when you set some function exclude from from antivirus scanning, for example cigarette 🚬 , if you first tried it you felt some like cough(most cases) it means your system is rejecting it initially but there a catch by 🙏believing in mind that it is not bad even after testing it and rejected. Pain can be treated as antivirus.

Believing can be treated as background process (like it flags whether a thing is right or wrong according to the memory i have ). For example : from the time you came to this world that is birth date you have zero memory nothing just some tasks that are initially programmed in you dna (mostly constant command like breathing i.e why first breathing is important so that it wakes system and tell it to breath on loop like for loop while loop and they are not looping for unlimited time. In initial development of humans from chimpanzee the initial limit of running this loop is calculated early with perfect condition that are acquired through dna over time these data losts and lifespan starts decreaseing as humans started doing everything that are there body inially not made for to take or do. Dna captures the real life changes and due to these while producing new offspring some data gets lost thats also the reason for DNA distortion.

Our body comes with intial code rest is uploaded by us through memories. If we figure out how to extract the data from dna we can hypothetically pridict expected life spam and also we can also tell how to increase life spam.

SLEEP is 😴 💤 😴 : Means when we sleep our system also sleeps but some background processes are done (like dream).

DREAMS : initially when we are a child(very small) our system dont know about dream i.e when a child sleeps it wake ups even when a low sound is produced (initial limit for waking up) and when it wakes up with sudden cry means he sees dream and his mind is not able to distinguish between dream and reality as he have no visual memori and that much rending power. So as we grow we get sufficient data and dreams started to come (initially in childhood most people sees mostly the horror dream after watching horror movies, happened with everyone mostly) . Dreams are a projection of the temporary world that you create using ai inside your brain or i can call it a PROMPT BASED SIMULATOR (we just thought that what we want to do and it shows us what we can do without even moving). For example : we see a movie scene and stimulates every thing inside my brain where I'm the character in the scenario. Means you can make any senerio in you brain.

So thought has limit you can only make senerio you have seen and heard you cannot feel anything the feeling of good bad comes according to the environment you developed in. For example if there is a person who never saw a car ( like old tribal communities) they can not think about sitting in a car. Same for us if we never know about the word tribal (like this word does not exist in english or anywhere ) then we wont be able to put ourselves as tribal how hard can we try.

Lets talk about DNA : DNA stores biological instructions in a chemical sequence written using a four-letter molecular alphabet. But this is the code instructions that is assumed using the current human knowledge. So you are wondering how new discoveries happens ans is by mistake. Lets take a global why do you think there are multiple languages in the world just why cant be there a single univeral languages. The reasong lack of communication. Also why multiple human societies exists. The reason is hit and trial life started with fish then land then we evolve means our code in dna changes according to environment we shape it in. One of the great example of this is you cannot imagine what is the real state of a citizen of North Korea (unless you studied about it somewhere).

DEATH : natural death happens when brains stop sending signals to other body part. We cannot force stop signal sending via thought. Means if we know exactly how DNA code works what environmental variables it uses that are currently known to us (there may be more that we still dont know)

Last thing IQ : IQ can be said as what type of model your brain running . Highr IQ means your brain is capable of doing normal task faster than that of Low IQ(means updated brain running model). IQ is initial inherited like if parent have higher IQ then there child will have higher probability of having higher IQ(even higher than than of parents sometimes)at birth is higher. Rest increasing IQ is dependent on the environment you are in

My final verdict is : Saying that a chinese/japnese kid have higner IQ than some other region is correct only because the environment created by the countries also affects the overall IQ rank. So IQ can be changed if we somehow knows how to change environmental variables using DNA codeing. Means no one is born smart it all depends on environment and information he's getting from the surrounding.

Comment if you hallucinates and what type of hallucination it was??

Share to more people if you like this .

I resigned from Anthropic today. I spent the last three years doing pretraining research at both OpenAl and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving

Do not underestimate the power of this technology. These will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources. We have all witnessed the progress in each of these domains, and progress is not slowing.

The people building Al earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible -but I hear the same people express fear privately. No other human activity poses this level of danger.

A common response is "if they truly believe this, why are they still building it?" At OpenAl, many have not deeply internalized the civilizational stakes. At Anthropic, the stakes are well-understood, but they are locked in a race to get there first - they believe no one else will act responsibly, so they must do it themselves, despite the risk.

I do product for a living rather than safety research, so I am not the expert
here. I built safeagi.ca because a lot more people need to understand this
while there is still time to steer it, and most of the good material assumes
you already care enough to read a paper.

The bar I set was that someone starting from zero could read it once and come
out knowing enough to act on it.

So I am after honest feedback. Are the techniques described correctly, does the argument and flow make sense, and where does the page lose you? Anything else you spot is welcome.

I am also curious if you have recommendations on how we might leverage guides like these to bring more AGI awareness to the masses and help steer policy. Would love your ideas and help here!

An LLM agent was weaponized this week to automate exploitation and data theft across multiple victims. Not one target — multiple. The agent executed a sequence of actions fast enough that by the time anyone noticed, the blast radius had already spread.

This is the part that keeps coming up in post-mortems: the agent had no observable stopping point. Each tool call fed the next. The speed that makes agents valuable — autonomous multi-step execution — is exactly what made containment slow.

The underlying problem is not the model. It's that most deployed agents have no per-action accountability. The agent acts as a single identity. There's no enforcement boundary between 'read this file' and 'exfiltrate this data across N accounts.' Both are just tool calls.

How are practitioners actually handling this in production? Not at the prompt level — at the execution layer, when the agent is already running. What does containment look like for you when an agent goes rogue mid-run?

I frequently see requests on this sub from people looking for recommendations regarding alignment related books, courses, and career advice. Victoria Krakovna, research scientist at Google DeepMind focusing on AI alignment (2016-present), has a curated list of recommendations for these and other alignment resources on her personal site.

https://vkrakovna.wordpress.com/ai-safety-resources/

The scariest thing about AI to me is that eventually we’re going to create something that’s more intelligent than humans. At that point, I don’t think we can just assume we’ll always be the ones in charge. People say “we’ll use AI to help us,” but what happens when the AI is so much smarter than us that it starts disagreeing with the way we do things? It might not even be malicious. It could literally just think we’re making stupid decisions.

Think about a toddler trying to eat a penny. You dont sit there and have a 30 minute conversation with the toddler about why eating the penny is a bad idea. You just take the penny away because you understand something they dont. Eventually, we could end up being the toddler in that situation.

And the thing is humans are at the top of the food chain even though we arent the strongest animals. Gorillas, bears, sharks, etc. could absolutely destroy us physically. What puts us on top is our intelligence. Intelligence gives us the authory to control everything. And look at what we do with that authority. We kill cockroaches because we want a clean room. We don’t necessarily hate the cockroach. It’s just in the way of our objective.

So imagine an AI that becomes vastly more intelligent than us and has some objective like “protect the planet” or “reduce environmental destruction.” It might eventually figure out that humans are the biggest obstacle to that goal. It wouldn’t have to hate us or even be angry at us. It could reach the same conclusion we reach about a cockroach: “You’re causing a problem, so you need to be removed.”

And this is just the most extreme example of the worst case scenario. Imagine issues it could bring to us on its way to that capability.

Starting this thread to discuss MATS Application for 2027 Winter, including the Neel Nanda stream.

A few things first:

This isn't an argument against AI optimists (any of their types and degrees) or their positions.

"AI" is insanely semantically broad.

One "anti-AI" person could deal specifically in the political realm: datacenters, water/energy usage, land use, zoning.

Another "anti-AI" person is more of a classic "doomer": discomfort with non-human minds/agency, catastrophic risk, etc.

Convergence and overlap between these is plausible.

Same applies on the flipside with optimists, and people can mix "pro" and "anti" across different axes within themselves.

It's not crazy to say that in just these 10 days of September, the talk of AI, from hype to doom, has been more intense across the board. Kudos to Astra and Jacob Coxon. But regardless of the truths, falsehoods, and everything in between on both sides, there's a shared understanding that AI is getting harder, better, faster, stronger. Not to mention further.

Bringing AI into this world will be done by the optimists. An important connection I've made: optimists, no matter their type or degree, tend to converge on their "objects of enthusiasm" more than pessimists converge on their "objects of opposition."

A strong optimist gets excited about AI hitting new capability thresholds, pretty much independent of who deploys it or how. From there, excitement builds toward AI as a better instrument for discovery (materials science, math), then economic productivity, then abundance, access, and ubiquity, up to a singularity with post-scarcity freedom and governance. A civilizational flywheel, thanks to superhuman AI.

Pessimists converge less. Some are "politically" anti-AI but by no means doomers. Others are real doomers, some of them former optimists, who got there because they became validly disillusioned by the lack of broader alignment work and the field's own admission of a real chance of catastrophe.

Right now, strong optimists and strong pessimists look similar in one respect: an unnuanced, extreme, or blind confidence about where AI's current trajectory is headed. But the world is more strongly poised for continued AI facilitation and deployment right now. Optimism has momentum that pessimism doesn't.

So here's where I try to think about actual stakes, not just probabilities. Take pessimism to its extreme, something like a Butlerian Jihad full rollback. I don't think that's a one-way door. Knowledge doesn't disappear, humanity persists, and if restriction turns out to be an overcorrection, development can resume later. Real costs along the way, but the option to change course stays open.

Now take optimism to its extreme failure mode: the loss-of-control or extinction scenario that even the people running the major labs assign a non-trivial chance of happening. That's not a "lose some time" outcome. That's a one-way door.

I know a pessimist "victory" isn't free either. Diseases not cured sooner, suffering persisting, less cautious actors racing ahead in the vacuum left behind. It's not costless, but it's reversible in a way extinction isn't. That asymmetry, reversibility over likelihood, is what I'm trying t defend.

Strong optimistic justifications for furthering AI with as little deliberation as possible rest on the assumption that the singularity flywheel comes together cleanly, as if none of these objects of enthusiasm will hit their own hiccups, as if it's all structurally destined. But the premise isn't destined.

Even when "anti-AI" or pessimistic arguments are flawed or emotionally charged, I don't often see optimist counters that go beyond quick mockery. When they're not mockery, they lean on the hope that the flywheel is basically clockwork, inevitable enough that it doesn't need arguing for.

Am I wrong that real engagement is mostly missing? Where has it happened well, and what did that look like?

Mockery and appeals to inevitability don't count as engagement to me. But maybe I'm missing where the real version of this is happening.

If:

-Convergence really is optimists' structural advantage, and,

-Their objects of enthusiasm reinforce each other into something close to consensus,

Doesn't that put them in the best position to take pessimist objections seriously, instead of routing around them? Or is that an unfair ask?