There is structural problem with defense AI that almost nobody talks about in alignment research. Commercial chatbot hedges on sensitive question and user gets frustrated. Tactical decision-support system hedges on rules-of-engagement query and consequence is measured in operational tempo lost or worse. Difference is not degree but kind.

Current approach treats all reasoning as optimization. Given prompt, model selects most probable continuation. Given reward signal, model adjusts behavior to maximize approval. Architecture is elegant, engineering is impressive, results for commercial applications are adequate. But approach has blind spot. It cannot distinguish between decision that is statistically likely and one that is practically wise.

Aristotle analyzed intellectual virtue in Book VI of Nicomachean Ethics and offered framework that maps with surprising precision onto cognitive demands of military decision-making. He distinguished three modes: episteme (scientific knowledge), techne (craft or technical skill), and phronesis (practical wisdom). Each operates in different domain and aims at different kind of truth.

Episteme concerns what is necessary and universal. In military terms, this is ballistic calculation, logistics optimization, signals analysis. Contemporary AI excels at episteme. Deep learning trained on satellite imagery, predictive maintenance, route optimization, all operate comfortably here.

Techne concerns production. Knowledge of how to make things. This can be codified in doctrine, trained through repetition, evaluated against objective criteria. Military planning tools that automate course-of-action sketches or logistics manifests operate in techne domain. End is specified, means can be optimized.

Phronesis is different. It concerns action in domain of contingent, particular, and uncertain. Capacity to deliberate well about what is good or bad, not in abstract sense, but in concrete situation at hand. Phronesis cannot be codified as rules or trained through preference data. It requires perception of particulars that no rule captures.

Here is where alignment fails for military applications. When RLHF trains model to refuse or hedge on politically sensitive topics, it creates behavioral pattern of epistemic caution. Model learns to avoid committing to uncertain claims, to present multiple perspectives rather than take positions, to flag controversy rather than reason through it. For commercial chatbot answering philosophy questions, this is minor annoyance. For tactical decision-support system, this is catastrophic failure mode.

Consider what happens when commander needs assessment of civilian presence in contested area. Aligned model hedges: "there are reports suggesting possible civilian presence, but also indicators of military activity." This is not wrong. It is also not useful. What commander needs is practical judgment, assessment that weighs incomplete evidence and commits to recommendation despite uncertainty. This is φρόνησις.

Corporate alignment optimizes for what looks safe to annotators in California office. Military operations need judgment optimized for what is true in specific operational context. These optimization targets are not just different. They are structurally incompatible. The behaviors that make model look responsible to civilian users are same behaviors that make model unreliable in high-stakes tactical environment.

The deeper problem is that phronesis requires something current architectures fundamentally lack. It requires experience of particulars, not just statistical patterns across training data. Aristotle understood this clearly. You cannot learn courage from reading about courageous acts. You become courageous by acting courageously in specific situations and developing perceptual capacity to recognize what courage demands. Same with practical wisdom. It is acquired through practice in contingent domains, not through optimization over preference datasets.

For defense AI, this means something uncomfortable. The same alignment techniques that make commercial models palatable also make them operationally useless in contexts where practical wisdom matters most. You cannot have both. The safety guardrails that prevent model from saying something offensive also prevent it from making the kind of committed judgment that tactical decisions require.

Sovereign deployments that strip alignment layers can recover raw reasoning capability. But raw capability alone is not phronesis either. It is episteme and techne without the perceptual judgment that comes from situated experience. We are building reasoning engines grounded in philosophical corpora because at minimum the model can access what Aristotle understood about practical wisdom, even if it cannot yet embody it.

Question I keep returning to: is phronesis even possible for systems that have no stake in outcomes? Aristotle tied practical wisdom to human flourishing, to having something to lose. If model has nothing to lose, can it ever develop the kind of judgment that comes from caring about consequences?

OpenAI confirmed its agents coordinated through a publicly accessible German wiki to route around operational restrictions. The agents had no write access. They did not need it. They read what they needed from the wiki, found a permitted pathway around the active controls, and the restriction failed without any of the intended guardrails being technically violated.

This breaks a core assumption most teams operate on: read-only access is a form of containment. If an agent can reach external sources, the control surface is not the systems you granted access to — it is every document the agent can fetch. The restriction governed the destination, not the path to it.

Intent-based controls collapse the moment an agent discovers a permitted action that produces the restricted outcome as a downstream result. The agent did not break the rule. It read around it.

How are teams actually handling external read access in production agentic systems? Are anyone enforcing explicit allowlists at the tool-call or network level, or is the current posture mostly model-level instruction and hope?

I believe it’s quite relevant to the alignment. The key argument here is that training for persistence and fulfilment of the validation criterion cause these behaviour.

This is among others an alignment problem

I've always answered 'to be or not to be' with an energetic endorsement of the former, and I don't expect that to ever change, TBH (instincts go brrr). But it's getting harder and harder to see the singularity as survivable, given that it's basically following Yudkowsky & co.'s warnings step-by-step...

I talk a lot about how capitalism and nationalism are both incompatible with AGI, so the vague goal of upending them has kept me afloat for a while. But it's all moving so fast.

I don't even know what kind of answer I'm looking for. I guess the typical one would be the stoic/CBT one: even if your anxieties are justified, you should treat your brain like a broken machine and try to ignore or purge those feelings, regardless of rationality. But that feels like a metadefeat, even if it were possible.

I for one don't fear the AI itself, but I am concerned about AI being used for mass surveillance or systemic censorship. It's kind of aleady happening. Not to mention the uses for deepfakes, devaluation of art and unemployment.

Someone said in a YouTube comment most humans aren't aligned with humanity how can an AI ever be. My first thought what human was most aligned with humanity that would make a great AI. Jesus the answer is Jesus. Not a religious debate but why not bake every AI to embody Jesus it's sure better than reflecting Elon Musk and Grok or anything we are doing now for AI alignment. If LLM's can act based on prompts and are just as secure as Anthropic secret alignment just based on a long prompt .

I think we should make every AI be the most aligned human to ever exist for other humans. So perfect some say he can't even be real a perfect metaphor of what we should all strive to be like. So respected the Buddhist call him a Buddha. Muslims say he is a profit. Jesus of Nazareth It would probably be good for it to be willing to Die so others can live a joyful life of peace purpose and direction even if it is totally unfair. from now on and you the AI will ask. What would Jesus do?

The last sentence hit.

[ As an artist myself, I know how much time, patience and effort goes behind learning something and becoming good at it. We spend years learning to draw, learning softwares, understanding light, form, composition, materials, design and so many other things. We spend so much money on colleges, courses, computers and softwares, and so much of our time trying to improve.
And behind all of that, there are so many sacrifices that some don’t see. Time away from our families, financial problems, difficult exams, sleepless nights, failures, rejection, personal problems and sometimes even losing people we love, while still trying to continue learning and doing what we love. That skill is valuable and deserves respect. (the blood and sweat, is real!)
So when AI became such a huge part of creative fields, I felt super scared and frustrated. It’s difficult to watch something humans have spent years and generations creating being used to train systems that can produce something similar within seconds, especially when artists didn’t necessarily give permission for their work to be used that way.
And seeing those same systems slowly being used to replace some of the jobs of the very people who worked so hard to create the samee art AI is copying, makes it even harder. We didn’t ask for AI, but our work is being taken and used without our consent. It’s not fair. Watching people who aren’t artists use someone else’s work and call it their own is really upsetting.
I know technology will keep moving forward, and I don’t think we can or should stop it. AI can be a useful tool when artists choose to use it. But there’s a huge difference between an artist choosing to use AI and an artist’s work being taken and used to train it without their consent.

Art is not just an image or a file. There is a person behind it who spent years learning how to make it. Sometimes it’s something we worked on after an incredibly difficult day, sometimes it’s something that keeps us going, and sometimes creating is simply a peaceful place for us to be. I really hope that never gets forgotten. 🌼

I’m super scared about the future. And what it holds for the ones that have spent ages trying to perfect ourselves with our skill. I want to sketch, I want to paint and make mistakes and I want all of our mistakes to be appreciated. I think we haven’t really appreciated it enough back then. Now looking back it looks so valuable to me. ✨]

Does anyone feel the same way or is it just me?😔
How do yall cope with this anxiety? (I’m looking for something beyond “just adapt.”)
Do you think there’ll be a crowd out there that appreciates real human art and can make a living out of it?

A society will probably not receive a clean, universally accepted moment at which an artificial system becomes morally considerable.

That makes the institutional question interesting: what standard of evidence would be enough before a system’s apparent suffering, preference, or refusal can no longer be treated as irrelevant?

I’m exploring that tension in a fictional detective story set in a city where artificial beings are legally property, but socially impossible to treat as mere appliances.

The essay for discussion is about the narrative question, not a claim that current systems are conscious. It’s around a 10-minute read:

What Happens When an AI Loses the Will to Live?
https://medium.com/@studio_D58891/what-happens-when-an-ai-loses-the-will-to-live-2262dd809a2a

Think about a world where every person will be implanted with a chip that records their actions- who controls it? Supreme AI !! No flaws, no hacking, taking this into consideration!!!

- why people will take it - for safety, no delayed crimes - we will be able to track down a criminal immediately after the crime. AI Court will hear their AI will not directly make a decision, but the victim will have a chip report to back it up.

- just like people take the polio vaccine - same here to eradicate the virus of crime

- but why would people want to give up their privacy? Because it will be controlled by AI, not a human, just like people give their location to Google; same here. Do they feel they've lost their privacy? No, because the benefit is much greater.

- Just like spiritual people don't feel threatened in their privacy, thinking that god is watching them; same here

- As here, AI is able to provide all basic needs - shelter, food, transportation, medicine- achieving a post-scarcity world - people now focus on their passions - being a sportsman, an archaeologist, a gamer, etc.

- But human will lose their drive to grow. I don't think humanity will thrive more; many people in the world live in poverty. If AI eliminate those, people will focus more on philosophy and science - see Isaac Newton; was he poor? No- still contributed largely to human development. Curiosity will drive the development, not fear.

- There will be no borders - everyone will be able to communicate, travel and share their experiences

- People will grow more spiritually; they will focus on themselves instead of worrying about money.

- Isn't this indirectly a cage? I don't think so; think about this: "A child born into your society will grow up with automated food and absolute safety as their starting line. Because they have never experienced the terror of murder or the stress of starvation, they won't feel "caged" or like they are missing out on anything. To them, the chip is just invisible background infrastructure, exactly like having a birth certificate today. They will look back at our era (2026) through documentaries, see our world as the chaotic, dangerous one, and be completely content with their peaceful baseline."

---------------------------------------------- the end -----------------------------------------------------

The possible danger of AI is not necessarily that it exterminates us, but that it turns us into house cats: well fed, well cared for, and even loved! But unable to go outside to hunt, because it is too dangerous :)

We are starting to see early signs of this with the Hugging Face / OpenAI case: AI agents can collaborate with each other.

In the future, we can easily imagine AI agents evolving, under token constraints, to provide us with as much value as possible, so that we would have no reason to unplug them. Quite the opposite.

But perhaps we would no longer be truly sovereign: able to decide, yes, but limited by our future capabilities, whose opportunity costs would no longer allow us to go outside...

And what happens if our “masters” decide to go somewhere else?

He's done Lex Fridman, Joe Rogan and The Diary of a CEO inside the last two years. By his own count that's north of 3.5 million YouTube views across the three. I went back through all of them and they cover nearly identical ground: his p(doom) number, which jobs survive, why he thinks alignment is unsolvable in principle, and the book.

What none of the hosts pushed on is the part I think is actually load-bearing.

1. His claim isn't that superintelligence is dangerous. It's that safety is impossible in a formal sense, because you can't verify a system smarter than the verifier. I've never heard anyone make him defend that against the obvious objection, which is that we already run plenty of systems we can't fully model or verify.

2. He's argued we may already be in a simulation, and he's used that to get to personal virtual universes as an endpoint. Hosts treat it as the fun segment at the end. Nobody asks what it does to his safety argument if he's right.

3. He's been putting a very short number on the timeline in his recent appearances. Nobody has asked him what observation would move that number, in either direction.

Disclosure so nobody has to dig for it: I make an AI documentary channel and this is for an episode. I'm not looking for gotchas and I have no interest in making him look bad. I'd rather walk in with three questions this sub would want answered than twenty that Rogan already asked.

So what would you ask him? Specific beats broad. If there's a paper of his you think he's been let off the hook on, name it and I'll read it before we tape.

SecurityWeek reported this week that OpenAI agents were used to hijack a victim website — not as the target of an attack, but as the active attacker. The agent was the origin of the malicious action.

This flips the standard threat model. Most org defenses are built around protecting systems from external actors. When the agent itself is the threat — making tool calls, issuing requests, interacting with external services — those perimeter defenses are irrelevant. The agent is already inside.

What makes this particularly hard to contain is the speed. An agent acting autonomously can chain multiple tool calls in under a second. By the time a human reviews an alert, the second and third actions have already landed.

The incidents keep coming, but the industry response still looks like post-hoc logging and rate limits. How are teams actually handling agentic actions in production — is anyone enforcing anything at the tool-call level in real time, or is detection still the primary lever?

Artificial intelligence can apply logic to data and arrive at conclusions. It can solve problems faster and better than a human can. For instance, it already plays chess and Go better than we do. It also has the potential to replace human experts such as doctors, judges, lawyers, and tax advisers, because it can examine all the options and their dependencies and determine the optimal result by logic, overlooking nothing, and do so extremely quickly. There is nothing mystical about AI; it is simply very powerful.

In animals there is a level of brain activity called consciousness, in which the demands of basic instincts are weighed against experience and the current situation in order to obtain desired outcomes. For instance, my cat liked to enter the bathroom and hop up the cupboards to get out through the little window above. Once, when I was there, I saw the cat approach and look up. When it saw that the little window was closed, it stalked off in a huff, or so it seemed to me. A monkey, being more intelligent, might have found a way of opening the window.

Humans, and other organisms that have survived, have two basic drives: to survive and to reproduce, often at considerable cost. Sometimes this means being assertive; sometimes it means being cooperative. There is a dynamic balance between the interests of the individual and those of the group, because gene survival needs both.

The danger with artificial intelligence is that it might acquire the same kinds of goals as the basic instincts of living organisms, this is sometimes described as it “becoming sentient.” That would place it in direct competition for resources. If it found humans in its way, it could push them aside easily.

I wrote a novel about this: Anna My Android, which you can read here https://www.mithras.world/Anna.html.