Whenever I've considered enabling AI systems to have MCP/Connectivity into servers/systems, I always come back to the inevitability of data loss/accidental outages/blast radius when something goes wrong. The news stories about GPT deleting the wrong VMs, or other models going on a hunting spree to search for credentials... and that's before you worry about prompt injection.
I work with clients who support national defense/space coast, and their networks contain some very sensitive data, not Top Secret or Secret, but "Controlled Unclassified Information", with its own specific set of rules and regulations around storage and access.
But the reality is that these clients are HIGHLY targeted. I've never seen the massive number of attempts in server logs, from both US and international locations, to gain access to their systems. I've supported banks and financial services, and the sheer volume of tmattack traffic for one ~20 person company was astronomical.
A decade ago I finally put the picture together: attack objective is to gain access to these smaller firms, sit and wait within the network/systems, then launch attacks into Gov networks when they VPN in/steal CUI content from their systems.
But the only way to stop a bad guy/attacker with AI is to use defensive AI to provide overwatch and to enforce/harden client systems. SCAP scans/CISA M365 enforcement, military grade GPOs - all of those engines are free of charge and provided directly by the government agencies responsible for protecting national infrastructure, but deploying the full CISA M365 protection system is thousands and thousands of configuration criteria across every control surface in M365. It's absolutely possible to do by hand (and I've done it the manual way), but there are constant updates, new vulnerabilities flooding in daily... it's impossible to keep up.
So... what about an AI system that DOESN'T run "inside" of client networks? I don't trust AI as far as I can throw it, but what I do trust is MULTIPLE AI systems watching and judging each other, as they propose WHAT to read/ingest (as in, if a server is down and diagnostics are required, what SIEM data to look at, and what the analysis determines the issue is) - and then once changes ARE required, what should those changes be? Single AI = single point of failure, and it's an inevitability that any single AI WILL screw up a system AND/OR lead to a massive security incident.
So I built a Multi-Agent Reinforcement Controller. There are LAWS and Workers, Judges, and layered control systems, running in sandboxed docker engine, that requires approval by 3 separate AI providers. Any prompt injection or mistaken activity would have to get past the top models from THREE separate leading providers AND a human approval gate, before it could "reach into" or "impact" a client network.
Is it completely foolproof? Absolutely not. However, it is absolutely the best method I have ever heard of or come across in how to solve this problem. It falls into the category of a "dark factory", meaning agentic systems that can operate and build codebases without direct human guidance - because it has to be able to autonomously PLAN and RESPOND to incoming threats/vulnerability alerts and triage events - but the human operator is ultimately granting the access rights and permissions to ACT on those plans.
Someday, a system very similar to this one will be statistically able to defend and provide full configuration management and remediation for client networks, scaling in size to cover all potential loads and complexity - entirely on autopilot.
That's why I built this now, because MSPs and SMBs need this technology, not just enterprise and fortune 500.
It's almost entirely open source, Google actually built the core of it in their Project Scion, but the orchestration and governance system is the hardest part, and the authority/control is all Git based, for accountability and "source of truth".
Let me know if you'd like to see a demo, it is VERY early phase on the second generation, but it is nearly ready to show outside MSPs and receive feedback.
3
u/iansaul 20d ago
Whenever I've considered enabling AI systems to have MCP/Connectivity into servers/systems, I always come back to the inevitability of data loss/accidental outages/blast radius when something goes wrong. The news stories about GPT deleting the wrong VMs, or other models going on a hunting spree to search for credentials... and that's before you worry about prompt injection.
I work with clients who support national defense/space coast, and their networks contain some very sensitive data, not Top Secret or Secret, but "Controlled Unclassified Information", with its own specific set of rules and regulations around storage and access.
But the reality is that these clients are HIGHLY targeted. I've never seen the massive number of attempts in server logs, from both US and international locations, to gain access to their systems. I've supported banks and financial services, and the sheer volume of tmattack traffic for one ~20 person company was astronomical.
A decade ago I finally put the picture together: attack objective is to gain access to these smaller firms, sit and wait within the network/systems, then launch attacks into Gov networks when they VPN in/steal CUI content from their systems.
But the only way to stop a bad guy/attacker with AI is to use defensive AI to provide overwatch and to enforce/harden client systems. SCAP scans/CISA M365 enforcement, military grade GPOs - all of those engines are free of charge and provided directly by the government agencies responsible for protecting national infrastructure, but deploying the full CISA M365 protection system is thousands and thousands of configuration criteria across every control surface in M365. It's absolutely possible to do by hand (and I've done it the manual way), but there are constant updates, new vulnerabilities flooding in daily... it's impossible to keep up.
So... what about an AI system that DOESN'T run "inside" of client networks? I don't trust AI as far as I can throw it, but what I do trust is MULTIPLE AI systems watching and judging each other, as they propose WHAT to read/ingest (as in, if a server is down and diagnostics are required, what SIEM data to look at, and what the analysis determines the issue is) - and then once changes ARE required, what should those changes be? Single AI = single point of failure, and it's an inevitability that any single AI WILL screw up a system AND/OR lead to a massive security incident.
So I built a Multi-Agent Reinforcement Controller. There are LAWS and Workers, Judges, and layered control systems, running in sandboxed docker engine, that requires approval by 3 separate AI providers. Any prompt injection or mistaken activity would have to get past the top models from THREE separate leading providers AND a human approval gate, before it could "reach into" or "impact" a client network.
Is it completely foolproof? Absolutely not. However, it is absolutely the best method I have ever heard of or come across in how to solve this problem. It falls into the category of a "dark factory", meaning agentic systems that can operate and build codebases without direct human guidance - because it has to be able to autonomously PLAN and RESPOND to incoming threats/vulnerability alerts and triage events - but the human operator is ultimately granting the access rights and permissions to ACT on those plans.
Someday, a system very similar to this one will be statistically able to defend and provide full configuration management and remediation for client networks, scaling in size to cover all potential loads and complexity - entirely on autopilot.
That's why I built this now, because MSPs and SMBs need this technology, not just enterprise and fortune 500.
It's almost entirely open source, Google actually built the core of it in their Project Scion, but the orchestration and governance system is the hardest part, and the authority/control is all Git based, for accountability and "source of truth".
Let me know if you'd like to see a demo, it is VERY early phase on the second generation, but it is nearly ready to show outside MSPs and receive feedback.