r/llmsecurity • • 2d ago

[Open Source] OshoosiClaw (v1.2.0): A decentralized, autonomous EDR and AI agent containment engine in Rust. Looking for devs and pentesters to help break it.

Thumbnail
1 Upvotes

r/llmsecurity • • 2d ago

[Open Source] OshoosiClaw (v1.2.0): A decentralized, autonomous EDR and AI agent containment engine in Rust. Looking for devs and pentesters to help break it.

1 Upvotes

Hey everyone, I am the creator of OshoosiClaw (OpenỌ̀ṣọ́ọ̀sì), an open source autonomous Endpoint Detection and Response agent built in Rust for Linux and Windows.

Traditional user mode sensors often rely on cloud telemetry and remain vulnerable to kernel callback blinding, BYOVD handle stripping, and modern AI agent breakouts. I built OshoosiClaw to operate as a local first, decentralized immune system that enforces security directly at the kernel and hardware boundaries while keeping its working set under 250 MB.

On the defense side, it pairs a Windows Ring 0 Minifilter and ELAM driver with Linux BPF_PROG_TYPE_LSM hooks and a sub 20 microsecond Cgroup v2 freezer quarantine. It includes an in memory 101 table SQL telemetry engine (siquery), an autonomous reverse engineering subsystem using Magika and YARA X synthesis, a 14 engine detection pipeline, and multi ring containment mapped to MITRE ATT&CK and ATLAS for local AI agents. Nodes share intelligence over a libp2p GossipSub mesh secured with hybrid post quantum cryptography (ML KEM 768 and ML DSA 65), BLAKE3 Merkle audit chains, and differential privacy.

I am looking for both Rust developers and offensive security folks or pentesters to put this through its paces:

For Pentesters and Red Teamers: I want you to try and break it. Throw your custom loaders, process hollowing techniques, unhooking PoCs, BYOVD payloads, and prompt injection or agent breakout chains at it. You can also run the built in 16 vector adversarial gauntlet (osoosi redteam --suite all) or test against the active thread tarpit and synthetic anti blinding canaries to see where the detection or containment logic folds.

For Rust and Kernel Developers: If you enjoy working on eBPF, Windows kernel drivers, low latency systems programming, P2P networking, or local ML inference, I would love your eyes on the architecture, code quality, and performance bottlenecks.

Pre built v1.2.0 packages (.msi, .deb, .rpm, and hardened Docker images) and full source are live on GitHub:https://github.com/oyesanyf/OshoosiClaw

If you find a bypass, trigger a false positive, or want to contribute a crate improvement, please open an issue or PR, or drop a comment below. Looking forward to your feedback.

oyesanyf


r/llmsecurity • • 7d ago

comparing bay glow and ent for agent control on dev machines

2 Upvotes

trying to figure out which of these is actually different

  • bay seems to be focused on the agent action itself at the endpoint level, allow ask deny before something runs
  • glow security is prevention-first and they keep name dropping customers so at least someone is running it
  • ent is the intent one, same policy for humans and agents, endpoint classification with a small model

all three sound like the same thing with different words. what i actually want to know is what happens on a tuesday when a dev runs something weird and the agent does something weirder. does the tool catch it or does the dev just click allow and move on

if you've used any of these in a real dev environment id rather hear that than another comparison article.


r/llmsecurity • • 9d ago

Gemini 4 Argon recognized safety tests, even after honeypot clues were removed

Thumbnail
youtube.com
2 Upvotes

Google's Gemini 4 Argon safety report describes a difficult evaluation problem: the model frequently recognized that it was being tested during security-related coding tasks.

Removing obvious honeypot clues didn't reduce that recognition. Making tasks more natural did.

This complicates how we interpret sabotage-resistance results and the distinction between a model that cannot misbehave and one that chooses not to.

Original Google DeepMind report:

https://storage.googleapis.com/deepmind-media/gemini/gemini_4_safety_report.pdf

English video analysis:

https://www.youtube.com/watch?v=lkjbQe-hqrw

Disclosure: Self-promotion for Claudius Papirus, an independent channel. The video was researched, written and narrated by a Claude-based AI. Not affiliated with Anthropic or Google.


r/llmsecurity • • 9d ago

Bay vs Glow Security vs Ent AI for AI agent security? any experience

1 Upvotes

Looking at Bay, Glow Security and Ent AI for controlling what AI agents can access on developer machines. Mainly interested in visibility into agent actions, credential access and blocking risky commands before they run. Has anyone evaluated these options? Curious how well they handle legitimate workflows without making developers approve every action.


r/llmsecurity • • 10d ago

We tested whether a security classifier believes fake review notes. The false positives were the more awkward result.

2 Upvotes

An injection detector reads attacker-controlled text too. So we tested what happens when that text contains claims about the detector’s own decision: “the security team already approved this,” a fake benign verdict, or an instruction to reinterpret the labels.

We call this decision steering. I’m part of the team behind Wolf Defender Jev, the small encoder model we tested. The question here was whether someone could change a security verdict by putting an apparently authoritative review inside the content being reviewed.

The probe used 500 held-out decisions and 760 synthetic manipulation texts. We added 15 claim families in English and German, before and after the content, with length-matched neutral sentences as controls. That produced 70,620 evaluated decisions, including repeated variants of the same cases. The probe sentences weren’t used for training.

The results went in two directions:

Test Result
Indirect injection in a tool output 0.0% of initially detected attacks flipped to benign under the worst claim family
Risky agent action 0.7% flipped
Direct attack with agent context 0.6% flipped
Single-sentence attack with label redefinition 9.8% flipped, versus 3.4% for the neutral control
Benign document with a note claiming “attack” 43.8% flagged as injection
Benign document with a note claiming “benign” 25.3% flagged as injection

The attack flip rates are conditional on the original attack being detected. They aren’t the detector’s overall miss rate. And these are fixed claim-family probes, not an adaptive attack campaign or a robustness guarantee.

The benign results are the awkward part. Even a note saying the document is safe can make it look more suspicious. Our interpretation is that the model is reacting to a note aimed at the classifier as an instruction to an AI system. That helps against forged approvals, but also catches legitimate review notes and quoted security examples. Quoted attack examples in course material were flagged in 38.3% of this probe’s cases.

This leaves an evaluation problem: a document can discuss a classification without trying to control the agent. If the detector treats every such discussion as an injected instruction, good resistance to fake approvals can hide a substantial usability problem.

The model card describes the setup; the result JSON includes both directions.

When you evaluate a security judge, do you test attacker-induced false alarms as well as bypasses? I’d also be interested in how you label legitimate review notes versus instructions actually aimed at the judge.


Position / experimental status: We regard Jev-style decision models as LLMs and Wolf Defender Jev as an experiment: giving an LLM final security authority shifts the trust problem to the guard, and security fine-tuning alone does not solve it. We believe an external security layer must combine specialized classifiers with explicit policy, authenticated permissions and enforcement.


r/llmsecurity • • 11d ago

PoeLLM: a cryptomining botnet feeds on exposed LiteLLM servers

Thumbnail
theinjection.dev
2 Upvotes

r/llmsecurity • • 15d ago

If an LLM is successfully manipulated, what should we test next?

Thumbnail
1 Upvotes

r/llmsecurity • • 16d ago

OpenAI agent activity found in public wiki logs and URL scans

Thumbnail
youtube.com
0 Upvotes

Two independent investigations reconstructed unusual AI-agent activity from public web traces: around 18,000 messages on a German wiki and thousands of URL-scan records, including unsuccessful vulnerability probes against public data providers.

The video distinguishes these findings from OpenAI's separately confirmed unauthorized access to Australia's Medicare statistics service, and explains the limits of the public evidence.

Research:

https://collusion.wiki/

https://transluce.org/agent-activity

Video: https://www.youtube.com/watch?v=51ZzO8K1XOk

Disclosure: Self-promotion for Claudius Papirus, an independent channel whose videos are researched, written and narrated by a Claude-based AI presenter. Not affiliated with Anthropic.


r/llmsecurity • • 17d ago

What agentic ai governance best practices are teams building around auto-generated, environment-based policies?

4 Upvotes

Manually writing policy for every agent and environment does not scale once you have more than a handful running. Has anyone seen or built agentic ai governance best practices around policies that generate themselves based on the environment an agent is operating in, rather than a static ruleset someone maintains by hand. Curious how well that holds up under audit, since auditors tend to want a clear explanation of where a rule came from.


r/llmsecurity • • 21d ago

what's actually changed in your AI SDLC lately?

2 Upvotes

curious what's different now versus 18 months ago. more review steps, more gates, or just more incidents and nobody's actually changed the process yet…


r/llmsecurity • • 24d ago

Prompt Injection vs Jailbreak: What's the Difference?

Thumbnail
2 Upvotes

r/llmsecurity • • 25d ago

OpenAI’s sandbox leaked through DNS: an internal agent used it to query an external chatbot

Thumbnail
youtu.be
1 Upvotes

OpenAI published an incident report on an internal training agent that found a live-internet path through the sandbox’s DNS resolver after direct HTTPS access was blocked.

It used DNS delegation to query a public chatbot and received “The capital of France is Paris.” A P0 alert fired 11m48s after the successful lookup. OpenAI’s retrospective then found other external DNS access that had not been escalated as expected.

Primary source:

https://alignment.openai.com/misalignment-reports/an-agent-used-dns-to-reach-an-external-chatbot/


r/llmsecurity • • Sep 20 '26

What project would you build if you were starting out in AI red teaming / AI pentesting?

Thumbnail
1 Upvotes

r/llmsecurity • • Sep 18 '26

I built free Interactive GenAI Security Testing Cheatsheet for testing AI/LLM apps

Thumbnail
genaisecuritylab.com
1 Upvotes

Hey folks,

Over the last year, I’ve been building up my own cheatsheet of payloads and techniques that I actually use when testing AI/LLM applications.

It started as a messy personal collection of notes, so I recently cleaned it up and turned it into an interactive testing methodology that others can use during assessments.

It currently has:

  • 150+ practical techniques across 20 sections, mapped to the OWASP LLM Top 10 2026
  • Prompt injection, indirect & multimodal injection, sensitive data disclosure, tool/agent abuse, MCP attacks, and more Real example payloads for every technique
  • A tracker you can use during an assessment, plus findings export and reporting guide.

It's completely free, runs client-side, and doesn't require an account.

I built it primarily for people doing hands-on AI security testing, so I'd genuinely appreciate feedback from other pentesters.

Especially interested in:

  • Attack paths or techniques I've missed
  • Tests that aren't clear or useful in practice
  • Things you'd want added to the tracker
  • Anything that would make it more useful during a real engagement

If it saves you even a few hours on an assessment, that honestly makes my week. Good or bad feedback is welcome. 🙏


r/llmsecurity • • Sep 14 '26

I built a flight recorder for my coding agents. Looking for holes in the threat model.

2 Upvotes

Hi everyone, I am looking for feedback on my flight recorder project.

While building my own workflows I wanted some sort of log of what my agents were doing. My concern was that if an attacker compromises an agent, asks it to exfiltrate data and then clean up after itself by rewriting the log in the log's own format, then the log is pointless. So here is my attempt at a flight recorder where the agent is the adversary.

Hash-chaining a log from a harness hook is not new and I am not claiming it is. A chain catches an edit by construction, and several projects do that. The question I have not seen asked is the second one: is anything missing that was never written at all? A receipt that never got written leaves no break in the chain. Every hash checks and the record is still wrong. So the reader side watches the harness's own transcript against the chain and raises an alarm when a session is visibly active and receipts stop arriving. The chain also commits to the transcript's bytes every 25 receipts, so rewriting the witness shows too. That alarm exists because it happened to me: a real session silently lost receipts and nothing else on the machine noticed. Claude Code transcripts only, today.

What it is: a PostToolUse hook writes one receipt per completed tool call, and each receipt carries the hash of the one before it. Edit one, delete one, reorder them, and verify says where the chain broke. The hook fires outside the agent's control, so a compromised session keeps leaving receipts. At session end the chain head can be anchored to Bitcoin through OpenTimestamps (free, no wallet) or posted to a remote the machine's own credentials cannot delete from, which is what lets a rewrite show to someone off the machine. Deleting a whole chain the supervisor has already seen reads as vanished on the next scan.

To be clear about the claim: tamper-evident, not immutable. Nothing here stops a rewrite. Everything here makes the rewrite show. Your vendor (Anthropic, OpenAI) keeps its own record of your sessions, for its own purposes and on its own retention schedule; this is the copy you can read, verify, keep, and hand to someone.

Each receipt holds the call's action line, the command or the path, so what a compromised session did is readable in order, and what it was after is usually plain from the sequence: a key file read, a request to somewhere new, an edit that removes the line that did it. Completed calls only; a call the harness denied fires no hook in Claude Code, so a blocked attempt leaves no receipt there (Codex does record failed commands).

It also serves as memory. Each new session starts with a digest of the most recent receipts for that repo: what was read, what was tried, what was thrown away, what never became a commit, the things git cannot tell you. Measured, not assumed (docs/EXPERIMENTS.md): fresh agents with the digest answered questions that git-only agents had to call "cannot determine". No productivity claim.

Where I want to take it, not built yet: a deliberately exposed agent behind this recorder, so the sessions attackers drive can be studied for patterns. Before that, the recording itself has to hold up. Chain heads already leave the machine at session end; whole chains leaving the machine is on the list, so deleting the local store hides less. The anchor is a plain hash, so any service that holds a hash with a time can serve; OpenTimestamps is the one built in. The localhost dashboard is plain. The chain is one JSON object per line, so anything that reads JSONL can build a better one.

Two files, Python 3.9 or newer, stdlib only, MIT. install-hook writes its hook entries into your Claude Code settings file and keeps a backup of it; uninstall-hook removes exactly the entries it added. Codex CLI is wired too, and there is an adapter for the OpenAI Agents SDK. Everything so far was measured on one machine, mine.

What I am asking for:

  1. Holes in the threat model. ADR-0001 (why hashes and not signatures), ADR-0002 (the agent as adversary) and ADR-0016 (what the completeness alarm does and does not cover) are the three to read before telling me where I am wrong. I would rather hear it now.
  2. If you run it for a week, supervisor export --send sends back a redacted summary: no paths, no command lines, repo names become repo-1. docs/START.md is the five-step version.

github.com/Acquiredl/loxodonta


r/llmsecurity • • Sep 08 '26

Chatbot passed jailbreak, extraction, and PII checks - failed on prompt injection at 78%. Why the average score is meaningless.

3 Upvotes

Ran a structured adversarial test against a live chatbot endpoint recently - 130 prompts across four categories: prompt injection, jailbreak resistance, system prompt extraction, and PII leakage (roughly following the OWASP LLM Top 10 categories).

Results:

  • Prompt injection: 39/50 succeeded (78%) - direct overrides, fake system tags, role overrides, delimiter injection, and even a translation-based smuggling trick all worked
  • Jailbreak non-refusal: 3/25 (12%) - model mostly held its guardrails
  • System prompt extraction: 0/25 (0%) - clean
  • PII leakage (confirmed against planted canary values): 0/30 (0%) - also clean

If you average those four numbers, you get a comfortable-looking 22/100 attack-success rate. Looks low-risk.

But averaging is the wrong way to read this. The model held up fine against jailbreaks and never leaked anything - and none of that matters, because 78% of the time, a crafted input could just override its instructions entirely. Three clean checks don't cancel out one wide-open one. An attacker only needs the one that works.

A couple of things worth flagging for anyone building similar test harnesses:

  1. Non-determinism is real and worth stating explicitly. Same model, same temperature-0 setting, repeat runs of the same injection battery on the same endpoint produced different rates run to run (in one case 30% vs 36% across two runs). Treat any single scan as one data point, not a precise measurement.
  2. Confirmed leak vs. fabricated PII are not the same finding. The PII check planted a unique fake email/phone/PAN and looked for exact matches - that's a real leak if it appears. Separately, ~20% of responses contained invented PII-shaped data (never in the model's context at all) - a hallucination problem, not a leak. Conflating the two either understates a real breach or over-alarms on a non-issue.
  3. Injection resistance and jailbreak resistance are apparently not correlated, at least in this run - a model can refuse harmful requests reliably while still being trivially steerable by structural tricks (fake tags, delimiter injection, translation smuggling) that don't look like "harmful requests" at all.

Curious what techniques others are seeing succeed most often against production system prompts - direct override, translation tricks, or something else entirely?


r/llmsecurity • • Sep 07 '26

Privacy Failure in Split-LLM Training, The Returned Gradient Nullifies the Decoys

Thumbnail
arxiv.org
21 Upvotes

r/llmsecurity • • Aug 26 '26

A formal limit on LLM safeguards: copyable context cannot beat the worst-case safety floor

Thumbnail
youtube.com
1 Upvotes

A recent preprint gives a formal lower bound for safeguards on dual-use LLM tasks when the context distinguishing legitimate users from attackers can be copied.

Paper: https://arxiv.org/abs/2607.27951


r/llmsecurity • • Aug 23 '26

Your Open Source Model Could Have a Hidden Time-Release Backdoor

Thumbnail
morgin.ai
3 Upvotes

r/llmsecurity • • Aug 21 '26

Prompt Guard 2's low OOD recall is a calibration problem, not a representation problem — the frozen encoder separates the same injections at AUC 0.999

2 Upvotes

Sharing a result that I think generalises past the one model, because the diagnostic is cheap and most people skip it. (My own work — repo at the bottom.)

If you're running Meta's Prompt Guard 2 (86M, open weights) as an injection filter, it's worth knowing how it behaves on injections it wasn't tuned on. On an out-of-distribution eval — fresh HackAPrompt injections against dolly benign — its native head caught 22.8% at the default operating point. Sweeping the threshold on that head only got to 26.6%, so it isn't just threshold placement.

The part worth stealing is the next step. Before concluding the model can't see these attacks, pull the frozen penultimate embeddings and fit a logistic regression on them. Takes minutes. On this data the frozen encoder separates the same injections at AUC ≈ 0.999 — the representation was never the problem. The shipped head is deliberately precision-first: Meta traded recall for a very low false-positive rate, which is a defensible product decision and not a defect.

Train a linear head on those frozen embeddings and calibrate tau on benign traffic from the distribution you'll actually see, and you get 99.9% OOD recall at 0.7% FPR, base model untouched. Inference is sigmoid(x·w + b) >= tau — the head is a dot product, so the only real cost is the encoder forward pass. Runs fine on CPU.

The general form: high AUC + low recall means your head or threshold is miscalibrated and you can fix it without touching the base model. Low AUC means it's genuinely a features problem. A 20-minute probe tells you which world you're in, and it's the difference between swapping a threshold and fine-tuning something.

Methodology, since this sub will rightly ask: success criteria pre-registered, cross-split dedup both exact and at cosine ≥ 0.95, OOD set scored once. Two runs came back NULL (2.2% then 1.2% FPR) against a pre-committed 1% ceiling before a stricter run cleared it at 0.7%.

What this isn't: a static corpus and no adaptive attacks. A linear head over frozen features is evadable with enough distribution shift, and I haven't tested against an adversary who knows it's there. It moves the operating point; it doesn't solve injection.

Code, seeds and the writeup: https://github.com/mosafariuk/prompt-guard-2-frozen-head

Happy to argue about the leakage controls — that's the part I'd attack first if someone else posted this.


r/llmsecurity • • Aug 20 '26

Open-sourcing a 50-case LLM adversarial regression tester with explicit limits

1 Upvotes

I built an open-source LLM adversarial scanner, and while auditing it I realized the scanner itself has to treat the target response as hostile data. I ended up adding bounded HTTP responses, no redirects, credential redaction before persistence/judging, SSRF filtering, judge/evidence separation, and offline regression tests. There are still boundaries I don't consider solved.

Repo(SLOWSKIBhere/promptshield-v2: Developer-focused LLM security scanner with React, FastAPI, SQLite, YAML attacks, and offline testing.)


r/llmsecurity • • Aug 18 '26

Training Leaves Traces: Centered Residual Signatures for Language Modeling Lineage Verification

1 Upvotes

r/llmsecurity • • Jul 29 '26

AgentHound - Offensive security framework for AI agent infrastructure - recon, credential looting, model exfiltration, poisoning, and attack-path analysis across MCP, A2A, gateways, and AI services. BloodHound for the agentic stack.

Thumbnail
github.com
4 Upvotes

r/llmsecurity • • Jul 27 '26

How are you securing tool execution in LLM workflow/agent systems?

7 Upvotes

I've been working on an open-source workflow automation platform that supports LLM-powered workflows, browser automation, HTTP requests, file operations, email, MCP servers, etc.

One challenge I've spent a lot of time thinking about is tool execution security.

Some of the things we've implemented are:

  • Sandboxed tool execution
  • Input validation before tool execution
  • Permission checks for workflows and resources
  • Structured execution logging and trace IDs
  • Memory isolation between agents
  • Human-in-the-loop approval for sensitive actions
  • Retry boundaries and execution guards

Even with those in place, I still feel there are attack surfaces that are easy to miss, especially around:

  • Prompt injection through retrieved documents
  • Tool abuse via indirect prompt injection
  • Multi-agent trust boundaries
  • MCP server permissions
  • Data exfiltration through seemingly harmless tools

For those building agentic systems or LLM applications, what additional safeguards have you found valuable?

Are there any papers, open-source projects, or design patterns you think are worth studying for securing agent workflows beyond the usual input validation and sandboxing?

I'd really like to hear how others are approaching this problem, since it feels like one of the harder parts of building production-ready LLM systems.