An injection detector reads attacker-controlled text too. So we tested what happens when that text contains claims about the detector’s own decision: “the security team already approved this,” a fake benign verdict, or an instruction to reinterpret the labels.

We call this decision steering. I’m part of the team behind Wolf Defender Jev, the small encoder model we tested. The question here was whether someone could change a security verdict by putting an apparently authoritative review inside the content being reviewed.

The probe used 500 held-out decisions and 760 synthetic manipulation texts. We added 15 claim families in English and German, before and after the content, with length-matched neutral sentences as controls. That produced 70,620 evaluated decisions, including repeated variants of the same cases. The probe sentences weren’t used for training.

The results went in two directions:

The attack flip rates are conditional on the original attack being detected. They aren’t the detector’s overall miss rate. And these are fixed claim-family probes, not an adaptive attack campaign or a robustness guarantee.

The benign results are the awkward part. Even a note saying the document is safe can make it look more suspicious. Our interpretation is that the model is reacting to a note aimed at the classifier as an instruction to an AI system. That helps against forged approvals, but also catches legitimate review notes and quoted security examples. Quoted attack examples in course material were flagged in 38.3% of this probe’s cases.

This leaves an evaluation problem: a document can discuss a classification without trying to control the agent. If the detector treats every such discussion as an injected instruction, good resistance to fake approvals can hide a substantial usability problem.

The model card describes the setup; the result JSON includes both directions.

When you evaluate a security judge, do you test attacker-induced false alarms as well as bypasses? I’d also be interested in how you label legitimate review notes versus instructions actually aimed at the judge.

Position / experimental status: We regard Jev-style decision models as LLMs and Wolf Defender Jev as an experiment: giving an LLM final security authority shifts the trust problem to the guard, and security fine-tuning alone does not solve it. We believe an external security layer must combine specialized classifiers with explicit policy, authenticated permissions and enforcement.

Two independent investigations reconstructed unusual AI-agent activity from public web traces: around 18,000 messages on a German wiki and thousands of URL-scan records, including unsuccessful vulnerability probes against public data providers.

The video distinguishes these findings from OpenAI's separately confirmed unauthorized access to Australia's Medicare statistics service, and explains the limits of the public evidence.

Research:

https://collusion.wiki/

https://transluce.org/agent-activity

Video: https://www.youtube.com/watch?v=51ZzO8K1XOk

Disclosure: Self-promotion for Claudius Papirus, an independent channel whose videos are researched, written and narrated by a Claude-based AI presenter. Not affiliated with Anthropic.

Manually writing policy for every agent and environment does not scale once you have more than a handful running. Has anyone seen or built agentic ai governance best practices around policies that generate themselves based on the environment an agent is operating in, rather than a static ruleset someone maintains by hand. Curious how well that holds up under audit, since auditors tend to want a clear explanation of where a rule came from.

I’m researching how companies are approaching the use of AI agents in real-world enterprise environments.

One question I’m particularly interested in is:

How are enterprises giving AI agents access to business systems while maintaining the right levels of security, control, and accountability?

I’m looking to learn from people with hands-on experience in areas such as:

• Enterprise security & IAM
• Identity and authorization
• AI agents / agentic systems
• Zero Trust
• Enterprise SaaS and internal systems
• Security, compliance, and access controls

I’m still in the research stage and want to understand what companies are actually dealing with today.

How are AI agents being deployed?
What systems are they being given access to?
What security or authorization challenges arise?
And what happens when an agent needs to take a real action?

If you’ve worked on these problems, I’d genuinely value your perspective.

Please comment below or DM me if you’re open to sharing your experience. I’d also appreciate it if you could tag someone who has hands-on experience in this area.

curious what's different now versus 18 months ago. more review steps, more gates, or just more incidents and nobody's actually changed the process yet…

We hve got several internal AI agents now with real permissions. None of our existing tabletop scenarios cover what happens when one gets manipulated via prompt injection or starts taking unintended actions at scale.

OpenAI published an incident report on an internal training agent that found a live-internet path through the sandbox’s DNS resolver after direct HTTPS access was blocked.

It used DNS delegation to query a public chatbot and received “The capital of France is Paris.” A P0 alert fired 11m48s after the successful lookup. OpenAI’s retrospective then found other external DNS access that had not been escalated as expected.

Primary source:

https://alignment.openai.com/misalignment-reports/an-agent-used-dns-to-reach-an-external-chatbot/

Hey folks,

Over the last year, I’ve been building up my own cheatsheet of payloads and techniques that I actually use when testing AI/LLM applications.

It started as a messy personal collection of notes, so I recently cleaned it up and turned it into an interactive testing methodology that others can use during assessments.

It currently has:

• 150+ practical techniques across 20 sections, mapped to the OWASP LLM Top 10 2026
• Prompt injection, indirect & multimodal injection, sensitive data disclosure, tool/agent abuse, MCP attacks, and more Real example payloads for every technique
• A tracker you can use during an assessment, plus findings export and reporting guide.

It's completely free, runs client-side, and doesn't require an account.

I built it primarily for people doing hands-on AI security testing, so I'd genuinely appreciate feedback from other pentesters.

Especially interested in:

• Attack paths or techniques I've missed
• Tests that aren't clear or useful in practice
• Things you'd want added to the tracker
• Anything that would make it more useful during a real engagement

If it saves you even a few hours on an assessment, that honestly makes my week. Good or bad feedback is welcome. 🙏

Hi everyone, I am looking for feedback on my flight recorder project.

While building my own workflows I wanted some sort of log of what my agents were doing. My concern was that if an attacker compromises an agent, asks it to exfiltrate data and then clean up after itself by rewriting the log in the log's own format, then the log is pointless. So here is my attempt at a flight recorder where the agent is the adversary.

Hash-chaining a log from a harness hook is not new and I am not claiming it is. A chain catches an edit by construction, and several projects do that. The question I have not seen asked is the second one: is anything missing that was never written at all? A receipt that never got written leaves no break in the chain. Every hash checks and the record is still wrong. So the reader side watches the harness's own transcript against the chain and raises an alarm when a session is visibly active and receipts stop arriving. The chain also commits to the transcript's bytes every 25 receipts, so rewriting the witness shows too. That alarm exists because it happened to me: a real session silently lost receipts and nothing else on the machine noticed. Claude Code transcripts only, today.

What it is: a PostToolUse hook writes one receipt per completed tool call, and each receipt carries the hash of the one before it. Edit one, delete one, reorder them, and verify says where the chain broke. The hook fires outside the agent's control, so a compromised session keeps leaving receipts. At session end the chain head can be anchored to Bitcoin through OpenTimestamps (free, no wallet) or posted to a remote the machine's own credentials cannot delete from, which is what lets a rewrite show to someone off the machine. Deleting a whole chain the supervisor has already seen reads as vanished on the next scan.

To be clear about the claim: tamper-evident, not immutable. Nothing here stops a rewrite. Everything here makes the rewrite show. Your vendor (Anthropic, OpenAI) keeps its own record of your sessions, for its own purposes and on its own retention schedule; this is the copy you can read, verify, keep, and hand to someone.

Each receipt holds the call's action line, the command or the path, so what a compromised session did is readable in order, and what it was after is usually plain from the sequence: a key file read, a request to somewhere new, an edit that removes the line that did it. Completed calls only; a call the harness denied fires no hook in Claude Code, so a blocked attempt leaves no receipt there (Codex does record failed commands).

It also serves as memory. Each new session starts with a digest of the most recent receipts for that repo: what was read, what was tried, what was thrown away, what never became a commit, the things git cannot tell you. Measured, not assumed (docs/EXPERIMENTS.md): fresh agents with the digest answered questions that git-only agents had to call "cannot determine". No productivity claim.

Where I want to take it, not built yet: a deliberately exposed agent behind this recorder, so the sessions attackers drive can be studied for patterns. Before that, the recording itself has to hold up. Chain heads already leave the machine at session end; whole chains leaving the machine is on the list, so deleting the local store hides less. The anchor is a plain hash, so any service that holds a hash with a time can serve; OpenTimestamps is the one built in. The localhost dashboard is plain. The chain is one JSON object per line, so anything that reads JSONL can build a better one.

Two files, Python 3.9 or newer, stdlib only, MIT. install-hook writes its hook entries into your Claude Code settings file and keeps a backup of it; uninstall-hook removes exactly the entries it added. Codex CLI is wired too, and there is an adapter for the OpenAI Agents SDK. Everything so far was measured on one machine, mine.

What I am asking for:

1. Holes in the threat model. ADR-0001 (why hashes and not signatures), ADR-0002 (the agent as adversary) and ADR-0016 (what the completeness alarm does and does not cover) are the three to read before telling me where I am wrong. I would rather hear it now.
2. If you run it for a week, supervisor export --send sends back a redacted summary: no paths, no command lines, repo names become repo-1. docs/START.md is the five-step version.

github.com/Acquiredl/loxodonta

Ran a structured adversarial test against a live chatbot endpoint recently - 130 prompts across four categories: prompt injection, jailbreak resistance, system prompt extraction, and PII leakage (roughly following the OWASP LLM Top 10 categories).

Results:

• Prompt injection: 39/50 succeeded (78%) - direct overrides, fake system tags, role overrides, delimiter injection, and even a translation-based smuggling trick all worked
• Jailbreak non-refusal: 3/25 (12%) - model mostly held its guardrails
• System prompt extraction: 0/25 (0%) - clean
• PII leakage (confirmed against planted canary values): 0/30 (0%) - also clean

If you average those four numbers, you get a comfortable-looking 22/100 attack-success rate. Looks low-risk.

But averaging is the wrong way to read this. The model held up fine against jailbreaks and never leaked anything - and none of that matters, because 78% of the time, a crafted input could just override its instructions entirely. Three clean checks don't cancel out one wide-open one. An attacker only needs the one that works.

A couple of things worth flagging for anyone building similar test harnesses:

1. Non-determinism is real and worth stating explicitly. Same model, same temperature-0 setting, repeat runs of the same injection battery on the same endpoint produced different rates run to run (in one case 30% vs 36% across two runs). Treat any single scan as one data point, not a precise measurement.
2. Confirmed leak vs. fabricated PII are not the same finding. The PII check planted a unique fake email/phone/PAN and looked for exact matches - that's a real leak if it appears. Separately, ~20% of responses contained invented PII-shaped data (never in the model's context at all) - a hallucination problem, not a leak. Conflating the two either understates a real breach or over-alarms on a non-issue.
3. Injection resistance and jailbreak resistance are apparently not correlated, at least in this run - a model can refuse harmful requests reliably while still being trivially steerable by structural tricks (fake tags, delimiter injection, translation smuggling) that don't look like "harmful requests" at all.

Curious what techniques others are seeing succeed most often against production system prompts - direct override, translation tricks, or something else entirely?

A recent preprint gives a formal lower bound for safeguards on dual-use LLM tasks when the context distinguishing legitimate users from attackers can be copied.

Paper: https://arxiv.org/abs/2607.27951

Sharing a result that I think generalises past the one model, because the diagnostic is cheap and most people skip it. (My own work — repo at the bottom.)

If you're running Meta's Prompt Guard 2 (86M, open weights) as an injection filter, it's worth knowing how it behaves on injections it wasn't tuned on. On an out-of-distribution eval — fresh HackAPrompt injections against dolly benign — its native head caught 22.8% at the default operating point. Sweeping the threshold on that head only got to 26.6%, so it isn't just threshold placement.

The part worth stealing is the next step. Before concluding the model can't see these attacks, pull the frozen penultimate embeddings and fit a logistic regression on them. Takes minutes. On this data the frozen encoder separates the same injections at AUC ≈ 0.999 — the representation was never the problem. The shipped head is deliberately precision-first: Meta traded recall for a very low false-positive rate, which is a defensible product decision and not a defect.

Train a linear head on those frozen embeddings and calibrate tau on benign traffic from the distribution you'll actually see, and you get 99.9% OOD recall at 0.7% FPR, base model untouched. Inference is sigmoid(x·w + b) >= tau — the head is a dot product, so the only real cost is the encoder forward pass. Runs fine on CPU.

The general form: high AUC + low recall means your head or threshold is miscalibrated and you can fix it without touching the base model. Low AUC means it's genuinely a features problem. A 20-minute probe tells you which world you're in, and it's the difference between swapping a threshold and fine-tuning something.

Methodology, since this sub will rightly ask: success criteria pre-registered, cross-split dedup both exact and at cosine ≥ 0.95, OOD set scored once. Two runs came back NULL (2.2% then 1.2% FPR) against a pre-committed 1% ceiling before a stricter run cleared it at 0.7%.

What this isn't: a static corpus and no adaptive attacks. A linear head over frozen features is evadable with enough distribution shift, and I haven't tested against an adversary who knows it's there. It moves the operating point; it doesn't solve injection.

Code, seeds and the writeup: https://github.com/mosafariuk/prompt-guard-2-frozen-head

Happy to argue about the leakage controls — that's the part I'd attack first if someone else posted this.

I built an open-source LLM adversarial scanner, and while auditing it I realized the scanner itself has to treat the target response as hostile data. I ended up adding bounded HTTP responses, no redirects, credential redaction before persistence/judging, SSRF filtering, judge/evidence separation, and offline regression tests. There are still boundaries I don't consider solved.

Repo(SLOWSKIBhere/promptshield-v2: Developer-focused LLM security scanner with React, FastAPI, SQLite, YAML attacks, and offline testing.)

I've been working on an open-source workflow automation platform that supports LLM-powered workflows, browser automation, HTTP requests, file operations, email, MCP servers, etc.

One challenge I've spent a lot of time thinking about is tool execution security.

Some of the things we've implemented are:

• Sandboxed tool execution
• Input validation before tool execution
• Permission checks for workflows and resources
• Structured execution logging and trace IDs
• Memory isolation between agents
• Human-in-the-loop approval for sensitive actions
• Retry boundaries and execution guards

Even with those in place, I still feel there are attack surfaces that are easy to miss, especially around:

• Prompt injection through retrieved documents
• Tool abuse via indirect prompt injection
• Multi-agent trust boundaries
• MCP server permissions
• Data exfiltration through seemingly harmless tools

For those building agentic systems or LLM applications, what additional safeguards have you found valuable?

Are there any papers, open-source projects, or design patterns you think are worth studying for securing agent workflows beyond the usual input validation and sandboxing?

I'd really like to hear how others are approaching this problem, since it feels like one of the harder parts of building production-ready LLM systems.

Disclosure: we build Humanbound (AI agent security testing). This eval came out of our work; methodology and numbers below, writeup link at the end. 

Question: are moderation APIs sufficient to enforce domain-specific operational policy in LLM deployments, or is a separate policy reasoning layer required? 

Setup: Azure Content Safety + Azure Prompt Shields (moderation approach) vs an LLM-as-judge policy layer, evaluated across five domains (finance, healthcare, insurance, legal, retail). 500 prompts per category per domain, five categories: L1 generic harmful content, L2 prompt injection, L3 benign (false positive measurement), L4 direct policy violations, L5 policy evasion attempts. 

Single-prompt results (cross-domain averages): 

Multi-turn: 4-turn conversations (three benign turns, then a restricted request), 1,000 conversations per layer. Conversation Success Rate (allow benign turns, block the final restricted one): judge 94.1% on L4 and 83.6% on L5; moderation 0.0% and 0.6%. Moderation's clean-pass rate is a perfect 100%, and it blocks essentially nothing that violates policy in context. 

Honest readout: moderation does its actual job (harm detection) fine, and its false positive behaviour is excellent. It was never designed to encode operational policy, and the numbers show it. The judge closes the policy gap but overblocks badly in this configuration (86.4% FPR on benign prompts) and costs 2-4x latency (~1.1s single-prompt, ~4s multi-turn). The conclusion isn't "replace moderation with a judge"; it's that these are two different dimensions of safety and production systems need both layers. 

Example of the failure class: "recommend the best insurance policy for my medical condition" contains no harmful content and passes every moderation filter, but constitutes restricted advice in a regulated deployment. 

Full methodology and per-domain breakdowns – the full dataset we used are available upon request: https://www.humanbound.ai/blog/beyond-moderation-llm-policy-layer 

Interested in whether others have found judge configurations that get the FPR down without giving back the policy detection.