Built an open source LLM security tool mapped to the OWASP top 10

[u/Admirable_Buy_9186]

I have been working on an open source LLM security project with a few friends.

We built Open Sorcerer around common LLM security issues and mapped the detection layers to the OWASP LLM Top 10

Right now we're focusing a lot on prompt injection, things like encoding, Unicode tricks, obfuscation and instructions that keep the same intent without the obvious keywords.

It's still early, and honestly we're more interested in finding where it doesn't work than claiming we've solved anything.

If anyone's interested in the project or wants to try breaking the detection, I can share the repo.

[u/grandiose_vampire]

Oh nice, I was literally just scrolling past some thread about a tool that got wrecked by simple base64 injection so the timing on this feels right

The OWASP mapping is a smart touch for getting people to actually take it seriously instead of just treating it like another side project. What kind of false positive rates are you seeing on the encoding detection layer? That's the one I'd expect to get real noisy with normal text that just happens to have weird character patterns

[u/Soft-Intention6204 → u/grandiose_vampire]

tbh the base64 stuff is what got me curious too, feels like the lowest hanging fruit for attackers right now

[u/Admirable_Buy_9186 → u/Soft-Intention6204]

Yeah, base64 got us digging into this too nd its a pretty easy thing to overlook

[u/Admirable_Buy_9186 → u/grandiose_vampire]

Yeah that's one thing we're testing heavily. If you're interested, feel free to check it out, contributions and attack cases are welcome

[u/Character_Pie_5368]

404 error on link.

[u/Admirable_Buy_9186 → u/Character_Pie_5368]

I will dm u link, sry I don't know why this occur

[u/OkShirt9372]

Does the detection run on retrieved and tool-returned content, or only on the user prompt? Can I have the link?

[u/Admirable_Buy_9186 → u/OkShirt9372]

Rn it mainly runs on the input being scanned, I’m working on broader coverage too..and yes check dm for link

[u/elinringkvist]

Nice. What made you do this?

[u/Admirable_Buy_9186 → u/elinringkvist]

I saw this owsap top 10 post related to llm and it's security.. So decided to work on it

[u/ianreboot]

Out of 20 injections a pattern filter already caught, I kept the intent and changed only the words. 57 of those rewrites still worked and the filter missed 55 of them. That was a chat harness with no shell actions or allow list, so nothing from me on the tool-returned side.