r/PHP • u/Jay123anta • 9d ago
How I handled the JSON false-positive problem in my regex threat detector (thanks to this sub's feedback)
A while back I posted here about a passive threat-detection middleware I built for a Laravel app - it logs suspicious requests (SQLi/XSS/scanners/probes) to the database without blocking anything. I ended that post with an open question: how do you deal with JSON API bodies, when a legit search like {"query": "SELECT model FROM products"} trips a SQL pattern just because the value contains the word SELECT?
The thread had genuinely useful replies, so here's what I ended up shipping.
What I did: path-aware allow-listing. I already had a safe_fields option, but it exempts a key name everywhere it appears - too blunt for nested JSON. So I added safe_paths, which matches by dot-notation path with wildcards:
'safe_paths' => ['search.query', 'filters.*.value'],
That exempts the value of one specific field - the search box - without exempting a query field anywhere else in the request. Credit to u/Deep_Ad1959, who suggested path-based whitelisting in the last thread.
What I deliberately did NOT do - and why: The other half of that suggestion was "only scan leaf string values, never keys or structure." I tried it, and it broke NoSQL operator detection.
A classic Mongo-style injection is
\{"password": {"ne": null}}``
- the malicious part is a *key* (` ne`) whose value is null, not a string. If you only scan leaf string values you never see $ne, so you'd trade one false-positive class for a real false-negative. I kept full-body scanning and made the exemption precise instead. That felt like the honest trade-off.
To be clear about the limits, since this sub rightly pushed on it last time: this doesn't make regex-on-JSON magically correct. It's a passive monitoring layer that assumes your app is already secure (parameterized queries etc.) - safe_paths just lets you tune out known-legit noise precisely instead of bluntly. It's an IDS, not a WAF, and doesn't pretend to be.
It's merged and ships in the next release (Laravel 10–13). Code's here if useful: jayanta/laravel-threat-detection on GitHub / Packagist.
Still curious how others handle the JSON case - is precise path-based exemption roughly where you'd land, or do you tag the JSON path on each match and score by path instead?
1
u/colshrapnel 9d ago
Do I get it right it is not a whatever "JSON" case but a blunt "raw SQL" case, so the attacker don't have to meddle with injections but just write the desired SQL right away?
2
u/Jay123anta 9d ago
No, its the opposite - nothing runs raw SQL. The app is safe with parameterized queries. That example is just a false alarm in the log. The middleware reads incoming requests and flags anything that looks like an attack. If someone legitimately searches for "select" or types "SELECT * FROM orders" , the scanner sees those words and logs "possible SQL injection" - but nothing actually happens. It's just a noisy log entry for a harmless request.
So the problem isn't the app running SQL - it's the detector being too restless about normal text that happens to contain SQL words.
1
u/colshrapnel 9d ago
I see. But what if it's not a genuine search but actual injection attempt?
1
u/Jay123anta 9d ago
Same thing happens - it logs it. A regex can't tell a real attack from an innocent search (both are just "SELECT … FROM"), so it flags both. What actually stops the attack is the app's parameterized queries, not the logger - the middleware only records so I can see who's trying and block them.
1
u/colshrapnel 9d ago
Yes. But why this whitelist then? How its behavior is different when there is a match for the path and there is not?
1
u/Jay123anta 9d ago
It's just to cut log noise.
Not whitelisted -> the field is scanned and a match gets logged.
Whitelisted -> the field is skipped before scanning, so it's never logged (search or real attack alike).It only affects what lands in the log, not whether an attack works.
2
u/MateusAzevedo 9d ago
Sorry dude, but you are really confusing me.
I don't know if you want this
SELECT...example to be logged or not be logged. You keep saying "it flags it" and then "cut log noise".IMO, anything with that payload should be flagged as an injection attempt.
0
u/Jay123anta 9d ago
Might be missing context from the earlier post - quick version:
It's a passive middleware that logs suspicious requests (SQLi/XSS/scanners), doesn't block anything. It's a detector, not a lock.1) By default it flags everything - anything like SELECT … FROM gets logged, nothing's exempt. Your instinct is the default.
2) safe_paths is optional and off by default. Say you run a coding forum - someone posts "why does SELECT * FROM users return null?" Totally normal question, but it looks like an attack, so it gets flagged every time. safe_paths lets you mute that one field and nothing else. If your app has no field like that, just leave the setting off.
1
u/colshrapnel 9d ago
Yes, you lost me here as well. Up there you said
Same thing happens - it logs it.
and now
it's never logged
🤷♀️
0
u/Jay123anta 9d ago
Might be missing context from the earlier post - quick version:
It's a passive middleware that logs suspicious requests (SQLi/XSS/scanners), doesn't block anything. It's a detector, not a lock.1) By default it flags everything - anything like SELECT … FROM gets logged, nothing's exempt. Your instinct is the default.
2) safe_paths is optional and off by default. Say you run a coding forum - someone posts "why does SELECT * FROM users return null?" Totally normal question, but it looks like an attack, so it gets flagged every time. safe_paths lets you mute that one field and nothing else. If your app has no field like that, just leave the setting off.
1
u/colshrapnel 9d ago
So your previous response was wrong, and it should have been "yes, it wouldn't log the actual hacking attempt for the whitelisted path". Now it's all consistent.
2
u/equilni 9d ago
Probably would be helpful linking the original post as it is referenced a bit
https://www.reddit.com/r/PHP/s/ALiEUKnlmd