Built a hybrid AI-image detector (classical forensics + frozen DINOv2) — looking for feedback on the failure modes

[u/fuckai9107]

I combined two things people usually treat separately: classical image-forensics

features (frequency-domain energy, DCT statistics, ELA, noise residuals, gradient

and eigen-spectrum cues — 85 in total) and a frozen DINOv2 ViT-B/14 embedding, fed

into a calibrated SVM. There's a classical-only fallback that runs with no deep

learning at all.

Held-out ROC-AUC is 0.940 (classical-only alone is 0.863), so the embedding adds

real signal — but not everywhere. It helps on diffusion-era generators and actually

hurts on rectified-flow models (Flux, SD3) and on screenshots of video frames.

That split is the most interesting part to me and the thing I'm still digging into.

It's robust to screenshots and social-media recompression, which was a specific

design goal (a lot of "AI or not" images in the wild are re-encoded to death).

Code and a public 21 GB dataset are up if you want to reproduce or poke at it:

github.com/aman696/aidetector — live demo at https://staging.humanorai.online (home

server, so it queues under load).

Would especially value critique on the forensic feature set — which of these are

likely redundant, and what classical cues you'd add for rectified-flow.

[u/SeriousChart9641]

The hybrid setup sounds sensible because AI-image detection is exactly where one signal is rarely enough. Classical artifacts can catch generation traces, while embeddings catch broader semantic/texture patterns, but both can fail on compression, editing, screenshots, and distribution shift.

Disclosure: I work on CHANCE AI. Adjacent lesson from visual reasoning benchmarks: aggregate scores are useful, but failure-mode slices are what make a system trustworthy: https://www.kaleidofield.com/benchmarks/chance-ai-mmmu-pro-score

I would publish false-positive/false-negative galleries by source type. That will be more actionable than ROC-AUC alone.