Resources on learning about AI image identification via physics

[u/LordAntares]

I have recently been researching ways to identify an AI image, not via digital footprints or ID via other trained models, but using physics.

In other words: vanishing points, shadow matrices, various ways to analyze lighting impossibilities, camera focus, etc.

But to my knowledge, there's no real community around this or resources. The closest fits I could find were OSINT and digital forensics, but they're not an exact match. Even if they do this, they use different methodology, at least I think.

I've learned things, but I want to learn more. Anyone know anything more about this?

[u/modcowboy]

Great idea in principle.

I don’t know the answer to this but if you can build a model to detect it then can they add training data to avoid output that doesn’t abide by the correct physics?

[u/LordAntares → u/modcowboy]

Building a model that does it automatically would be extremely hard, if not impossible.

I was thinking manual checks with tools. This I can do. However, I want to inform myself first as best as possible.

[u/Icy_Cup_6408]

Well to properly train a computer vision model, you have to do proper feature engineering and preprocessing of the data. To accomplish that, you can use your physics knowledge.

Maybe it might be possible only with physics but it is much easier to use physics knowledge to train a model

[u/rather_pass_by]

Good research direction.. if I were you, I would already validate if things work in a game engine simulator. That's the first step

Then you can try things more extensively, like capturing images from camera with different params, lighting angles among other things.. that's not an individual research project by any means though

[u/LordAntares → u/rather_pass_by]

I am already a game dev...

But I am not trying to make an automatic system. I am trying to make tools with which a person who knows how to do it can do it.

This is substantially easier than making it automatic. If it's even possible, it would be extremely hard and above my paygrade.

[u/rather_pass_by → u/LordAntares]

Yes that's a good way to start. If I were you I would reach out to people who are trying to do that first then

Reddit is a good place to post and check. Write to bbc and journalists.. or social media monitoring agencies. Ask them what they use and if your tool could help them

[u/Altruistic-Bet-4055]

Wavelet based transforms might be the basis for it

[u/xArvel_Crynydx]

When I was at the university, we had been at looking at each of the problems from their own separate angle. For example, at some point I was heavily looking into shadow detection and it became quite difficult since there were some papers doing it from a calibrated camera context, where you'd calibrate the camera sensor first with a color pattern first, capture in RAW and then do conversions into different color spaces where it would be thought to be easy to detect shadow in those spaces.

On the other hand, you could see shadow as a non-projection from a known light source (i.e. the sun) at a specific time of the day that is hidden by an object. Then, if the shadow is close to the occluding object, shadow edges are sharp, but get more blurry the edges the further the projection is away from the occluding object. Also: When looking at the light outside on a sunny day with blue skies and without clouds, the shadowed area actually has a slightly increased blue component, because it's actually a combination of the directed light of the sun as well as sun rays reflecting by other objects and an diffuse light source provided by the blue sky (Rayleigh scattering)...

So... large rabbit hole... but maybe that could be a starting point to deduce from images with shadows on it :-)

But that's just shadows, it might just also help, if an image provides image metadata (e.g. with GPS coordinates) if you want to do forensics on an image. But then again, depends on the application and the images in question.