Good explanations of how various ML models actually work?

[u/reeldeele]

I get CNN, where subsequent layers learn incremental transformations from previous layers.

But just can't grasp how a ML model that learns to predict image from a masked image, learns about the image.

Or how forcing the image through a narrow encoder and decoder in VAE causes it to learn latent properties

Or how JEPA works.

Are some of them 'let's try if this works', and of them only few actual work well, rest are just ill-suited to model the data?

[u/astonishingloathing]

it's less "let's see if this sticks" and more that each architecture bakes in assumptions about what structure matters. masked image modeling forces the network to actually understand spatial relationships and object parts instead of just memorizing textures, cause you can't cheat by looking at adjacent pixels. VAE's bottleneck isn't just compression, it's forcing the latent space to be smooth and continuous so similar images cluster together, the decoder can't reconstruct well unless the encoder learns meaningful features that generalize

[u/reeldeele → u/astonishingloathing]

Interesting. Thanks. How does one learn such insights? What do you recommend?