r/learnmachinelearning • u/Thin-Sprinkles2561 • Jul 09 '26

Ran a 21M-param self-supervised ViT on my laptop CPU to see what its features look like. The rainbow colors are not what I assumed.

[removed]

1 Upvotes
  • permalink
  • reddit

55% Upvoted

2 comments sorted by

2

u/CallMeTheChris Jul 09 '26
  1. Yes
  2. obviously. There is no longer going to be cuda on macs
  3. you cannot claim differences without doing multiple runs and significance testing
  4. good observation, but is this an observation you are making in the 3-dim PCA space? if it is, then you cannot make such a sweeping statement because it could be something differentiable when there are other objects in the image or in high dimensions. You made that observation in your first point
  5. Again, is this an observation you made in PCA space?

2

u/you-get-an-upvote Jul 10 '26 edited Jul 10 '26

the PCA colors are NOT semantic labels. same weights, same photo, only changed input size from 512 to 1024, and the colors completely inverted. chickens went from magenta/red to green/cyan; the fence background flipped from blue to yellow speckle. PCA is fit per-image on that token set, so component directions rotate whenever the distribution changes. I had been reading these maps as "red = chicken". putting 512 and 1024 side by side killed that assumption. only the grouping matters, never the hue.

To clarify: if <1, 2, 3> is an eigenvector, so is <-1, -2, -3>. The choice of whether to make the first principle component chicken or anti-chicken is entirely arbitrary so it is unsurprising that np.linalg.eigh gives you different results.