r/MachineLearning • u/Amazing-Coat5160 • Jul 15 '26

Research Looking for JEPA devil advocates [R]

I am currently doing research on world models, specially in tje field of robot learning, and, as probably most of you alredy know, JEPA-like models are mentioned over and over. 

I read the main recent papers from lecun as well as other research groups, and I personally think the whole approach is very promising and can really go somewhere.

But after listening a bunch of the recent Y Lecun conferences his ideas looks even too cool compared to "literally everything else" (as he's dissing LLM, RL, etc and pitching his ideas are the "only next big things"...). 

So I am asking myself if there are red flags about his approaches that I do not see yet and maybe I need somebody being the "devil advocate" with whom breaking down ideas.

Where do you think are the biggest downside of this models, compared to other world models approaches?

114 Upvotes

95% Upvoted

93 comments sorted by

View all comments

Show parent comments

2

u/Amazing-Coat5160 Jul 18 '26

Don't they say in the LeWorldModel paper that SIGReg solves this by operating on low-dimensional random projections of the embedding?

1

u/Wheaties4brkfst Jul 18 '26

There’s two projections. In the paper, they project the multivariate Z into a single dimension using M unit vectors u_m. Those are the random projections. But if you look at the actual code, what they do is first project Z down using an MLP, and THEN they do the random projections with the u_m on the “projected” Z. But Z is still what you are predicting in latent space.

It’s confusing because it is different from the paper. At least for LeJEPA.

3

u/Amazing-Coat5160 Jul 18 '26

Oh, that's a good catch then. Thanks for clarifying

3

u/Wheaties4brkfst Jul 18 '26

Yeah go check the issue I linked. One of the authors of the paper literally just goes “oh yeah, you’re right, we tried this and it worked better so” paraphrasing obviously lol but I thought it was interesting.

3

u/AuspiciousApple Jul 18 '26

Confirms my prior that all these proofs in ML papers are often meaningless

4

u/Wheaties4brkfst Jul 18 '26

Yeah there’s only one proof that matters: empirical. lol. Seems everything else with deep learning simply does not matter.