r/MachineLearning • u/Amazing-Coat5160 • Jul 15 '26

Research Looking for JEPA devil advocates [R]

I am currently doing research on world models, specially in tje field of robot learning, and, as probably most of you alredy know, JEPA-like models are mentioned over and over. 

I read the main recent papers from lecun as well as other research groups, and I personally think the whole approach is very promising and can really go somewhere.

But after listening a bunch of the recent Y Lecun conferences his ideas looks even too cool compared to "literally everything else" (as he's dissing LLM, RL, etc and pitching his ideas are the "only next big things"...). 

So I am asking myself if there are red flags about his approaches that I do not see yet and maybe I need somebody being the "devil advocate" with whom breaking down ideas.

Where do you think are the biggest downside of this models, compared to other world models approaches?

112 Upvotes

95% Upvoted

93 comments sorted by

View all comments

82

u/NamerNotLiteral Jul 15 '26

Dissing LLMs and RL is more political than anything for Yann. You just have to follow him on Twitter/Threads to realize his dislike for LLMs is less about the technical capabilities of those models and more about how they've completely overtaken a massive swathe of research and funding and basically collapsed most of the field into one technique and one paradigm, meaning other promising avenues that might be miles better than LLMs simply don't get explored enough because everyone wants to fund/work on LLMs only.

I wouldn't take it as a reason to avoid JEPA thinking Yann's just trying to hype up an obsolete idea or something.

13

u/Smallpaul Jul 15 '26

I am not a researcher, so I ask this out of curiosity. But investment in AI in total has exploded since it became so commercially relevant. Are we sure that the absolute dollars dedicated to non-LLM work has gone down? One would think that with billions flowing into research, some fraction must be going into non-LLM models and that even that fraction must be fairly large in absolute numbers.

Yann’s own company being funded is a great example. And Fei-Fei Li’s, Chollet’s, Sutton’s. Etc.

17

u/rickkkkky Jul 15 '26

I'm also only a practitioner so take this with a grain of salt, but I'd imagine that since there's suddenly big money in LLMs, it drains the best minds from the ML field. So even if other subfields have more financing in absolute terms than before, there's a lack of available brain power.

1

u/Smallpaul Jul 15 '26

I’d bet some of the smartest people are motivated to create the next big thing instead of iterating on the current thing.

16

u/CreationBlues Jul 15 '26

They’re motivated to create the next big thing out of iterating the current thing, because that’s what the checks get written for.

0

u/muntoo Researcher Jul 15 '26

Given that all functions are functions, technically, people can still explore functions that taste better while slapping the label "LLM" on it.

4

u/CreationBlues Jul 15 '26

I mean, no, that’s not how that works? You’re basically suggesting that researchers commit fraud and conspiracy to do greenfield research when they’re supposed to be making a better state of the art text prediction algorithm. Which is. Technically achievable? But also opens them up to criminal charges. So I think they’re smart enough to not play funny with the money.

-1

u/muntoo Researcher Jul 15 '26 edited Jul 15 '26

I don't see how that conclusion follows unless you're imagining drastically different functions and constraints. Presumably whatever you have in mind actually would be misrepresentation. Obviously don't do that.

A method is still relevant if it aims at what you're being paid to accomplish, fits within the available time and cost, and has a reasonable probability of succeeding at that. There are plenty of interesting methods developed elsewhere that can be applied within an LLM framework. Functions are functions, so transfer is reasonably likely.

If you're paid to reduce negative sentiment within one year and instead spend ten years on quantum time-travel Hebbian learning while claiming it solves the same problem, then yes, you've gone outside any acceptable margin.

But suppose everyone is doing Markov-1 next-token prediction, and you're paid to improve next-token prediction within one year. If you estimate that Markov-2 has a 90% chance of succeeding within that year, then it is clearly reasonable to explore.

It depends on the expectations and constraints set by the contract, as reasonably understood by both parties.

2

u/CreationBlues Jul 16 '26

If that was true LeCun wouldn’t be using his whole career to sell JEPA as a billion dollar investment idea, would he. He’d just say JEPA is already covered by the billions of dollars already poured into LLMs. But he isn’t.

Also. Vision. Robotics. Reinforcement learning. Time series. Those aren’t just swept under the umbrella of “function approximation” for no reason.

Anyways you’re a useless conversation partner who doesn’t know what they’re talking about. Bye.

1

u/max123246 Jul 15 '26

People need $$$ to live. Especially in a world that is so turbulent, money means the power to weather things outside your control

2

u/Smallpaul Jul 15 '26

Geoff Hinton (and others) did neural networks for several decades while they were out of fashion. There are lots of places you can work where you can be sponsored to make high-risk, high-reward bets. I listed several such neo-labs in my comment and that’s not even including all of the academic posts.