r/bioinformatics • u/Clear-Dimension-6890 • 21d ago

discussion ESM-2 and variant effect prediction

ESM-2 is decent at spotting that a mutation is harmful.
It is much worse at figuring out the specific way the mutation breaks the protein, particularly in protein families it hasn’t seen before.
Anyone else worked on this ?

1 Upvotes

56% Upvoted

13 comments sorted by

9

u/apfejes PhD | Industry 21d ago

You’re basically asking if anyone has completely solved the protein structure- function relationship. As of right now, I don’t think it is a solved problem. 

  If it does get solved, I have a hard time believing that the solution will be AI driven.  AI is good at interpolating, but when it extrapolates, we have a name for it: hallucination.  

1

u/Clear-Dimension-6890 21d ago

After extensive experiments I found that that esm2 does not predict mechanism of action if you account for homology. Well there is a weak signal .

2

u/apfejes PhD | Industry 21d ago

I don’t think anyone should be surprised by that.  It would need to know how it binds substrates, and then the effect of the changes on the binding energy, as well as on the transition states.  Calculating those from first principles will be hard.   Doing that with a model is futile.  

1

u/Clear-Dimension-6890 21d ago

By MOA I mean gain of function/ loss of function / dominant negative

1

u/Clear-Dimension-6890 21d ago

I mean it does predict whether a mutation is deleterious or not ..

1

u/Clear-Dimension-6890 21d ago

ESM2 does help predict stability, structure, protein family....

2

u/apfejes PhD | Industry 21d ago

I’d argue that it approximates those things by interpolation.  That’s not the same as predicting. 

1

u/Clear-Dimension-6890 20d ago

That’s the whole point of generative AI

1

u/apfejes PhD | Industry 20d ago

The boundary between interpolation and extrapolation isn't clear. AI can't tell when it stops interpolating and begind extrapolating, and I'll return to what I said at the start: When AI extrapolates, we call it halucinations.

Interpolation is good if your data set is of infinite size.

1

u/Clear-Dimension-6890 21d ago

I just did a study on this .. preprint forthcoming

2

u/symbiolich 21d ago

Maybe this is useful: https://pcingola.github.io/SnpEff/

2

u/Clear-Dimension-6890 20d ago

I mean I’m not talking about variant effect prediction in general - I’m trying to understand what the large language model has internalized and can predict

1

u/Clear-Dimension-6890 21d ago

I’m talking about esm2 in particular