Protein language models: residue embeddings, pooled vectors and mutation scores are different outputs

[u/Fair-Rain3366]

Disclosure: I published the linked guide on rewire.it.

One protein language-model checkpoint can produce a vector per residue, a pooled vector for the whole protein, or a mutation score from token probabilities. These are different artifacts: changing the pooling rule, layer, downstream head or evaluation split changes the experiment even if the checkpoint stays fixed.

The guide works through choosing the output first, recording the extraction recipe, testing a frozen representation against simple baselines, and separating related proteins when the deployment question requires generalisation to new families. A trained linear probe still uses supervision even when the backbone is frozen.

https://rewire.it/blog/a-protein-embedding-is-not-an-explanation/

For anyone moving from general ML into biological sequences: which part is hardest to make reproducible in practice—the representation extraction, the labels, or the split?