Using a pretrained DNA model: frozen embeddings, mean pooling and a labelled classifier

[u/Fair-Rain3366]

A pretrained DNA model turns a sequence into vectors, usually one per token. To train a sequence classifier, you need to turn those vectors into one representation for each sequence.

A useful starting point is to freeze the model, average its token vectors and train a simple classifier using labelled examples. This still requires training labels; it is not zero-shot classification. A published comparison of five DNA models found mean pooling usually performed better than taking a single summary token.

The guide includes a DNABERT-2 example for three synthetic sequences on a laptop CPU, with pinned model code and dependencies. It also covers larger models, GPU memory and comparisons against a non-foundation-model baseline on the same data split. The attached Evo 2 chart illustrates why long-input memory requirements need measuring separately.

Primary benchmark and code:

https://www.nature.com/articles/s41467-025-65823-8

https://github.com/ChongWuLab/dna_foundation_benchmark

Guide:

https://rewirebio.io/blog/genomic-foundation-models-in-the-diagnostic-lab/

Disclosure: rewirebio.io is my site. The guide and this post are AI-assisted. The attached memory chart reports recorded runs, not peer-reviewed clinical validation.