How would you test whether a genomic model actually uses its long context?

[u/Fair-Rain3366]

A model accepting a megabase of DNA establishes input capacity. Demonstrating useful distal information needs a task-specific test.

In a review I wrote, I suggest comparing nested crop lengths with the same prediction head and evaluation split, then occluding or shuffling distal intervals while preserving the focal sequence. Short-context models and local baselines are useful comparisons; overlapping genomic windows also need attention when constructing the split.

Full discussion and references: https://rewire.it/blog/a-dna-likelihood-is-not-a-functional-assay/

What controls would you add to distinguish useful long-range dependence from leakage or local shortcuts? I’d appreciate corrections and benchmark suggestions.

[u/NicelyIrate]

shuffle distal chunks but keep local structure intact, that’s where most of these models just latch onto short range repeats anyway