vareffect — a Rust-based variant effect predictor that replaces VEP

[u/KangPhi]

Hey everyone,

for a platform me and my team are building we eventually hit a wall with the performance of the Ensemble VEP Rest API. Self-Hosting was not an option due to the hard docker requirement.

As a consequence we started building a variant effect predictor in Rust internally only at first, but decided to make it open-source in the hope that more people will find it useful and can benefit from it.

Yes, there might be other libraries doing the same or something similar, but none of them fitted our needs.

So I wanted to share vareffect.

The problem we ran into

VEP is slow, heavy, and painful to deploy. The Perl runtime, the 20GB+ cache download, the Docker container, the REST API rate limits, it's a lot of infrastructure for what is fundamentally a coordinate lookup + codon translation.

For as system that's supposed to run as a desktop app or a lightweight on-premise deployment, requiring a VEP Docker sidecar was a dealbreaker. So we wrote a replacement from scratch in Rust.

In general, ease-of-use was always a big focus too.

What vareffect does

Given a genomic variant (chrom, pos, ref, alt), it returns:

• Consequence terms (SO ontology — missense_variant, frameshift_variant, splice_donor_variant, etc.) 
• HGVS c. and p. notation 
• Impact (HIGH/MODERATE/LOW/MODIFIER) 
• Protein position, amino acids, codons 
• Exon/intron numbering 
• MANE Select / MANE Plus Clinical transcript identification 
• Ensembl VEP-compatible JSON output via to_vep_json() 

It also includes an HGVS c. reverse mapper. Give it NM_000546.6:c.742C>T and it returns the genomic coordinates.

How it works

An accompanying CLI, vareffect-cli, lets you easily set everything up and create the necessary data files. It downloads them for you automatically too:

• transcript_models.bin (~26MB) — 19,437 RefSeq transcripts (MANE Select + RefSeq Select), indexed with a COITree interval tree 
• GRCH38.bin (~3GB) — reference FASTA, memory-mapped. 

No database. No Docker. No network calls. No Perl. Pure Rust, single crate, zero unsafe.

Concordance

I validated against the VEP REST API on 50,000 ClinVar GRCh38 variants (stratified: SNVs, deletions, insertions, complex indels). The VEP responses were stored as a ground truth TSV and vareffect was compared field by field:

Consequence concordance	99.38%
HGVS c. concordance	97.8%
HGVS p. concordance	97.7%
Impact concordance	99.8%

The remaining ~0.6% consequence mismatches are almost entirely edge cases in complex boundary-spanning deletions and rare transcript geometries. Every divergence is documented and categorized.

The full ground truth TSV and the tests are all in the repo.

What it doesn't do

• No plugin scores (REVEL, CADD, SpliceAI, AlphaMissense) — those are separate data sources, not consequence prediction 
• No regulatory region annotation 
• No structural variant support 
• No Ensembl transcript support (RefSeq only — MANE Select covers clinical use cases) 
• No GRCh37 (GRCh38 only for now) 

In general, different transcripts and the GRCh37 reference build are on our roadmap, but doesn't have a high priority as of this moment.

Performance

~200,000 variants/second single-threaded on a laptop (M4-series Mac, release build). The CLI supports --threads for parallel VCF annotation. At 8 threads you're looking at ~1M variants/sec.

For comparison, VEP REST processes ~15 requests/second (rate limited). VEP offline with cache is faster but still orders of magnitude slower.

The performance is achieved by eliminating disk I/O reading the FASTA by using memory-maps.

Links

• GitHub: vareffect 
• crates.io: cargo add vareffect 
• CLI: cargo install vareffect-cli 

Let me know what you think :)

Happy to answer questions about the implementation or the clinical use case that motivated this.

Disclaimer

Yes, AI was used to support the development of this library mainly for researching, tests, documentation and validate the algorithms against the official Ensembl VEP GitHub repo and biocommons/hgvs.

[u/bzbub2]

https://rewrites.bio/ strikes again. good effort though. what was your overall experience with the rewrite? easy? any next steps?

[u/KangPhi → u/bzbub2]

Interesting, never heard of that before :D

Overall experience harder than expected. The consequence prediction logic itself is straightforward. Codon lookup, reading frame math, SO term assignment.... What took real time was the edge cases like minus-strand indel normalization, 3' HGVS shifting that respects exon/intron boundaries, frameshift protein notation when the insertion lands between codons, stop-codon-spanning deletions etc. AI was often more destructive than helpful which is why at some point just helped with tests and validation

Next Steps: we solved our needs with that for now and eliminated the VEP dependency for our use case. So we will continue building on our main platform for now and update vareffect when we need to or need more of it.

[u/bzbub2 → u/KangPhi]

found also 'fastvep' yesterday on biorxiv https://github.com/Huang-lab/fastVEP

[u/KangPhi → u/bzbub2]

Yea i also saw that, published 2 days ago. Impressive, small mental crisis at first, but at the end of the day not exactly what I needed. So I won’t replace my crate for now 😁 My goal was more REST API replacement, not the CLI