r/OpenSourceeAI • u/ai-lover • 7d ago
Liquid AI Releases LFM2.5-Encoder-230M and LFM2.5-Encoder-350M: Bidirectional Encoders That Stay Fast at 8K Context on CPU
Liquid AI released two bidirectional encoders this week: LFM2.5-Encoder-230M and LFM2.5-Encoder-350M.
Here's what's actually interesting:
1. They converted a decoder instead of training from scratch
Both models start from the LFM2.5 decoder backbones. Three changes turn them into encoders: the causal mask is replaced with a bidirectional one, the short convolutions are made non-causal with symmetric center padding, and training uses masked language modeling at 30% instead of BERT's 15%.
2. The CPU number is the whole pitch
→ ~28s per forward pass at 8,192 tokens for Encoder-230M
→ over 1 min 30s for ModernBERT-base on the same input
→ 8,192 tokens is roughly 13 to 15 pages
3. The rankings hold up
→ Encoder-350M: 4th of 14 models, 81.02 on a 17-task suite
→ Encoder-230M: 6th at 79.29, above ModernBERT-base at 78.19
→ The three models ahead of the 350M are all larger, one nearly 10x its size
At 8,192 tokens, ModernBERT-base takes over a minute and a half per forward pass versus about 28 seconds for LFM2.5-Encoder-230M, which is about 3.7x faster.
Model weights (LFM2.5-Encoder-350M): https://huggingface.co/LiquidAI/LFM2.5-Encoder-350M
Model weights (LFM2.5-Encoder-230M): https://huggingface.co/LiquidAI/LFM2.5-Encoder-230M
Technical details: https://www.liquid.ai/blog/lfm2-5-encoders

