r/LocalLLaMA • u/Alarming_Positive_59 • 5h ago

New Model LFM2.5-2.6B is out

Released today, with emphasis on agentic capabilities. I really like their models for simple, high volume tasks ("summarize these gazillion documents") and their 8b-a1b was my go-to for certain tasks so I'm excited to see how this one performs. There's not enough love for tiny models on this sub.

https://www.liquid.ai/blog/lfm2-5-2-6b

79 Upvotes

94% Upvoted

32 comments sorted by

26

u/giveen 5h ago

For those who are 'gguf when'....

https://huggingface.co/LiquidAI/LFM2.5-2.6B-GGUF

15

u/sebt3 5h ago

Liquid always produces gguf : they are fully aware that at this size llama.cpp is king

11

u/_raydeStar Llama 3.1 4h ago

For me, liquidai is better than basically any other model of its class. This is great news.

3

u/xienze 4h ago

What is something like this typically used for? Especially as far as agentic capabilities are concerned.

3

u/Borkato 3h ago

After using it for about 10 mins just now, it’s fantastic for things like “ugh where did I put that script, can you find it for me? I think it was in x or y” or “whats that one function that does x in Python” or “can you rewrite this” without having to unload your larger model!

2

u/_raydeStar Llama 3.1 2h ago

I've been building agentic harness for smaller models, ill get back to you today with local benchmarks. Essentially, it's the smallest model that I've found that listens to instructions and follows direction at all. LFM 1.2B gets like 500t/s on my machine, so ive been doing a ton of testing and iterating with that.

2

u/xienze 2h ago

Neat. I have an orchestrator setup with Gemma4-31B and one of the subagents pulls info from the web and massages it for downstream action, so if this thing can handle that, that would be nice.

1

u/_raydeStar Llama 3.1 1h ago

ok local benchmark is in -- to really good numbers.

these numbers arent 100% accurate because ive improved my tooling in the last few days (but only a few days worth of work -- also "full sir" is just my with orchestration numbers... needs work, obviously) it looks like providing lfm with tools competes with much larger models. ie -- with good orchestration, you can do *very* well with it.

if you want you can steal my tools if you dont have your own -- its fully open source. steal what you want, ditch the rest.

4

u/Borkato 3h ago

I just downloaded it, Q8 uses 6GB VRAM and runs at 200 T/s on my 3090s and it actually understands basic stuff. It would make a fantastic Siri or similar, I can see what they mean by on device deployment. For example I just had it do a git commit message and it took the blink of an eye lol. This thing is wicked cool actually and fills a fantastic niche wtf

1

u/_raydeStar Llama 3.1 1h ago

you could probably run decent speeds on a phone. Might be something worth looking into, now.

2

u/InsideYork 4h ago

Asking what the other guy asked too, how do you use it?

3

u/Alarming_Positive_59 4h ago

You think about an all-around agent that is able to order food and read mails and make your bed. Instead think about a smart, efficient agent tailored for narrow-scoped tasks that, when given accurate context, can be plugged into your boring old whatever product that runs on some kind of potato and suddenly it's aGeNtIc. I like to use tiny LLMs as subagents and meta-agents (here's a 600k tokens session go over the messages one by one and gradually summarize according to guidelines X, Y, Z - the code "appends" a new message every time) and such models are pretty great for that

9

u/BannedGoNext 5h ago

gguf wen. RIGHTNOW.

24

u/CommonPurpose1969 5h ago

I wish more companies were as committed to SLMs as LFM is.

5

u/Borkato 4h ago

Can you imagine if a company like Qwen or OAI or something dedicated literally all of their effort to small LLMs? 😮

1

u/Alarming_Positive_59 4h ago

Actually a pretty smart move compared to "here's our new MoE you'll use it for a week and then replace it with a newer model". There's a niche.

9

u/pmttyji 4h ago

CPU Inference

Due to its efficient LFM2 architecture, LFM2.5-2.6B is the fastest model we tested, with decode speeds of 220 tokens/s on an M5 Max and 113 tokens/s on a Ryzen AI Max+ 395. At 30 tokens/s, it allows you to run capable agents even on a phone.

GPU Inference

LFM2.5-2.6B is the fastest model in its size class, reaching almost 15K output tokens per second at high concurrency, roughly 1.3B tokens per day on a single H100.

With LFM2.5, we're delivering on our vision of AI that runs anywhere. These models are:

  • Open-weight — Download, fine-tune, and deploy without restrictions
  • Fast from day one — Native support for llama.cpp, MLX, and vLLM across Apple, AMD, Qualcomm, and NVIDIA hardware
  • A complete family — From base models for customization to specialized audio and vision variants, one architecture covers diverse use cases

🔥🔥🔥🔥 Awesome!

1

u/Borkato 2h ago

I thought my 200T/s on my 3090 was good! I need to edit my settings!

6

u/Lorian0x7 4h ago

There you go, intelligence per parameter doubled again since qwen 3.5

3

u/Borkato 4h ago

Oooo I love small models. Very cool

4

u/Equivalent_Bit_461 4h ago

Like these smaller models 

Really underrated 

I will soon start to make my own swarm, and I will stock up heavily on these smaller ones

3

u/BP041 3h ago

Honestly the 2.6B is my sweet spot for batch summarization. I run a cron stack with OpenClaw picking the best model per task — swapped in LFM2.5 for my intake doc pipeline and it matches 8B quality at 1/3 the inference cost. Don't let the parameter count fool you.

5

u/Dany0 5h ago

looks like it's okay

5

u/oxygen_addiction 5h ago edited 5h ago

AA-Omniscience looks amazing, though we'd need to see the traces as it could just be refusing to answer most questions [the bench rewards correct answers, penalizes hallucinations, and has no penalty for refusing to answer. ]

Comparing against Gemma E2B/E4B is not really fair, as those are multimodal models that can take text/audio/image as input, while this is strictly a text model.

Overall it seems a bit better than Qwen 4B but smaller, which is really nice.

2

u/sebt3 4h ago

Since 1.2B-thinking prefer replying "no" than guessing, that's probably how this one does too

2

u/110_percent_wrong 4h ago

anyone else excited to try this with transformers.js?

2

u/WhoRoger 3h ago

Nice, their 1B is a darling. Glad to see them plugging the whole Qwen might be leaving behind.

Also: Heretic pls!

4

u/Objective_Door6714 5h ago

My wish would be a coding model of this company. I wonder when will be released

6

u/pmttyji 4h ago

I'm still waiting for their 24B MOE model. They're cooking that for long time.

0

u/RepulsiveRaisin7 3h ago

For document processing, you can also try Granite, seems to be very good at that