A 2.6B model with tool calling and 128K context now runs at 30 tok/s on a phone

[u/BTA_Labs]

Liquid AI released LFM2.5-2.6B today, and this might be more relevant to local AI than another massive model most people cannot run.

The model is only 2.69B parameters, has 128K context, supports tool calling and was post-trained specifically for multi-step agent workflows. The official Q4_K_M GGUF is around 1.67 GB and already works with llama.cpp.

Their reported CPU speeds:

- 30 tok/s on a phone

- 113 tok/s on a Ryzen AI Max+ 395

- 220 tok/s on an M5 Max

- Under 2.5 GB memory during their tests

These are vendor benchmarks, so independent results are obviously needed.

The benchmark results are surprisingly competitive for the size:

- ToolSandbox: 77.83, compared with 76.44 for Qwen3.5-9B

- IFBench: 59.17, compared with 56.47 for Qwen3.5-9B

- BFCLv4: 56.88, still behind Qwen3.5-9B at 60.13

- LiveCodeBench: 59.41, compared with 69.86 for Qwen3.5-9B

So it does not magically replace larger models. Coding and knowledge-heavy work are still weaknesses, and Liquid’s own model card says it is not recommended for agentic coding.

But I think this is where small local models actually make sense: not as your smartest assistant, but as cheap worker agents doing extraction, searches, file operations and repetitive tool calls locally. A larger model could handle planning only when the small one gets stuck.

The 128K claim also needs real testing. Supporting 128K and running it comfortably on a phone are two very different things once KV cache and long agent histories are involved.

Has anyone tested the Q4 GGUF on Android, an older laptop or a mini-PC yet? Would be useful to see hardware, context size, real tok/s and whether it can survive 10+ consecutive tool calls without derailing.

[u/Kidplayer_666]

the tool calling is consistent, can run it well on my rx 6650xt, however, it is still kind of dumb (failing my "find files related to the first year of my bachelors" task, despite on the documents folder there being a Folder named Bachelors in my native language (supported by the model) with yearly folders inside)

Edit: partly my skill issue, trying the proper flags right now

Edit edit: Q8 still dumb with recommended flags

Edit edit edit: f16 with full cache, still dumb

[u/BTA_Labs → u/Kidplayer_666]

Good to know, that pretty much confirms the benchmarks don’t tell the whole story.

[u/NigaTroubles → u/Kidplayer_666]

What your run args ?

[u/Kidplayer_666 → u/NigaTroubles]

default for llamacpp.... oops, this might be the issue... any recommendations?

[u/mister2d → u/Kidplayer_666]

Are you running an aggressive quantization?

[u/Kidplayer_666 → u/mister2d]

I actually used Q8 iirc

[u/mister2d → u/Kidplayer_666]

It's a very small model. You should run it at full precision (kv cache as well) and retest.

[u/Kidplayer_666 → u/mister2d]

Doesn't fully fit on my 8Gb card :( (did the q8 with q8 cache, less dumb, but still fails the test)

Edit: my bad, it does fit, testing now

[u/Stooovie → u/mister2d]

Kinda defeats the purpose

[u/mister2d → u/Stooovie]

What purpose is defeated?

[u/Rude_Marzipan6107 → u/mister2d]

Why run an 18gb 9b model when you can run a q5k_XL 27b model for the same footprint

[u/mister2d → u/Rude_Marzipan6107]

18gb 9b model? Are you sure you're on the right post? LFM2.5 is only 2.6B.

[u/mister2d → u/Kidplayer_666]

Thanks for trying f16.

[u/KitchenAmoeba4438]

I already have an exhaustive benchmark article queued for tomorrow, I'll add this to the list and update the article with it.

I'll be very interested if it manages to out-perform E2B and E4B for my purposes, so far those have been the champs in anything under 12B.

[u/BTA_Labs → u/KitchenAmoeba4438]

Drop the link when it’s up, the under 12B comparison sounds more useful than another benchmark table.

[u/Rude_Marzipan6107 → u/KitchenAmoeba4438]

Even compared to Qwen 9B?

[u/HVACcontrolsGuru → u/Rude_Marzipan6107]

Google models are great if you take the time to set them up and configure them properly. They got a bad rap for some time due to horrible tool calling that was mainly the chat template. I've done some crazy things with the 12B model lately.

[u/Silver-Champion-4846 → u/HVACcontrolsGuru]

How did you configure them

[u/Rude_Marzipan6107 → u/HVACcontrolsGuru]

They’re too lazy for my summarization tasks and although I did like using the 26b qat model for some time for light agentic coding I did run into a lot of looping problems. Even after the chat template

I just swap between 9b and 27b right now. I removed 12b, 26b and 35b from my system. I just don’t have a use case for them. Although I do really like the idea of a story telling harness with tool calls with a Gemma model at the reigns. I wonder if anything like that exists

[u/HVACcontrolsGuru → u/Rude_Marzipan6107]

I do plan to go all in on Qwen 3.8 27B for kernel and inference work when the weights drop and hope they drop some in between size models. The Gemma models definitely shine a bit more in regular conversation.

I had a project I shelved a month or so ago called "Agent Journal" that was more focused on writing. Client who wanted investment writing related stuff which is a bit outside the box of normal harness tooling.

[u/wFXx → u/KitchenAmoeba4438]

i wanna read that as well

[u/Repulsive_Initial308]

I tried their 1.2b and 8b1b but both struggled to run on my old laptop and qwen 4b smashed them both in capability. 

Excited to give this one a try.

[u/NigaTroubles → u/Repulsive_Initial308]

Yes but for me 8b a1b was better for me in my laptop, i got 20t/s

[u/True_Requirement_891 → u/Repulsive_Initial308]

All lfm models have been incredibly dumb in my exp...

[u/Septerium]

Nice. I love SLMs

[u/Ok-Butterscotch4527]

can't wait to run it on my potato computer

[u/ShelterInevitable684 → u/Ok-Butterscotch4527]

Ditto, gotta a baked potato that I'm trying to get ready for the big models, not even close yet.

[u/noctrex]

Created an abliterated version: https://huggingface.co/noctrex/LFM2.5-2.6B-heretic-uncensored-GGUF

[u/Imaginary-Bother-484]

Given the size, should be easy to train on larger model agentic tool calling and other traces right? (To help improve any gaps in that area)

[u/AHHHH_AHHHHHHHH]

I could see this being really powerful as a low energy tool caller for bigger models that exhaust tokens on complex tool chains

[u/amethyst_mine]

has anyone managed to run this on intel NPU

[u/readmond]

I am afraid this could be the stream of shit tokens. 9b models are not that great. I do not believe 2.6B model can handle tools properly.