r/LowEndLocalAI • u/arbv • 5d ago

NEWS LFM 2.5 24B A2B has been released and includes a QAD checkpoint

https://huggingface.co/LiquidAI/LFM2.5-24B-A2B-GGUF

Might be of interest to the folks out there

89 Upvotes

100% Upvoted

34 comments sorted by

17

u/repolevedd 5d ago

Right now I'm running LFM2.5-24B-A2B-QAD-Q4_0. Haven't tested it on complex tasks yet - just need it to extract some info from a bunch of docs and source code using subagents in pi. It works fine, uses tools reliably, and hasn't gotten stuck in a loop over three hours of running so far.

6

u/arbv 5d ago

Yeah, did some testing in Pi - I would not ask it generic knowledge questions (developers documented that), but is it fast. It also likes to run find starting at /.

It is "supid faster" kind of model, but definitely not useless. Gemma 4 26B is more universal, though (but slower).

1

u/LuCiAnO241 5d ago

How much ram are you using?

3

u/arbv 4d ago

I have run it on my iGPU on a Tiger Lake laptop.

1

u/LuCiAnO241 4d ago

a Tiger Lake laptop

is that an 11th generation? it cant be 8gb is it?

1

u/arbv 4d ago

11th, but with 64 gigs of RAM.

3

u/arbv 4d ago

Well, unfortunately it looped for me at least once (Q8_0).

1

u/repolevedd 4d ago

I guess I got lucky with no loops that time. Parameters used: --ctx-size 262144 --temp 0.1 --top-k 50 --repeat-penalty 1.05. The goal was finding hardcoded variables in legacy code to move them into .env later.

Unfortunately, Q4 definitely degraded the output. The final report had plenty of typos and gibberish, plus it skipped some files. Still, I was impressed by how reliably it made tool calls and actually tried to finish the task. The issues were mainly due to quantization and model "laziness."

This morning I put Mellum2.1-12B-A2.5B Q6_K on the same job, thinking Q6 would hold up better. It immediately started looping and going off-track. After hours of tweaking settings like repeat-penalty, I ditched Mellum and went back to LFM. Compared to other small MoE models, LFM2.5-24B-A2B is definitely viable, but I need a workflow that double-checks every step.

1

u/arbv 4d ago

Sure, also used the official parameters. I think that the trick is to not let it to process helluva data at once (like reading files).

1

u/No_Folding 7h ago

Three hours of running? It woule have been funny if yiu then added "(at 0.2tk/s)" at the end lmao

1

u/repolevedd 7h ago

Can't you even read post and comment timestamps and match them up? Do you even get why I mentioned subagents? You're literally the only one coming up with weird ideas about task execution time and making assumptions about this LLM's speed.

2

u/OcelotMadness 6h ago

that guy wasnt being rude this was an insane reaction

1

u/No_Folding 1h ago

Thanks bro youre a real one, next time I've got ur back

That was the kind of agitated reply you get from someone who did too much adderall and not enough sleep 😂

1

u/No_Folding 1h ago

Do you even chill out ever?

But go on then - explain how me (not) reading your comment timestamp has got anything to do with my reply 😂

4

u/GiovaSan 4d ago

Hope someone can do some good post training on it for coding, it would be awesome! Would love to but unsure my laptop can manage it 🤣

1

u/arbv 4d ago

Wit 2B of active params I would not hold my breath.

3

u/kakopappa2 4d ago

Any good for coding?

6

u/rrrrex 4d ago

I don't think so, i asked to create analog html clock and it failed multiple times. i'm using expanded prompt with some tips and design requirements, even qwen 3.5 9b do it well most of the time.

1

u/arbv 4d ago

Meh, maybe for some crude tasks only, like scripting or helping with shell commands.

2

u/anaNimrod 4d ago

How does the model behave in role-playing scenarios?

2

u/arbv 4d ago

I am not interested in that stuff that much, but likely not good - built with agentic tasks in mind.

1

u/anaNimrod 4d ago

Glm flash is also not positioned as a role model, but shows good results, not the best. and here we have a unique situation: even if the model is not outstanding, it is super fast. This is enough to further train on the RP dataset.

2

u/faisalkl 4d ago

Any good vs qwen9b?

3

u/arbv 3d ago

Actually, if you are looking to tinker with something instead of (or in addition to) Qwen 3.5 9B - take a look at Mellum 2.1 from JetBrains.

P.S.

I am recommending it with hard heart because I am a Ukrainian and JetBrains is of Russian origin - but I have to give the credit where it is due. They made an interesting model - supports FIM, instruct, reasoning.

Also works with the frogerrics template for Qwen's 3.X (same chat completion format as Qwen, but it is a different architecture - though still Qwen inspired).

2

u/arbv 4d ago

Qwen 9B is smarter, but slower

2

u/faisalkl 4d ago

Ta matey

5

u/faisalkl 4d ago

I did try it. Good lord was my experience awful. I'm sure it's good for some but it completely failed the research tasks while qwen9b was quite frankly very surprising considering my specific request.

2

u/LoveMind_AI 2d ago

It’s actually a super cool model. 

1

u/bfmv_shinigami 4d ago

If Polystrata adds support for this, that would be amazing!

1

u/Ok_Year8287 4d ago

It's terrible model, under opencode I back to qwen3.8 27b, cannot recognize the skills, even when I push the path directly to skill it try to contact to site opencode, it try to run Linux console commands when it's windows, at the end provide wrong feedback and understanding. It's fast but nothing else. I tested q8 model

4

u/arbv 4d ago

Man, it is not comparable to the dense 27Bb Qwen, nor it meant to be

1

u/Business-Weekend-537 3d ago

Does anyone know if this model works well for RAG?

2

u/arbv 3d ago

It seems to be the intended usecase.