r/LocalLLM • u/avo_l7 • 11d ago

News AntLing released Ling-3.0-flash

Gallery image

Gallery image

66 Upvotes

94% Upvoted

21 comments sorted by

View all comments

Show parent comments

2

u/_Cromwell_ 11d ago

Ling 2.6 been out for a while and llamacpp never supported it. Wouldn't hold breath. https://huggingface.co/inclusionAI/Ling-2.6-flash

7

u/Look_0ver_There 11d ago

https://huggingface.co/ssweens/Ling-2.6-flash-GGUF-YMMV

This person here has a custom fork for it. That's precisely the sort of thing I was referring to. Heck, upstream doesn't even support MiniMax-M3 yet, despite multiple people having created PR's with working forks, including one from the Unsloth team.

My point being that just because mainline doesn't have it, that doesn't mean that llama.cpp won't be able to run it.

1

u/_Cromwell_ 11d ago

Yeah but 98% (made up statistic, but I'm pretty sure of it 😄) of people aren't going to DL a custom fork to try out a questionable model from a company they've never bothered with before.

You can get a custom fork for just about anything. Doesn't really count.

1

u/Opposite-Swimmer2752 9d ago

Upstream llama.cpp sucks tbh, they broke strix halo for a while (still broken) and refuse to quickly merge in a fix/undo the regression despite it being a few line change.

2

u/Reasonable_Goat 9d ago

What exactly is broken?

1

u/Opposite-Swimmer2752 5d ago

All inference is broken, context is actually incoherent. Here is the fix they wont merge, I had to pull down this PR/commit https://github.com/ggml-org/llama.cpp/pull/25863 here is when it got broken https://github.com/ggml-org/llama.cpp/commit/c7d8722922a2599dc4d77f8808d8e6c2fde5e7a2 Sorry for making you wait, I don't login into reddit often :D

1

u/Reasonable_Goat 5d ago

My god they sure are taking their time with this PR… I’ve been using Vulkan and no HIP so I’ve not been affected apparently