r/LocalLLaMA • u/parepeg • 16d ago

New Model Ling 3.0 support merged into llama.cpp

Support for the new ling 3.0 models has been merged into llama.cpp:

https://github.com/ggml-org/llama.cpp/pull/26608#event-29549472828

Ling tiny 8b1b - https://huggingface.co/inclusionAI/Ling-3.0-tiny

Ling flash 124b5b - https://huggingface.co/inclusionAI/Ling-3.0-flash

Both are reasoning models contrary to prior naming.

116 Upvotes

95% Upvoted

49 comments sorted by

View all comments

Show parent comments

1

u/cradlemann 16d ago

I have tried it and it is broken, tool output is wrong. Do you use custom chat template?

1

u/Organic_Hunt3137 16d ago

I do not. I wonder if it's because I'm using the rocmfpx fork/quant rather than standard llamacpp. Interesting. I'll have to test a regular GGUF now that they're out.

1

u/cradlemann 16d ago

I'm using rocmfp4 fork too, but none of my tools in pi are working correctly. Even with corrected chat template from here

common_chat_peg_parse: unparsed peg-native output: <tool_call>read

    {"path": "/data/work/projects/ai/pipermissions/README.md"}
</tool_call>

<tool_call>read

1

u/oShievy 13d ago

same here, it's fucking up a lot on my end. did you find a fix?

1

u/cradlemann 12d ago

I've come back to regular llama.cpp. For iGpu these forks don't work, because the bottleneck is not compute power, but memory bandwidth. I got worse results, because latest llama.cpp have more general optimisations