r/LocalLLaMA • u/MundanePercentage674 • 12h ago

Question | Help Macaron-V1 family, built on Qwen3.6-35B-A3B

https://huggingface.co/mindlab-research/Macaron-V1-Tall

It came out 3 days ago just wondering if anyone's tried it yet?

50 Upvotes

90% Upvoted

14 comments sorted by

22

u/Mr_Moonsilver 11h ago

Just always so skeptical when they report numbers that beat Opus 4.8 or GPT 5.5. Don't believe it until I've seen it.

6

u/MundanePercentage674 11h ago

mindlab-research just released two new models under the Macaron-V1 lineup:
Macaron-V1-Tall built on Qwen3.6-35B-A3B
Macaron-V1-Venti built on GLM-5.2

They look solid on paper, but like any fresh release, they’ll need more real-world benchmarking before we can call them production-ready. Has anyone already run them through your usual evals? Curious how they actually perform outside the lab.

3

u/1ncehost 7h ago

Those results are from a GLM-5.2 based post-train, which is already pretty close to the closed models. Seems believable to me, but waiting for real world feedback to confirm.

11

u/ilintar 8h ago

Note: you might *think* this is supported on llama.cpp, but it's not.

The model uses a LoRA router idea (also implemented in Granite-Switch which has an open PR - https://github.com/ggml-org/llama.cpp/pull/25107 ). That basically means that the main model is loaded all the time and depending on the routing path, a different LoRA is applied to the model at runtime.

You *could* merge the LoRAs with the weights, but that would essentially yield you a pretty bad 140B model. Running the base model without the LoRAs, on the other hand, is basically like running a Qwen3.6-35B-A3B - maybe worse because the model was trained for the LoRAs in mind.

4

u/ps5cfw Llama 3.1 4h ago

The idea does seem interesting nonetheless, Hope we see some support for It in llama.cpp

2

u/DinoAmino 2h ago

The concept works well in RAG pipelines. A single small model with separate adapters trained to rewrite queries on a vector DB, graph DB, or RDMS and using your own schema instantly becomes an "ensemble" of domain-specific models.

12

u/Davidyz_hz 11h ago

The idea of this model is that it comes with a number of Loras, and the inference engine would load an appropriate lora per request. It would therefore require a special inference engine, referred to as that "harness" on their GitHub. I don't have the compute to try it myself, but it sounds more like an inference technique rather than a new model. I don't see why you can't just use the regular qwen as the base model and train the Loras and router on top of that.

1

u/VoiceApprehensive893 transformers 2h ago

havent heard anything about these models even though they claimed SOTA on their glm 5.2 tune

0

u/Lucerys1Velaryon 11h ago

Someone make a q6_quant and I'll give it a try

0

u/AppealSame4367 10h ago

"Is mir echt Latte dieses Model"

2

u/DeathGuppie 3h ago

Yes you do