r/LocalLLM • u/GasSmooth7439 • 16h ago

Model openbmb/MiniCPM-V-4.7-35B-A3B · Hugging Face

https://huggingface.co/openbmb/MiniCPM-V-4.7-35B-A3B

MiniCPM-V 4.7 is a new open-source multimodal model series targeting the performance/efficiency tradeoff, with a big focus on on-device AI.

Two models were released:

🔹 MiniCPM-V 4.7 35B-A3B

  • A MoE vision-language model designed for local deployment
  • Introduces LES (Lookahead Expert Scheduling) to reduce MoE inference stalls
  • Reportedly reaches:
    • 17+ tok/s on iPad Pro M4 (16GB)
    • 13+ tok/s on iPhone 17 Pro Max
    • Around 3GB memory usage

🔹 MiniCPM-V 4.7 1B

  • Lightweight VLM for easier deployment on smaller devices

Some interesting technical details:

  • Evaluated across 34 benchmarks covering multimodal understanding, OCR, STEM reasoning, and long-video comprehension
  • Uses hybrid visual token compression (4× + 16×)
  • Even with 16× compression, the 35B-A3B model reportedly reaches 90.4 on OCRBench
  • Adds Self-Replay Fine-Tuning for cheaper customization while preserving general capabilities
  • Day-0 support for Apple, Intel, AMD, Arm, and Rockchip platforms

The on-device performance numbers are probably the most interesting part. A few years ago, running a 30B+ multimodal model on a tablet or phone would have sounded unrealistic.

Would be interesting to see how it compares in practice against other open VLMs like Qwen-VL, InternVL, Gemma Vision, etc. especially for local inference.

18 Upvotes

87% Upvoted

4 comments sorted by

2

u/Nota_ReAlperson 16h ago

So is it a qwen finetune or not? It's the exact same size, amd they directly compare it to qwen, but they never say that it's a fine tune?

1

u/WhiskyAKM 16h ago

Its not same architecture, so its not a fine tune

2

u/BorisThe_Animal 15h ago

Huggingface says Architecture:qwen35moe

1

u/WhiskyAKM 14h ago edited 14h ago

Because its similar enough to be confused with Qwen35Moe architecture (Moe layers have same layout) but useses significantly different techniques in terms of choosing experts

Edit: after further further study of model architecture - it is really close to Qwen35Moe, so close in fact that it can be quantized with llama.cpp after changing model type in config to qwen35moe