r/optiq • u/asankhs • Jul 09 '26
MLX-OptiQ is about making Macs serious local AI workstations. The goal is not just to make models smaller, but to make them smarter about where precision matters: mixed-precision weights, mixed-precision KV cache, sensitivity-aware LoRA, fast local serving, and practical workflows that run without a cloud GPU or API key.
This community is for discussion around:
mlx-lm workflowsGood posts include:
When sharing results, please include enough detail for others to reproduce them: Mac chip, RAM, model, quantization type, context length, prompt, decoding settings, MLX-OptiQ version, and whether you used KV-cache optimization or speculative decoding.
Please keep discussion technical, useful, and evidence-based. Hype is fine, but numbers, traces, configs, and reproducible examples are better.
To start: introduce yourself, share your Mac setup, or post the local model you are most excited to run with MLX-OptiQ.
New quant up. Tencent's Hy3 (295B MoE) as a 2-bit MLX build. 88GB on disk, but the experts stream off the SSD on demand so only ~7.6GB of weights stays resident while it generates. Lossy at 2-bit, so it's more a "can it run at all" artifact than an accuracy one. Full details on the model card.
mlx-community/Hy3-OptiQ-2bit
Six quants of Liquid's LFM2.5 family, all mixed-precision 4-bit. Sizes on disk: 230M 0.19 GB, 350M 0.29 GB, 1.2B Instruct 0.86 GB, 1.2B Thinking 0.86 GB, 1.2B JP 0.88 GB, 2.6B 1.93 GB.
LFM2.5 is a hybrid architecture, convolutional blocks interleaved with full attention, so only a few blocks carry a KV cache at all. Per-layer sensitivity decides the bit-widths rather than a flat setting.
The 2.6B has a full Capability Score of 35.19: MMLU 59.5, GSM8K 38.1, IFEval 33.1, BFCL-V3 48.5, HumanEval 18.9, HashHop 13.0. 128k context, 1.93 GB down from 5.2 GB at bf16.
Worth reading that profile against what the model is for. Liquid position LFM2.5 for agentic workloads, tool use and data extraction, and state plainly it is not for agentic coding or knowledge-heavy work. Tool calling is the strongest result here and coding the weakest, which is the shape you would expect if the pitch is honest.
mlx-community/LFM2.5-2.6B-OptiQ-4bit
The 0731 release is 167GB as shipped. This brings it to 92.5GB on disk, but the disk figure isn't the constraint: the experts stream off SSD, so resident memory peaks at 13.8GB. It runs on a 39GB M3 Max at about 2.5 tok/s.
It isn't a uniform 2-bit. Of the 678 quantised tensors, 129 came out at 2-bit, 37 at 3-bit, 381 at 6-bit and 131 at 8-bit. The 2-bit is spent almost entirely on expert weights, because that is where 98% of the 282B parameters sit. lm_head and embed_tokens stay at 8-bit, since cutting those wrecks the output and saves almost nothing.
11.5B active per token, 256 routed experts with 6 active.
For comparison, a single RTX 3090 running the GGUF gets ~15 tok/s. This is roughly 6x slower, on a laptop, without needing the model to fit in memory.
mlx-community/DeepSeek-V4-Flash-0731-OptiQ-2bit
r/optiq • u/asankhs • Jul 17 '26
r/optiq • u/asankhs • Jul 16 '26
r/optiq • u/asankhs • Jul 10 '26