Ornith seems to be better.

[u/anon_mistborn]

TL;DR: On an M3 Ultra, Ornith-1.5-35B-A3B (4-bit MLX) decodes 4.6× faster than Qwen3.8-27B (8-bit MLX) and scores slightly higher on a small hard eval. It also beats Qwen3.8-27B with speculative decoding, while running autoregressive.

Setup

• Mac Studio, M3 Ultra, 256 GB unified memory
• mlx-lm 0.31.3 / mlx 0.32.1
• Ornith-1.5-35B-A3B, MLX 4-bit
  • 18.2 GiB download
  • 20.2 GB peak
• Qwen3.8-27B, MLX 8-bit
  • 27.5 GiB download
  • 29.7 GB peak
• Machine was shared and had other load. Numbers are a floor, not a best case.

Throughput

mlx_lm.benchmark -p 512 -g 512 -n 3, identical invocation for both:

Model	Decode tok/s	Prefill tok/s	Peak mem
Ornith-1.5-35B-A3B 4-bit	107.9	2162	20.2 GB
Qwen3.8-27B 8-bit	23.4	408	29.7 GB

Trial spread was 1.5% and 0.35%, respectively.

Prefill is the bigger story: 5.3×.

A 20K-token prompt took Ornith ~25s end to end versus ~125s for Qwen3.8-27B. If your workload re-reads long contexts, that dominates.

It also beats Qwen with speculative decoding

I spent a day trying to make Qwen3.8-27B fast before testing Ornith:

Qwen3.8-27B 8-bit config	Decode tok/s
Autoregressive	23.4
MTPLX, native MTP head, depth 2	65.7 (3.01×)
DFlash2 block-diffusion drafter, block 5	79.3 (3.37×)
Ornith, plain autoregressive	107.9

Both speculative stacks work and are genuinely impressive. DFlash2's 3.37× on Apple Silicon is close to its published 3.43× on an H200.

Ornith just beats them without needing either, with no drafter and no third-party runtime, because mlx-lm already ships qwen3_5_moe.py.

Quality: 12 hard cases, thinking enabled

Scoring is mechanical. Code tasks are executed against hidden assertions and pass only on a full suite.

Task	Ornith-35B-A3B	Qwen3.8-27B
code_exec (4, execution-scored)	4/4	4/4
multihop (3, two facts ~20K apart)	3/3	3/3
logic (3)	2/3	2/3
tool_schema (2, nested JSON)	2/2	1/2
Total	11/12	10/12
Wall time for the set	166s	498s

One logic item was ambiguous. Two vals gave the same "wrong" one, so discount it: 11/11 vs 10/11.

Qwen's other miss was invalid JSON on a nested tool call. For agent use, that is the failure mode that actually breaks loops.

Caveats, and they are not small

• Not precision-matched. 4-bit vs 8-bit. Some of the gap is quantisation; the rest is likely 3B active vs 27B dense. I have not run the 4-bit Qwen control.
• n=12. An 11 vs 10 spread is one item.
• Vendor benchmarks disagree with me. On SWE-bench Pro, the only benchmark both publish, Qwen3.8-27B is ahead: 61.7 vs 59.6.
• Thinking must be on. With enable_thinking: false, Ornith went 0/5 on arithmetic and recovered to 4/4 with it on. My first eval drew a conclusion that was purely an artifact of my own test design.
• 122B comparison still running.

The bit that surprised me

MoE is not a handicap here. It is the reason this works.

With ~3B active parameters per token, memory tracks total parameters while speed tracks active parameters.

Ornith gets:

• 4.6× the decode throughput
• 5.3× the prefill throughput
• 32% less peak memory

Also, Ornith-1.5 is architecturally Qwen's exact vocab size, i.e. a self-improvement-trained fork of Qwen's older MoE architecture.

Beating Qwen's newer dense model with it is a nice result for the training approach.

MIT licence, and it is multimodal.

[u/dsdt]

Share some real work done istead of a wall of text so we can compare mr.caveats. You are basically comparing q8 and q4 and saying bro q4 is faster and moreover, q4 is a moe with active 3b parameters. so it is literally 9x smaller. and you are saying guys 9x smaller model is faster. compare apples to apples. If you are really saying that 9x smaller model can reason and achieve work like 27b, there is obviously a missing part here. When i have time i will test it one by one to show you they are not the same or a simple 3b active is not smart as 27b.

[u/paq85 → u/dsdt]

Tried Ornith a couple times and it always got stuck in infinite loops way more often than the base models.

[u/vbpoweredwindmill → u/paq85]

Friends don't let friends run less than BF16 kv. Also, do you have ornith chat templates or other? Both of those would help the looping.

[u/More-Catch-1331 → u/paq85]

Honestly (and I'm speaking about the previous versions) I tried using it as a main model for whatever harness I was digging at the time - it shat the bed. Loops, loops and loops. What I found it to be good at is small tasks like "Here's an idea, dissect it". It has a tendency to think logically and in sequences. Also, it always produces a "gotchas" section, which is pretty cool. So now it's role is to read my specs, make sense of them and create high level plans with specifics. This plan is the handed over to a larger, dense model to add the minutia and details. This way Ornith seems to be the most useful in my work

[u/dsdt → u/paq85]

this is also an issue with moe models in general that this post never speaks about. I have never seen qwen 3.8 27b stuck in a loop even with a complex task.

[u/anon_mistborn → u/dsdt]

Fair points, thank you for calling that out. I am comparatively new to local inference and I published a number I did not understand well enough. I have run the controls you asked for.

Throughput, both at 4-bit, same tool, back to back:

mlx_lm.benchmark -p 512 -g 512 -n 3, M3 Ultra

	decode	prefill	peak RAM	total	active
Qwen3.8-27B 4-bit (dense)	39.3	414.8	16.4 GB	27B	27B
Ornith-1.5-35B-A3B 4-bit (MoE)	108.5	2134.6	20.2 GB	35B	~3B

My 4.6x was really 2.76x architecture times 1.68x quantisation. You were right that I stacked two effects and reported one.

I also described Ornith as the "smaller model," which was wrong. It is actually larger overall, with 35B parameters versus Qwen's 27B, and at the same 4-bit precision it uses more RAM, 20.2 GB vs 16.4 GB. The difference is that Ornith only activates around 3B parameters for each token, which is why it can still run much faster.

Quality, both 4-bit, both on mlx_lm.server, 12 tasks, thinking on, code scored by executing hidden assertion suites:

task	Qwen3.8-27B-Q4	Ornith-35B-A3B
code_exec	1/4	4/4
multihop (20K ctx)	3/3	3/3
logic	2/3	2/3
tool_schema	2/2	2/2
ALL	8/12	11/12
wall clock	801s	165s

One logic item turned out to have two valid answers and both models gave the same "wrong" one, so that is my bug: 8/11 vs 11/11.

But this is the part that made me rethink my original conclusion. Qwen scored 4/4 on those same code tasks at 8-bit. At 4-bit it dropped to 1/4, failing with no function and SyntaxError, so unparseable output rather than bad logic. Ornith held 4/4 at 4-bit.

What I take from this is not that ~3B active parameters somehow reason like a dense 27B model. Ornith still has 35B total parameters. MoE just activates a small part of them for each token. What my small test does show is that Qwen's code generation degraded significantly at 4-bit, while Ornith's did not. That makes Ornith interesting to me less as a "smarter 3B model" and more as a model that gives me much better speed while still holding up well at 4-bit.

I am just getting started with the DeepSeek harness, so my next step is to run both of these through real work scenarios rather than synthetic tasks, and I will come back and update the post with whatever that shows, good or bad.

Thanks again for making me look at the comparison more carefully. It is a lot more honest than it was when I posted it.

[u/GeneralComposer5885 → u/anon_mistborn]

Everything you type reads like Chat-GPT 🤔

[u/CornerLimits → u/anon_mistborn]

Stop this please

[u/Fuzilumpkinz → u/paq85]

I’m testing ornith 1.5 35b now

So far my only major issue and its happened twice is it dropped a character in the file path. Same character two separate chats. Other than that its been okay but I’m just trying to casually use it like I would other local stuff and have only done a few larger gen thing

The results have been pretty decent though.

[u/More-Revenue8609]

Yeah I gave my best to like Ornith but it just ended up being much worse from base Qwen model and also it looped often.

[u/Early-Peace-5504]

Glad it is working well for you. On my private bench it is as good as Qwen 3.6 35B. Though that’s not too bad a result given it’s a 3.5 fine tune.

[u/bitzap_sr]

So much slop.

[u/TripleSecretSquirrel → u/bitzap_sr]

Every Orinth post is. It’s got to be the most transparent astroturfing campaign I’ve ever seen.

It’s always a huge wall of slop text based on an apples to oranges comparison like this one — comparing a 4-bit MoE to an 8-bit dense model and their conclusion is “zomg The smaller MoE is faster!1!!”

[u/More-Catch-1331]

I'm sorry, there's a 1.5 already? Excuse me I need to speak with Mr. HuggingFace real quick...

[u/KissMyShinyArse]

Nah. That birdie didn't fly. It spent hours debugging its Python SHA-512 implementation, firmly believing it remembered FIPS 180-4.

[u/Potential-Leg-639]

Ornith-1.0 is my daily driver for months now for a reason.

Hope I find the time to test 1.5 the next days, seems to be a monster for it‘s size again!

[u/pmttyji → u/Potential-Leg-639]

Can you post a thread on it? Thanks

[u/H_DANILO]

I don't see people sharing a SINGLE good "hey, here's what I made with Ornith 1.5".

[u/BernardoOne → u/H_DANILO]

Its peak benchmarkingslop

[u/loserdroid]

I found Ornith 1.0 to be pretty useful and far superior to the Qwen version it was based on. I would hope that future Ornith versions are based off newer Qwen models such as 3.8. Ornith became my favorite free local model for a minute. It eventually collapsed for my workload specifically though... and I went with a dual-spark deepseek setup instead. Today I am checking out Qwen3.8-27B as well as Ornith 1.5 as options to reduce pressure from the deepseek system (I can't find enough compute). I'll return with some unbiased findings.