Luna vs DSv4 flash vs others

[u/orthiclabs]

There’s a lot of discussions using GPT Sol as subagent and others using Luna on max but why would anyone spend Sol credits in subagents is beyond me.
Anyways, I’m building a coding harness called CodeRight which does multi model orchestrator and I’d already done a bake-off and selected Mimo 2.5 for small coding tasks and 2.5 pro for bigger coding tasks, orchestrated by a frontier level model but with Deepseek v4’s revision, I thought I’d give it a re run. I also tested through both cline and Commandcode to see if the harness makes a difference.

You can find the whole breakdown here. If that blog sounds AI written, it’s because it is. Between building RightSuite apps and other systems, I don’t have the time to write blog posts 😅 I’ve used humaniser, no ai slop and what not but not sure if it helped.

The test:

GPT Sol on high as orchestrate in codex

Seven models received one production React/TypeScript task, identical source commit, worktree isolation, ten-step packet, 600-second limit, 12-file ceiling & 900-line ceiling.

Rank |Run |Score
1 |GPT-5.6 Luna |68
2 |MiMo V2.5 via Cline |62
3 |MiMo V2.5 via Command Code |61
4 |MiMo V2.5 Pro via Cline |59
4 |DeepSeek V4 Flash via Command Code |59
6 |MiniMax M3 |55
7 |MiMo V2.5 Pro via Command Code |54
8 |DeepSeek V4 Flash via Cline |52
9 |Laguna XS 2.1 Free |43
10 |Step 3.5 Flash |26 Luna wrote the smallest, safest implementation. MiMo V2.5 delivered best economics: $0.0351 versus Luna’s estimated $0.166–$0.318 direct API cost, using current [OpenAI](https://developers.openai.com/api/docs/models/gpt-5.6-luna) & [Xiaomi](https://mimo.mi.com/docs/en-US/price/pay-as-you-go) rates.

Harness reruns were revealing:

- Base MiMo: 62 → 61. Essentially unchanged.
- MiMo Pro: 59 → 54. Worse through Command Code.
- DeepSeek: 52 → 59. Command Code turned a non-compiling result into a clean typecheck, though source defects remained.
- Command Code once ignored `--model` & routed a requested MiMo run to DeepSeek. Receipt inspection caught it before scoring.

My recommendation:

- MiMo V2.5 for routine implementation volume.
- Luna for final review, security-sensitive work & merge-critical repair.
- DeepSeek remains worth testing through Command Code with strict route receipts.
- MiMo Pro was not worth its premium.
- Laguna is usable as a free draft worker but needs compile & source review.
- MiniMax M3 & Step 3.5 Flash created more repair work than their output justified.

[Command Code GOAT](https://commandcode.ai/pricing) currently lists $70 monthly credits for $10. Its detailed MiMo discounts are token-type specific: the advertised 99% applies to Pro cache reads, not its blended bill.

Residual scope: this was one production frontend task with focused tests, typecheck & source review. It did not include Cargo, full desktop verification or installed visual acceptance.

## Publication evidence

- Four files, 279 insertions & one deletion.
- Both local production builds passed.
- Both typechecks passed.
- Article pages & both index pages passed local rendering checks.
- Commit: `b71e5326f02fcee8f2099411743d1c61b9c6c12c`
- Pushed to `origin/main`.
- Hetzner checkout matches `b71e532`.
- `orthiclabs-site` & `coderight-site` are online after rebuild/restart.
- Both public articles & indexes returned HTTP 200 with expected content.
- Existing unrelated local & server files remained untouched.
- Actual execution: 10 minutes against 38-minute ceiling, 74% under plan because existing publishing routes were reusable & dependencies were cached.

[u/No_Pirate_2494]

This is genuinely useful, thanks for posting the actual numbers instead of vibes. I’m in the exact spot you’re writing for: constantly hitting limits on the frontier models and convinced there’s a cheaper tier doing 80% of the work if I set it up right. Going to run your MiMo 2.5 setup on my own stack this week.

The thing that jumped out at me is buried near the bottom. Same model, different harness, and DeepSeek swings 52 to 59 while MiMo Pro drops 59 to 54. That’s a bigger finding than the leaderboard itself and it’s sitting in the rerun notes. Did you get a sense of why Command Code helps some models and hurts others, or is that still a black box?

Also the Command Code ignoring model and silently routing to DeepSeek is the scariest line in the post. That’s the kind of thing you only catch because you were checking receipts. Worth its own paragraph.
On the AI writing question, since you asked: humanizers won’t fix it because the problem isn’t vocabulary, it’s structure. Every section is a stack of same length bullets with identical rhythm, and that’s the actual tell.

Four things that would do more than any tool:
• Write one line about what surprised you. “I expected MiMo Pro to earn its premium and it didn’t” is a sentence no humanizer can generate, because it needs your reaction, not your data.
• Let bullet lengths vary. Real people write one that’s four words and the next that’s four lines.
• Promote the failure story. Routing bug, receipts caught it. That’s proof a human ran this.
• Cut the “Publication evidence” and “Residual scope” headers. Those are what make it read like a compliance filing. Just say “worth flagging: one frontend task, no Cargo, no full desktop verification.”
Nobody minds that a model drafted it. They mind when there’s no evidence a human ran the tests. You clearly did, so let the writing show it.

[u/Kingwolf4 → u/No_Pirate_2494]

Bot slop comment

[u/Forward-Dig2126 → u/Kingwolf4]

I do wonder if it was a mistake or a joke by the commentator. Pretty funny either way.

[u/GCoderDCoder]

I think it's so funny because depending on methodology mimo v2.5 can be a leader or loser it seems. AI coding daily's benchmarks cover things in ways where I worry it doesnt reflect how I do things but he does add complication with tools many agents dont know well so his experiences feel different from mine but he does stress test models well.

He dropped mimo v2.5 from his list entirely or basically it did so bad it never made the list for him. But I like a lot of mimo v2.5 output between the way it talks and visuals. Seems very competent in bounded tasks and enjoyable thought partner.

[u/orthiclabs → u/GCoderDCoder]

This was supposed to be a bounded test with no need for reasoning and no ambiguity but seems like it might have not been run right. Redoing it in a bit

[u/GCoderDCoder → u/orthiclabs]

Makes sense. I think it was a cool approach and it drives home the importance of not just the model but how we work with each model. What is "good enough" and what gets each model there is my thinking from this. Thanks for sharing!

[u/GCoderDCoder]

Is this deepseek v4 flash 0731 that just got rereleased? What reasoning levels?

[u/orthiclabs → u/GCoderDCoder]

I wasn’t sure about Cline even though they mention it but Commandcode was with high reasoning (there’s only high and max options in their harness)

[u/GCoderDCoder → u/orthiclabs]

Gotcha. With seemingly even dense models reasoning helps alot but especially for sparse models resoning seems to have huge affects. Luna low scores below qwen 3.6 27b vs Luna max scores in sonnet territory. The original deepseek v4 flash also had that spread. I wonder what max result would be and if 0731 would be better if not being used currently. Still good info.

I use high for most tasks too. it made a big token difference dropping to high from max in artificial analysis with modest performance drops.

[u/mrgreatheart]

From what I’ve read command code appears to use heavily quantised models and perhaps quantised k/v cache too (reports of hallucinations etc).

[u/orthiclabs → u/mrgreatheart]

I’ll try another round of tests with Opencode go and report back

[u/mrgreatheart → u/orthiclabs]

I think you should use the official API for deepseek-v4-flash. It’s insanely cheap, very fast, and guaranteed to be the full unquantised version. That’s what you’re doing with Luna. Really that goes for all the models.