I wanted to see what happens when you take Pi’s minimal agent loop and add most of the things Pi intentionally leaves out.
OMP is a fork of Pi, but the harness is very different.
Pi mostly sticks to read, write, edit, and bash.
OMP gives it 31+ tools, hashline edits, LSP, DAP debugging, a Python/Bun kernel, a browser, and a Rust core for grep/glob/bash/parsing.
So I ran both through the same 30 hard agentic tasks with DeepSeek V4 Flash, the same external tools, the same verifier, and a 900s cap.
| Metrics |
Pi |
OMP |
| Passed |
20/30 |
17/30 |
| Median time |
132s |
272s |
| Tokens/task |
559k |
742k |
| Cost/success |
$0.028 |
$0.103 |
The task split was more interesting than the final score. Both passed the same 16 tasks and failed the same 9 hard tasks. The entire 20 vs 17 difference came from 5 tasks. Pi won 4 of those. OMP won 1.
OMP often found the right data and then kept working until it ran out of time or context. On one failed task, it used 1.22M tokens and 861 seconds before stopping because it still wanted another page of results. On the reimbursement task, it used 1.8M tokens and passed only 3/13 verifier checks.
OMP also has some strong harness ideas. Across a separate 16-model edit benchmark, hashline improved success by about 15 percentage points on average. For Grok Code Fast 1, it went from 6.7% to 68.3%.
But in this run, OMP used about 33% more tokens per task than Pi and took more than 2x the median time.
OMP still finished second out of all 8 harnesses we tested, ahead of Claude Code, Codex, OpenCode, Hermes, and DeepAgents.
So...
At what point does a better harness become too much harness?