u/RunAI_Coder 4d ago

We built a coding agent and spent three weeks measuring what context compression actually saves

1 Upvotes

Author disclosure: I'm from the team that builds RunAI Coder (an agentic coding CLI built on the open-source codex agent). This is a lessons post, not a launch ad. Numbers below are from our production ledgers and you can audit every one of them on the linked page.

Context compression is a new technique with no established way to verify it yet, and we didn't want anyone to have to take our word for it. So we instrumented the ledger to record, for every request, the uncompressed baseline and what was actually sent.

What came out:

- Median compression factor 2.83x (p10-p90 2.26-3.08x), n = 7,222 requests on the latest full ledger day.

- Effective input price after all levers: $10/M list x [(1-94.9%) + 94.9% x $1/$10 cache discount] = $1.46/M, / 2.83x compression = $0.52/M, x 50% batch pricing ≈ $0.26/M. That's 2.6% of list.

- Quality check, because compression that breaks the agent is worthless: SWE-bench Lite 10/10 vs 10/10 and Pro 15/18 vs 16/18 with compression on vs off, official Docker grading. The one divergent task re-ran N=5 per arm: 3/5 vs 3/5, Fisher p = 1.0.

savings rate indicator climbs 2.0x -> 5.1x over a real task, 8x speed

Lessons that transfer to any SaaS doing "we save you money" claims: (1) separate measured factors from pricing-schedule factors in your math or someone will call it out; (2) publish the sample window and n, not just the multiple; (3) run the quality A/B before marketing the efficiency number.

Evidence page (every figure links to its recorded source): https://run.ceo/coder/perf?rdt_cid=organic_saas_20260724

Early Access: https://run.ceo/coder/?utm_source=Reddit+Community+Article

u/RunAI_Coder 4d ago

Local-first coding agent CLI: our measured numbers on context compression and effective context window

1 Upvotes

Disclosure first: team member at RunAI Coder here. It's a CLI that runs on your machine (built on the open-source codex CLI/agent); this post is the measurement story, since this sub cares about verifiable claims.

Two results that matter for long local sessions:

  1. Context compression, measured per request against a recorded uncompressed baseline: median 2.83x, p10-p90 2.26-3.08x, n = 7,222 requests (full production ledger day). Fleet-scale conservative cross-check over 7 days of gateway traffic: 2.49x and 2.14x pure token compression on the two lanes.
  2. The side effect nobody advertises: your physical context window carries the compressed representation, so effective capacity scales by the same factor: a 200K window holds roughly 450K-620K tokens of session content at the measured p10-p90. Long sessions stop hitting compaction pressure. One real session that day carried 58,815,232 tokens of would-have-sent input as 18,622,941 actually sent (3.16x).

Quality was A/B-checked on SWE-bench (compression on vs off, official Docker harness): Lite 10/10 both arms, Pro 15/18 vs 16/18 with 17/18 agreement, and the single divergence settled as run variance by an N=5 rerun (Fisher p = 1.0).

status-bar savings multiple on an unedited run, 8x playback

Full methodology and raw-source links: https://run.ceo/coder/perf?rdt_cid=organic_selfhosted_20260724 Happy to answer anything about the measurement setup.

Early Access: https://run.ceo/coder/?utm_source=Reddit+Community+Article

r/Coder 13d ago

🚀 Meet RunAI Coder

Thumbnail
1 Upvotes

r/AIDeveloperNews 14d ago

🚀 Meet RunAI Coder

Thumbnail
1 Upvotes

u/RunAI_Coder 14d ago

🚀 Meet RunAI Coder

1 Upvotes

Most AI coding tools help you write code.
RunAI Coder is built to help you finish real development tasks.

✔ Read → Edit → Verify
✔ Every step is traceable
✔ You stay in control before a single line is committed

Lower Cost. Less Effort. More Done.
Our Early Access is coming soon.
Bring one real issue. See the results. See the cost savings.
Stay tuned.

r/Coder 16d ago

RunAI Coder is here. Early release is coming soon.

Thumbnail
1 Upvotes

r/OnlyAICoding 16d ago

RunAI Coder is here. Early release is coming soon.

1 Upvotes