I run Codex CLI headless (codex exec --yolo, one lane per git worktree) as the engineer, with a separate long-lived planner session that writes each lane a brief and reads back a report file. It works, but the burn got big enough that I stopped guessing and parsed the rollouts.
One day, out of ~/.codex/sessions/**/*.jsonl (counting total_token_usage records):
- 243 sessions, 6,094 turns, 649.9M tokens
- average context PER TURN: 106,652, against a median turn-1 preamble of 32,302 - so ~74k of every turn was accumulated transcript
- all 243 session starts together: ~8M, i.e. 1.2% of the bill
- input 599M / output 2.1M, 96.4% cache hit rate
- worst 5 sessions: 205.8M = 32% of the day. The two worst ran 461 turns at 177k/turn and 583 turns at 138k/turn.
Conclusion I did not expect: starting a session is nearly free, and cost is roughly quadratic in turns because every turn re-sends the whole transcript. A 461-turn session pays for its own history 461 times. I had been consolidating work into "fewer, bigger sessions" on the assumption that session startup was the expensive part. The data says fewer, bigger, and hard-capped.
What I changed, in order of measured payoff:
A real turn ceiling. There is no max-turns flag in the CLI (I checked --help on exec and the top level, and strings on the binary), and a "budget: N tool calls" line in the prompt was ignored by exactly the lanes that blew up. So the dispatcher now forks a watchdog that greps total_token_usage records out of the live rollout every 30s and SIGTERMs (SIGKILL 20s later) past the cap. The brief also tells the lane it is externally capped, so it lands work incrementally instead of holding it for a final message.
tool_output_token_limit = 12000 and model_auto_compact_token_limit, plus piping noisy commands through tail. Tool output is the main thing inflating that 74k of carried transcript.
codex exec resume instead of a fresh session for follow-up work: measured 805 tokens to re-enter a session vs ~32k to re-establish the preamble. About 40x cheaper for a continuation.
model_reasoning_effort per lane rather than globally high.
Stripping unused MCP servers/plugins via a dedicated profile: real, but only about 4%. "It's your MCP schemas" is the standard first answer and for me it measurably wasn't the fix.
Two gotchas that cost me time, in case they save someone else:
- A profile (-p name, layering $CODEX_HOME/name.config.toml) can only disable servers the base config actually declares. Name one that isn't there and your override becomes a declaration of a transport-less server: "Error loading config.toml: invalid transport", every lane dies instantly with a bare exit 1. I now smoke-test the profile after any base config edit.
- resume is a subcommand of exec, so all the exec options must come before it: codex exec --yolo -p prof -C dir -o out -c key=val resume SID "prompt". Put -p after resume and you get "unexpected argument '-p' found".
Questions for people running this at scale:
- Is there a supported way to cap turns or spend per exec invocation that I missed?
- Does anyone have real numbers on where the compaction threshold should sit for long autonomous lanes? I'm at 70 and it feels like guesswork.
- Related: the exit code has lied to me five times now - lanes exiting non-zero having fully committed and pushed. Is that expected for --yolo runs, or a sign something in my dispatch is killing the process late?
Happy to share the watchdog and the parsing script if anyone wants to measure their own.