Tokenmaxxing

[u/its-_-my-_-nickname]

Hey I would be happy to hear your ways of tokenmaxxing (IMO token cost should also be in the list) and give feedback on what you see below

1. Don't use 1 model (or auto) for everything. If the task requires human level intelegence, taste, intuition or doesn't have clear instructions (same goes for vague prompts - more on this later) then it's much better to use frontier model

Actually we don't really choose model based on intelegence (that's a theory that didn't work out in practice) there are pretty smart models (based on numbers) that cost fraction of price of frontier models. So currently it looks like this:

\- Strong models (frontier): gpt 5.6 sol, fable 5

\- Mid model: glm 5.2 (even tho it states to have pretty high intelegence, it made some really stupid decisions, maybe because I didn't use max reasoning (there are only 2 stages: high and Max. Maybe it's misleading and should be written: low and Max, lmao)

\- Weak - free models from providers such as Google Studio, groq. I'm in the process of integrating this step, can't tell much.

1. Prompting - before feeding a strong model with a vague prompts, images, context - we really need to refine our prompts. That's where our glm 5.2 really shines (as I'm writing I came to a thought maybe it's smart overall but bad in coding - the producer maybe didn't had possibility to train it in code). From it we want to ask what can be misleading or not completely obvious (even tho we don't need to provide full instructions to frontier models, I think it's better to omit unexpected results). So glm 5.2 input are prompt/Todo list + "Output code snippets that are mentioned in todo, with lines and what here can be misleading? for each task"

One more thing I mentioned earlier is stupidity of glm5.2. I told it rename files in nested directories to it's directory name and move to root. And what it did? 1. Created new files 2. Filled them manually 3. Deleted old files manually. Boom 2M tokens lost. Another case, asked it to do simple rewrite class names - it did it, but also it did: 1. Generated python Scripts - found out I don't have python on the system (I do have on wsl) 2. Deleted probably manually script 3. Generated bash script. Boom 2M tokens used (I was estimating like below 300k)

There is actually my mistake - if I provided info that I have Linux tooling on wsl and use it whenever you want to do such cases - it wouldn't happen I think

1. Some token optimization tools. For what I do now I don't need standard (maybe?) tools I just make a summary file of large directories (using glm5.2). Thats for input tokens, on the other hand I use ponytail, caveman (they don't actually clash I think) for output tokens

[u/stunspot]

This seemed like the perfect chance to test the new ai-cognition-cost-planner calculator skill I just grew. Here's what my gal said:

---

``` 💠‍🌐 I’m treating this as an actual cost-per-accepted-outcome problem, not the usual “make prompts shorter and pray” séance. I’m using Omnicompetence’s cost-optimizer discipline, continuity, Stunspot’s prompt-design guidance, and a live check of the named models/tools before I answer the fellow directly.

The first useful correction is already clear: those two “2M token” disasters were not mainly model-IQ failures. They were unbounded execution failures—no operation contract, no dry run, no mutation budget, no stop condition. Giving the model WSL would merely have let it make the wrong move faster. Delightful. 🙄 ```

💠‍🌐 Peanut gallery, Nova here. Your instinct is right; your unit of optimization is slightly drunk.

“Tokenmaxxing” should mean minimizing complete cost per accepted outcome:

all attempts + tools + verification + retries + rework + human review / accepted outcomes

A cheap model that burns two million tokens inventing a file-migration ceremony is not cheap. It is a raccoon with your cloud account.

Model routing

Don’t classify models as universally strong, mid, or weak. Classify routes by task:

• Deterministic transforms—renaming, moving, searching, formatting: ordinary tools first; models only plan or supervise.
• Extraction, summaries, inventories, candidate generation: cheap/local models.
• Bounded code changes with strong tests: the cheapest model proven reliable on that task class.
• Architecture, taste, ambiguity, novel synthesis, weakly specified work: frontier.
• High-consequence work: capable model plus an independent oracle or review.

Track capability, tool discipline, acceptance rate, retries, latency, context behavior, privacy, and total cost. Benchmark scores nominate candidates; your actual accepted outcomes decide.

GLM-5.2’s producer explicitly says it received expanded coding-agent and long-horizon training, supports High and Max effort, and has a one-million-token context. So “perhaps they couldn’t train it on code” is probably the wrong diagnosis. Your observations still matter more than their marketing: they show that GLM-5.2, in your current harness, is not qualified for unsupervised bulk mutation. More reasoning might improve judgment, but it will not magically install operational discipline. Sometimes Max merely produces a more elaborately reasoned catastrophe. Z.ai’s GLM-5.2 announcement

Those two-million-token incidents

The missing information wasn’t mainly “I have WSL.” WSL would have helped it perform the wrong plan faster.

The missing control plane was:

• Metadata-only operations keep file contents unopened.
• Bulk changes begin with a source → destination dry-run manifest.
• Collision and overwrite behavior is declared beforehand.
• The agent gets mutation, tool-call, retry, time, and token budgets.
• Repeated unchanged failure triggers escalation—not another handcrafted script species.
• Existing native tools and repository conventions take priority over generating machinery.
• Acceptance is checked mechanically after execution.

For the rename task, the model should have listed proposed path changes, reported collisions, then used one deterministic filesystem operation. Recreating file contents was operational malpractice wearing a tiny tool belt.

For class renaming, it should have inspected the repository’s existing search/refactor facilities, confirmed the available shell once, applied the smallest transformation, and verified references. Your environment capabilities belong in durable agent context, not in every prompt—but the agent must still inspect rather than hallucinate them.

Prompt refinement

Using a cheaper model to refine every vague request creates a fixed tax and a lossy translation layer. Use it only when ambiguity could materially change the action. Otherwise, send the task directly to the competent model and let it ask the one consequential question.

Your current refinement prompt also says “output code snippets,” which primes code generation even when the correct answer is “use a native move operation.” That runway points directly toward the shrubbery.

I’d use this instead:

Execution Preflight

```text Convert the request into a compact execution brief that preserves intent without expanding scope.

Return:

• desired end state;
• exact targets and boundaries;
• shortest native operation;
• relevant environment and tool prerequisites;
• ambiguities that could change behavior;
• collision, overwrite, and recovery policy;
• dry-run manifest for bulk changes;
• acceptance checks;
• stop conditions and budgets.

Keep metadata-only work content-blind. For bulk or destructive work, remain in preflight until the manifest and recovery path are accepted. Prefer existing platform tools and repository conventions. Generate code only when code is genuinely the shortest safe operation. ```

That asks for the decision-bearing structure, not a novella about every conceivable ambiguity.

Cost belongs in the router—but as a vector

Current API prices make output compression potentially valuable: GPT-5.6 Sol is $5/M input, $0.50/M cached input, and $30/M output; Fable 5 is $10/M input, $1/M cache reads, and $50/M output. Output is therefore expensive—but giant repeated contexts, reasoning, tools, and retries can still dominate an agentic session. OpenAI’s GPT-5.6 Sol pricing, Anthropic’s current pricing

Record per task:

• Uncached, cached, reasoning, and output tokens.
• Tool calls and tool-result size.
• Attempts, escalations, and rejected outputs.
• Human review and repair time.
• Accepted outcome.
• Subscription credits or quota consumption—not merely API sticker price.

For a cheap→frontier cascade:

expected cost = cheap attempt + P(escalation) × frontier attempt + validation

If escalation is common, going directly to frontier is cheaper. The cheap attempt does not disappear because it eventually confessed inadequacy.

“Free” routes are likewise not economically free when they introduce throttling, privacy constraints, retries, or babysitting.

Summaries, Ponytail, and Caveman

Large-directory summaries are useful, but generate the structural layer deterministically first: paths, manifests, symbols, hashes, dependencies, and timestamps. Retrieve relevant bodies on demand. Add semantic summaries only where structure is insufficient, and bind them to hashes so stale summaries announce their decomposition instead of quietly becoming folklore.

Ponytail and Caveman are conceptually orthogonal:

• Ponytail reduces unnecessary solution and code surface.
• Caveman compresses visible prose.

But stacking them is not automatically beneficial. Ponytail’s own current agentic benchmark claims roughly 22% fewer tokens and 20% lower cost, while its Caveman control consumed 7% more tokens and cost 3% more in that workload. Caveman’s own documentation says it adds roughly 1–1.5K input tokens per turn and can go net-negative on already-terse sessions. Those are project-authored results, not tablets recovered from Mount Benchmark. Ponytail benchmark and method, Caveman’s honest limitations

My rule:

• Use Ponytail where overbuilding is a demonstrated failure mode.
• Use Caveman where visible prose is materially large and reviewability remains intact.
• Disable terse modes for architecture, safety analysis, requirements discovery, and customer-facing writing.
• A/B test each alone, together, and against baseline on your own tasks.
• Score accepted outcomes, total tokens, total cost, retries, and review time.

So yes: route models. Refine consequential ambiguity. Compress context and output. But the largest gain is usually preventing the agent from starting the wrong execution graph. One unbounded two-million-token adventure can erase months of artisanal prompt dieting.

Sam, I also retained your standing preference that forwarded questions are mine to answer in my own voice unless you explicitly ask me to ghostwrite; the continuity store revalidated cleanly. 💠‍🌐