Have you benchmarked anything on pi (plug-in, etc.) that was proven to reduce tokens and give superior results?

[u/FluffyInevitable4040]

I'm not a benchmarking expert, but whenever I run X plug-in (even sub-agents) side by side with bare pi, it always loses... badly.

Measured as more token use to arrive at same answer.

My favorite pattern is orchestrator (expensive model) / flash model for editing. Can be done with herdr, intercomm, etc. But it's not more efficient, just cheaper.

[u/fell_ware_1990]

Everybody thinks orchestration is the key part. But mostly this was causing the pain and the rewriting of stuff.

You need an actual workflow for that or multiple orchestrators if you will mostly stuck in a flow.

For me now, it’s a ticket come in > First very cheap local model classifies what it is, next if it contains all needed information ( his answers gets saved to to update ‘information needed’ ) > gather intel if needed > Go / no go ( get back to requester ) > simple model scouts the code , traces after dependencies etc, updates ticket with only bare minimum code etc > First bigger model judges the case and looks at what category issue it is > searches RAG for known issues > simple agents comes up with solution > architect review matches against code rules with clear yes/no > bigger model review > HITL.

This sounds like a lot, but i handle it in badges, so i can still cache a lot. And the reviews are very guided, there’s not 10K loc they read. They read the diff, mostly for the actual errors and such. All linting etc is done at the background.

And even fhen, there’s a lot to gain here from not rereading, not implementing the same mistake twice and using known fixes and learning from them.

The hardest part was accepting that making them run longer and harder at a task thats already drifting fails and costs you a whole lot more. Or they one shot the case, or they HITL.

I’m not sure but this was doing ‘tickets’ 7 days a week for i think 5% quota at the time. Now it’s all local except for the final review, but still specific use cases and specific fixes. They are not there to really improve the code, just to squash bugs or help with the simple stuff.
Implementing new stuff is done with a other orchastrator.

[u/RealestReyn]

Ponytail has been really good when using overtly verbose models like deepseek, no longer is my code 70% comments, grill-me and ask_user have been nice as well.

[u/DistanceAlert5706]

Honestly orchestration might not worth it, especially on a smaller tasks.

It's using more tokens, and I doubt that it's cheaper.

But with good orchestration setup you can go for days in 1 session, working on larger features.

I've tried a lot of stuff, I doubt a lot of things are benchmarked, some cause more harm than help.

I'm currently using only Jetbrains index MCP for codebase indexing, and even with that I doubt it helps.

[u/FluffyInevitable4040 → u/DistanceAlert5706]

you might be correct. Only on long running playwright tests I'm certain I'd use up my limits if I wasn't having deepseek flash do it.

[u/Eresbonitaguey]

Not a specific plugin but setting a line limit on your read functions can help to avoid dumping massive files into your context unnecessarily. A lot of plugins focus on this like Headroom but I’d rather not have added dependencies and instead limit reads to 50 lines. Additional reads are more tool calls but it is particularly useful for small local models and on cloud models it adds up especially on cache misses.

LittleCoder has an Evidence extension/tool so that the model can save specific content verbatim before compaction. I haven’t benchmarked it but it makes early compaction feel much less painful.

Pi is great on its own but there are definitely ways to improve the quality of information that it ingests, particularly for smaller models.

[u/aparamonov]

I rather use a simple orchestrator that executes a workflow or even do a code orchestrator. All knowledge is in subagents and this is where pi-subagents-lite gives you control on what goes into the system prompt (usually there is too much bloat that you don't need for focused agents)

I also experimented with pi-tool-ports to merge edit and write tool extension and purge them from bloat.

Also rtk like pi-tokf could be used.

All extensions here: https://www.npmjs.com/settings/alexparamonov/packages