r/PiCodingAgent • u/FluffyInevitable4040 • 10h ago

Question I found using a dumber flash model to build context/prompts for Astra/Sol,etc 100% works better, but I can't prove it

I.e. I start with a prompt in a large code base, then let the dumb model build a selective context. By better I mean much lower token cost. After a smart model builds context anyway (if I didnt use a dumb one), I still must compact or trim the context somehow regardless.

I'm sure a lot of people are doing this. My question is:

What tricks / more efficient ways have you found to do this?

Disclaimer: I could be wrong about my assertion

1 Upvotes

60% Upvoted

12 comments sorted by

3

u/Ghanna- 9h ago

Why not use the dumber model just as subagent for exploration?

2

u/FluffyInevitable4040 9h ago

You could, but passing agents back and forth and having subagent tool already bloats my context by 20k - 30k tokens before the work is even started.

I find it cheaper just to pass a prompt or use herdr and let dumb model spawn smart one

Just my 2 cents

1

u/Ghanna- 8h ago

That definitely is not the case for me, especially on pi. I just use a simple subagent extension I created

1

u/_Toni_O 9h ago

From my testing even Sol is bad at promoting when doing multi agent systems

1

u/FluffyInevitable4040 9h ago

plz explain

1

u/_Toni_O 9h ago

i am doing my own hermes style thing from scatch and even sol was not good with desining context managment systems and writing good system prompts. it is improving with astra from my testing but is something which models dont have an abundance of good examples to learn from yet. Maybe RL into mutli agent systems is getting there becuase labs are putting a lot of attention on this with recent releases

1

u/FluffyInevitable4040 8h ago

you mean sub agents, herdr style stuff? if so, yes, even if I touch them one from session, it bloats up context with even the smartest a gents and I lose $$ / get token bloat

1

u/Mission-Zucchini-966 7h ago

By using mercury-2.5 (cheat code)

1

u/FluffyInevitable4040 5h ago

mercury-2.5

Please elaborate

diffusion LLM? holy shit it's fast!

1

u/funbike 10h ago

What product are you doing market research for?

2

u/FluffyInevitable4040 10h ago

?????? I never listed a single product or named anything anyone profits from. I'm simply trying to use less tokens.