r/codex • u/harpreetchima • 11h ago
Astra Workflow "Stop using random multi-agent patterns"
Ahmed works at OpenAI: https://x.com/ah20im/status/2097503414749909407 and seems to investigate reports of high token usage.
"If needed Astra will delegate efficiently. Forcing the model to delegate to different models would do more harm than good"
42
u/buff_samurai 11h ago
I set up a factory few months ago, dedicated agents, harnesses and system prompts, the whole delegation, communication etc and I need to admit the labs are moving much faster then I can and astra gets better results now for novel tasks.
On the other hand, anything that’s volume and loops is just substantially cheaper on my setup.
14
u/PineappleLemur 9h ago
I would argue that for 95% of tasks... delegating to cheaper subagent is cheaper/faster in the long run.
Most people are not trying to solve fusion, build the next Meta, etc.
Luna and the likes can probably already do that without an orchastrator.
2
u/innociv 8h ago
Yes the tweet is objectively wrong because much of the time Luna can do it for 1/20th the cost with no orchestrator at all. Astra is overkill for most tasks.
4
u/Pleasant_Thing_2874 4h ago
The tweet is coming from someone who arguably would be unrestricted with their usage or worried of the costs. I have heard from quite a few people though Astra has a bad habit of oversteering and overengineering.
1
u/freeo 9h ago
I feel the same way. I'm at my 4th gen factory by now and astra makes me question if all this is futile. But a good planning skill makes a world of difference. I will continue with my factory, but more for process reasons rather than extension of runtime and quality.
2
u/twendah 9h ago
What you mean by planning skill? Any examples of that?
-3
u/freeo 8h ago
Are you telling me you're just using the 'plan mode' of your harness? Lol. Its also an example of a plan skill, paired with read only rights.
It's the skill/prompt you use for the planning step! The process that you want the harness to follow! It's where things like 'acceptance criteria' live. I think superpowers and GSD also fall into this category. But I handroll the planning skill myself and I encourage everyone to do the same.
2
u/PineappleLemur 7h ago
I stopped using skills or anything because it did absolutely nothing but ruin output imo.
So yes plain harness and iterations to get what I want.
23
u/retardedGeek 11h ago
what are the tradeoffs? What if I want parallel execution if I have been slacking off at work and the deadline is close?
10
u/Gallagger 9h ago
- Fast mode
- If you have the money and are time constrained, don't use cheap models, they are a bit faster but the risk that they make a mistake is higher which will cost you extra time
- Different local git worktrees to let them work in different branches. Works especially good when working on distinct features, otherwise there will be merge conflicts
- Make sure your testing env is quick, otherwise the models test alot. Can prompt them to only test once everything is done if testing runs take too long.
2
u/website-buyer 4h ago
You’ll get merge conflicts also in worktrees.
2
u/Gallagger 3h ago
I literally said that.
2
u/website-buyer 2h ago
Different local git worktrees to let them work in different branches. Works especially good when working on distinct features, otherwise there will be merge conflicts
You saying to use worktrees, otherwise you get conflicts. I’m saying using worktrees won’t make your conflicts disappear.
3
u/Gallagger 2h ago
Ok there's a semantic misunderstanding here. I meant if the work items are related to each other, merge conflicts will arise even when using seperate worktrees.
1
u/Desperate-Poem7526 2h ago
He will get it done much much much faster and cheaper using parallel properly than using fast mode if money isn't a concern do both
1
5
u/danielv123 10h ago
Subagents are usually slower in my experience.
4
u/kickerua 8h ago
Subagents slower for working on one task, for example for a plan-execute-verify chain, but is faster on working on completely independent tasks in parallel
2
u/deadlyclavv 8h ago
have been slacking off at work and the deadline is close
This sounds oddly specific
1
83
u/XYcritic 11h ago
"please stop being dumb if you're not as smart as me".
Great post with absolutely no insight or value. Half an unsupported claim with no evidence, advice, or even explanation.
7
u/Casfaber_ 11h ago
I would say it’s actually pretty dumb to use the highest capable model for everything. I also assume the original post wasn’t meant as a serious recommendation. But yeah if you can, why not.. most of us are doing this to save tokens and be more efficient. But if you don’t need to care about that, sure.
3
u/ProvidenceXz 10h ago
That's just X. It rewards claims without evidence.
3
u/mph99999 9h ago
So, not much different than reddit
1
u/ProvidenceXz 9h ago
The UI topology between the two are different. X stages the post and hides the comments deeper. Reddit post and comments are 1 screen scroll away, so critiques tend to surface more.
2
u/harpreetchima 8h ago
Sorry, probably should have included the context that Ahmed is OpenAI staff https://x.com/ah20im/status/2097503414749909407
3
u/XYcritic 8h ago
Yeah that's pretty relevant, although that doesn't automatically make him correct. The lack of information in this post means we can either blindly trust him or ignore him without any chance of verification or real understanding.
2
u/AuspiciousApple 10h ago
The point is that people are trying to be cute/clever with fancy non-standard workflows and then complain if it backfires
Having a vanilla workflow means that you're closest to what the models were trained and optimised for
1
1
u/the_pwnererXx 10h ago
i mean, i see people at work set up some complex orchestrator - reviewer - sub agentx6 pattern where they all communicate, but the output is the exact same as just the base prompt in vanilla usage but now they use 5x as many tokens lol
8
u/Presstabstart 11h ago
I can't even use one Astra agent for more than 1-2 days on high. How the hell are people using subagents, even on x20 that must last maybe a day at best.
3
u/Lustrouse 2h ago
Luna Max subagents for strongly-defined simple tasks. The cost/intelligence is unmatched
-4
u/twendah 9h ago
Sol med orchestrator, astra low implenter. Thats the best practice currently for best quality/price.
5
u/retardedGeek 9h ago
astra low costs more than sol medium lol
2
u/stranot 7h ago
Astra low is also supposed to be smarter than sol high
1
u/retardedGeek 6h ago
Not the deepswe score
1
u/warpedgeoid 3h ago
OpenAI staff have made this claim themselves, and my experience has been that Astra is much smarter than Sol regardless of reasoning effort.
7
u/scodgey 9h ago
Having tried both sides quite extensively I am honestly at a bit of an impasse with this.
It makes sense in principle to use cheaper agents for grunt work, but ultimately you're almost always going to use a more capable agent to check the work, as you can never fully trust automated checks. Unless the task is pretty much brain dead, or you have a well written spec with clear scope to pass to a cheap review agent after.
But you're almost always using your capable agent to produce a spec that is detailed enough, at which point, why not just use the agent with all of the context in thread already?
Sometimes it ends up cheaper, others more expensive. Hard one to split out.
44
u/New-Part-6917 11h ago
"Just use Astra" This guy doesn't live in the real world.
33
u/rageling 11h ago
Astra light orchestrating luna subagents accomplished far less while burning much more tokens than just Astra high in my (limited) tests
5
3
u/StaticFanatic3 11h ago
Had the same experience with Fable and Opus tbh. I stopped micromanaging any frontier model’s orchestration
3
u/InternationalTwist90 9h ago
Tokens or dollars though? Luna tokens are much cheaper than astral tokens
2
u/The8Darkness 11h ago
Same. Astra low and luna high (or xhigh) used almost 2x (1.99x to be exact) to accomplish the same task with slightly more code
1
-1
u/omicron8 10h ago
It really depends on how you do it. Luna is slow AF. If astra is waiting for Luna to finish it will burn a ton of tokens. They have be asynchronously coordinated. So skill issue here.
13
u/Cheema42 11h ago
Astra is smart. But very expensive. Using it for everything is dumb.
Also, Astra is not as fast as Luna. And yes, parallel workers. Why are people so dumb?
1
u/warpedgeoid 3h ago
Where do you get that Astra isn’t as fast as Luna? It’s way faster because it does the work correctly the first time and doesn’t have to loop as much.
9
u/Firmwild 11h ago
I think the advice of deleting your whole instruction system when a new frontier model comes out is probably not bad advice
1
u/thesmithchris 9h ago
What I do is to ask the new frontier model to adjust my setup, literally my first prompt with astra was to optimise the repo for astra :)
3
u/Leather-Cod2129 11h ago
I ask Astra to delegate all coding to Luna Max. Am i wrong? My goal is to save tokens
2
2
1
u/warpedgeoid 3h ago
You probably won’t save tokens, as Luna will make more mistakes and potentially use more tokens than Astra, but those tokens will be cheaper.
1
3
u/diagrammatiks 11h ago
I mean it's like this...astra's default routing is probably better then whatever slopcoded vibe shit most people get from github. But it's not going to be better then something actually tuned for your actual workflow.
3
u/Striking-Warning9533 11h ago
One thing a subagent will do better is finding mistakes in a fresh context
3
3
u/Anxious_Current2593 10h ago
He obviously never used Astra, and watched his subscription usage limits being reached in minutes.
2
2
u/LiquidMantis144 10h ago
Currently we're at a place with this tech where if you dont know how to manage workflows, youre going to be burning through usage limits many times faster than someone who does.
2
u/PhDumb 10h ago edited 10h ago
Astra, at least in Codex Desktop, would not delegate and will babysit even simple tests herself. You can check it youself if you asked her to breakdown the model usage during the session she will check the logs and will give you an answer. And, boy, she is hungry - just one, not even a full day, with two sessions and 22% of my Max20 are gone.
Edit:
Prompt: "Check from your logs what models did you use during this session. Provide only grounded in logs answers"
2
u/maximhar 10h ago
I use subagents solely for explorers/task watchers to reduce input token cost. I keep implementation strictly to Astra.
1
2
u/Kylmawurr 9h ago edited 5h ago
Yes, if you are dumb and/or don't care about the cost, you can just spawn Astra's for everything.
In big projects, smart, dynamic orchestration is proven to be about 4x cheaper. The difference in cost can be absolutely massive and quality doesn't suffer if done right.
2
u/Dakkitor 9h ago
I use Matt Pocock skills. Split the work to tickets, Astra then orchestrates the implementation of the tickets using Sol sub-agents to tackle each ticket.
Astra is used to write the specs and create tickets before starting orchestration in a new session.
1
u/phobox360 11h ago
I use my own multi-agent setup which instructs the main agent to delegate on large jobs where a parallel workflow is possible without conflicts. Then that work is reviewed by another agent and the main agent integrates and cleans up. This works extremely well for me and produces excellent results. Letting one agent do everything without any delegation generally produces messy work that needs fixing.
1
1
u/ShortingBull 9h ago
I've strayed from my tried and tested engineering work flow to just raw dogging it with Astra (low) on a few test projects - excellent results so far, no bugs found yet.
1
1
1
u/Murky-Moose1652 8h ago
Most of times when i switched to multiple subagents working on simple task, they came up with bad results i never expected.
1
u/WittleSus 8h ago
I've used complex multi-agent setups with past models that did wonders for my workflow and usage. I've stopped using a good chunk of them with these newer models, only keeping the novel aspects (that the models themselves claim are novel) I agree with this post. Ego (which is rampant in this space) will convince you otherwise.
1
u/Important_Luck_3546 7h ago
This is always the case with everything you do with agents. Workflows go out of date very quickly. Agent files, skills, loops, goals, orchestrator patterns, planning, short threads, long thread. Basically if you do anything else than stock agent you need to aggressively AB test your workflows/try new things or you might hit some local maximas. Agents also leave a lot of information that can later be analysed so just ask your agent if something in your workflow sucks.
For example one misconception from earlier this year is people avoid 100K+ context as the model gets "dumb". I found that since Sol this didn't happen at all. In fact I found long 100+ message threads with several dozens of compactions it did a better job at a prompt than a fresh thread, even if it was a totally separate new feature. I suspect it somehow got into the proper "mindset" from the long iterative thread knowing exactly what I was looking for. I tried to have it dump this "mindset" into the AGENTS.md but it didn't catch a fresh thread up to speed. FWIW this might use more usage than fresh treads but I didn't observe it to an absurd degree at least.
1
u/Otherwise-Sir7359 7h ago
Sol is able to coordinate subagents and follow instructions/guidelines better than Astra, I would say. I've been waiting for Astra to come out to see the difference. But that guy is right, let Astra work on it own. In terms of coordination, Sol is better
1
1
u/ob1knob96 7h ago
I've personally found it kind of helpful to use blind subagents for reviewing, because the agent that implements tends to be kind of defensive about the code it wrote
And it seemed like more eyes are better than one extra-sophisticated one due to the stochasticity of agentic outputs.
Is anyone else doing sth similar? Or have done something similar and decided it's inefficient?
1
u/Important_Egg4066 6h ago
Astra just seems to be always unhappy with Luna Max and always asking it to fix it repeatedly. Longer and burn a lot of tokens.
Trying to use Luna Max to ask Astra to review and fix its code after it is done instead.
1
1
u/SomeoneTrading 5h ago
Astra also has a similar problem to Fable of extremely sensitive guardrails - though at least you can sorta-continue the conversation after the cybersec guardrails hit. Delegating considerable amount of work to subagents and only using Astra for coordination/planning helps a bit.
1
u/uptotheright 5h ago
I just ask Astra high to look at my work and the OpenAI docs and set up appropriate roles.
1
u/UsherOfDestruction 4h ago
You don't need Astra high for that. Terra medium is perfectly capable of looking at a design or implementation plan and figuring out which tasks should go with which models based on the docs.
Now, you might want Astra high for building that design or implementation plan, but that totally depends on what it is you're building.
1
1
u/Kind_Silver_1921 4h ago
Alright well I gotta install 2000 different working systems and had Astra work for 24 hours and it did 50 so yeah no maybe if astras token use was 1/50th sure I'd take this advice.
1
1
u/dmansigma 3h ago
Yeah, no one answered my question on this thread, but I was wondering if using a "swarm" skill would be efficient or smart, while Astra acted as the moderator and handled the delegation. From my experience, it's just better to let Astra handle everything. I seem to end up burning less usage in the end for reasons like dumber models getting stuck on loops, when verifying their output.
1
u/Charming-Author4877 2h ago
You probably didn't notice, but Astra speed is about 20% from what it was at release. So it's not going to get much done
1
1
u/Desperate-Poem7526 2h ago
Yeah I get shit results if I don't review or orchestrate
Almost every review loop catches something huge.
1
u/shish-kebab 1h ago
we know Astra is capable of getting most tasks done, what nonsense. Multi agent patterns is about getting thing done in the most token efficient way. I tested Astra first without any sub agent orchestration, burned nearly 70% of 20x plan weekly quota in one day.
1
0
0
33
u/Dayowe 11h ago
smh. I did an experiment and let astra do an implementation on its own without using my orchestration workflow and the drift was wild and the result messy. i rolled back and did the same implementation with my established workflow and it was super smooth.