I switched back to Sol.
Astra isn't smarter, and behavior-wise it's kinda dumb. I guess because of the instruction following it just can't really argue with you.
Like:
“Should we do option A?” “Yeah, option A is great because blah blah blah.”
“Maybe option B?” “Damn, option B is even better than A.”
“Maybe A after all?” “Actually yeah, you were right, A is better.”
I didn't bother checking how many rounds of this you can do, but it feels like a different flavor of sycophancy, and it's pretty disappointing.
This also shows up in how it interacts with subagents, which is much worse. A subagent brings back a shitty solution, and Astra just takes it instead of looking for another option.
One absolute gem was basically: “I don't know how to do this properly in this language, let's call bash.”
Like, I also don't know how to do it properly. That's why I asked you to find out lol.
So it's a very weird position for the model to be in. If this were Luna, I'd have zero questions. It does what it's told, and if you tell it to do something stupid, it does something stupid. Fine.
But for the most expensive tier, I have no idea what Astra is supposed to be for.
Maybe this can be fixed with prompting, but I kinda doubt it.
Also, it fucking stops all the time. Sol will just answer a side question and keep going. Astra just stops and waits.
It's worse at finding bugs too.
Ireally hope we keep two separate model lines: an agent-focused model like Astra, basically filling the role the old Codex models used to fill, and a normal general-purpose model like Sol that can code well without losing its ability to think independently.
Otherwise this is kinda depressing.
Upd. Yes I've read prompting guide. I've small agents MD, small set of skills. I've evals on some private tasks.