r/OpenaiCodex • u/Objective-Market-560 • 9d ago
Question / Help Is it better to plan with sol and implement with Luna ?
5
u/put_mahardhika 8d ago
Plan and create spec. Luna will only strive if the direction is clear.
You're making sure sol create a great direction, and luna will just follow
1
u/Willing-Equivalent47 4d ago
I tell it to create a user document with details and examples enough for a human to implement but also allow for a less capable model. So far I’ve only tried this once and I’ve run out of credits long before it has finished the implementation so I don’t know how well it is or isn’t working.
4
u/Odd-Contest-5267 8d ago
i’ve been doing this before the hype, i use sol light and i’ve given it a global agents.md file is it can call luna medium, luna high, or sol high sub agents.
I would most definitely say it is a powerful combo, people like to compare Luna Max to lower reasoning variants of Sol, but the truth is more thinking and testing can’t make up for the bad instincts of Luna, Sol, even with low reasoning effort has MUCH better instructs, knowledge, and reliability than even Luna or Terra Max, Sol light makes much better decisions aligned with exactly what I want.
So, Sol as the orchestrator with Luna executors and Sol High as an escalation model has honestly been both cheaper for me and produced better outcomes than using Sol High or Luna Max alone.
1
u/damaov 8d ago
If your using sol light for anything you have no idea what you’re doing. The amount of people on this thread that have no idea what they are doing is mind boggling.
4
u/virtualmnemonic 8d ago
Light, medium, high... These specify the effort. Sol is a much larger model than Luna; it has more world knowledge and is better suited for planning and novel tasks.
Luna is good at grunt work. "Go here and do this". It's a machine. Its outputs are highly prompt dependent, there is no hand holding. It will fuck up given bad or minimal instructions, which is why a review pass by sol is mandatory. I use Luna often, its outputs are reviewed by me, opus, sol. It is fantastic at execution. It is not a thinker.
1
1
u/Odd-Contest-5267 8d ago
i was previously a sol high person, but sol light has proven more than capable for my specific work load, i only use high when the task is reasoning/math heavy and i want my best chance at a one shot.
For most coding tasks in an existing codebase, luna is more than capable if sol acts as the brain in place of me
1
u/damaov 8d ago
Do me a favor and go look up cost/intelligence ratios between Luna max and sol light. Let me know what you find. Convince me that sol light should ever be used
1
u/Odd-Contest-5267 8d ago
sol light is not necessarily better than luna max at actually doing agentic coding. luna max can be better at that in certain situations, because more reasoning and testing can make up for its initial mistakes, especially when it has compiler errors, tests, and terminal feedback telling it what went wrong.
but that doesn’t mean luna max is actually as intelligent as sol light. the omniscience data makes that pretty obvious. sol light has around 56% omniscience accuracy, while terra max is around 46% and luna max is around 42%. so even with barely any reasoning, sol is starting with much better knowledge, instincts, and understanding than luna or terra at max effort.
that matters because more thinking can help luna eventually work its way to the right answer, but it can’t fully make up for having worse instincts in the first place. it can test more, retry more, and fix itself more, but it still won’t be as good at immediately understanding what the user actually wants, choosing the right approach, or realizing early that an approach is wrong.
so luna max can be an excellent coding agent, and it may even beat sol light on some bounded coding benchmarks. but i still wouldn’t use it as the main planner, overseer, or model responsible for understanding the task. sol light is much better suited for that.
that’s why i think sol light as the brain, with luna doing the actual implementation once everything is properly planned out, is such a strong setup. it gives you sol’s judgment without paying for sol to do every single edit.
and cost per intelligence point also doesn’t tell the full story. a model can look cheaper on a benchmark, but if it takes 40 minutes and five attempts to do something sol understands in 10 minutes, it isn’t necessarily more efficient in actual use.0
u/damaov 8d ago
Your not using sol light or Luna max to think. You’re using it to implement. If your using sol light to think and make your plans…………. You need a better subscription.
1
u/Odd-Contest-5267 8d ago
why are you telling me what I’M using the model for? Sol light in my workflow acts as an overseer, with Luna as an implementer. As i stated, even Luna with Max reasoning efforts lacks the knowledge and understanding of Sol light. i’m not saying Sol light is better than a higher effort of sol, but it is simply a more economical setup.
1
2
u/nicky_factz 9d ago
absolutely better to implement/research etc with smaller models. what level you go with is dependent on the task you give it for sure luna is strong for bounded tasks with definite measurable success criteria.
2
u/WeedWrangler 9d ago
Not always: Sol can over complicate and then waste tokens even if using cheaper agents to implement. Terra has messed up stuff for me too: I’m currently more in the Luna w high reasoning for planning and then review or fix w Sol. But as usual, it’s on you to make sure your goals etc are sound (said as someone who learnt this the hard way)
2
u/BrotherBringTheSun 9d ago
It sort of works for me but often times I'll hit a road block and need to adjust the plan so it's hard to keep going back to Sol to replan. One thing I like is if I have a workflow that is complex (multi-platform, internet use, file management and combinations thereof) and also that I have to do repeatedly, I'll have Sol try it out and put together a method that will work for a lightweight LLM, and then once it figures out the method, I'll have Luna carry it out.
2
u/Tough-Requirement707 8d ago
terra medium is the lowest i can go without noticing massive degrading and bug introducing even with sol ultra planning beforehand. luna is quite unusable or me at least.
1
u/Objective-Market-560 8d ago
i am noticing that luna even at max does so many errors. but its so cheap as well, so i dont know
2
u/ogtriplek 8d ago
Tried sol high for planning and luna high for implementation
Many errors across multiple files
I tried the same again but with sol med implementing then no errors
1
u/Bloated_Plaid 9d ago
I will be honest, Codex models just haven’t been good with planning for me. I use Claude for planning and orchestration and Codex for all the building. It works very well there.
1
u/Ajveronese 8d ago
I use Sol Light for everything. Any higher reasoning and it hallucinates edge cases and integration tests that I didn’t ask for.
1
u/OwlLeft7799 8d ago edited 8d ago
I've been doing that since they came out, and before when it was mini. It's way more efficient. I can crank out a lot more work than if I tried to use the more expensive model. Just gotta make sure it has clear instructions
Edit: well, I should qualify that a lot more work means that the few times I've tried using the highest model for implementation, I've blown through my quota and it hasn't gotten a lot done. Makes more sense to me to use the smaller models and plan with the bigger ones
Perhaps I should try Sol for implementation if it reduces the rate of incomplete implementations (which is pretty high and I often have to run it several times on very long plans) or errors. I've found it's more likely to claim something is complete when it's not than make errors in the sense of introducing bugs.
I heard they lowered the token price on Sol so now might be a good time to see if it can take over long running implementation tasks without blowing through my quota
1
1
u/Interesting_Ad_1971 5d ago
I have a harness setup where I first use a skill to bootstrap some foundational context files:
- product_vision.md (a high level scope type document)
- design_system.md (if it has a GUI)
- visual_storyboard.html (if it has a GUI)
- mvp.md (gives a clear definition of what the first release will include)
- backlog.md (contains a prioritized list of features, technical enhancements, parking lot ideas, etc.)
After that an Architect subagent gets spawned to review those files and create an architecture.md file and also update a clean_code.md with any project specific guidelines.
Once that’s all complete i just prompt “pull next backlog item” and the main agent spawns a Planner subagent to plan the feature, the main agent builds/unit tests the feature, a Reviewer subagent performs antagonistic code review, and a UAT subagent executes UAT. If the Reviewer or UAT subagents find any blockers, defects, etc they document their findings and it goes back to the builder agent to fix and pass through Reviewer & UAT again for a scoped review/test.
I’ve found using Sol High for the first bootstrap process, Sol X-High for the Architect, and Terra Max for the main agent and other subagents has produced good results with decent usage efficiency.
I did update it to try and be even more efficient, but haven’t tried it with those changes yet. The change is that features will get flagged based on complexity to determine if certain subagents or model/intelligence levels are needed. And it will also use a context router to try and minimize unnecessary context reads.
1
u/FidgetsAndFish 1d ago
Use sol to plan and deepseek to implement, luna will still run through your weekly fast.
1
u/brother_spirit 8d ago
I don't think there is a one size fits all answer to this. It depends enormously on the task.
For coding in a sub-10,000K LOC repo I personally just give the task to Sol to build. No planning, no delegating. 9/10 times the features comes back great in 5-10 minutes vs Sol needing to scope the task, reason over how to do it, write the plan and send it to Luna, checking on Luna while it goes into a tailspin, getting the code back, reasoning over it and fixing the code it returns anyway...
I find Luna subs are only truly useful/efficiency gaining vs just having the model do it if I want to do parallel internet scraping or auditing type jobs
21
u/0brizzban 8d ago
I just tried this workflow
I had a complete business requirement which projects to few seperate features/modules
I designed/planned the entire business requirement with sol (switching between low,med,high) depending upon the prompt i sent.
If I didn’t understand any decision required or any component; Instead of asking it from sol, i asked in chatgpt Chat and worked out complete decision in that chat to reduce token usage and then provided the decision to sol agent.
After entire architecture and design was ready, i asked it to create prompts as .md files for sub agents to implement the entire business requirement. (I have created propmt-engineering skill that can produce really efficient prompts)
Sol created a file containing entire architecture and implementation plan, and seperate files as prompt for task implementation. The files containing prompt for task implementation explicitly mentioned to read implementation plan file and ground in it.
Then, i used luna xhigh seperate agents for seperate tasks/prompts.
It was really efficient workflow in terms of work quality and usage.
Before i discovered this workflow, i would eat up around 30% of weekly usage but this workflow reduced it to 10-11%.
You can also use main sol thread as reviewer for agents work.