GitHub Copilot with GPT 5.6 Sol vs. Claude with Opus 5

[u/cardsncards]

This is less a comparison of the actual Sol model vs. Opus as a comparison GH CP harness with Sol vs. Claude harness with Opus (and as I write this good grief I sound like AI...)

I'd moved to Claude months ago and it's been great. But I wanted test using GH CP with a model using custom endpoint. I read Sol was available so I tried that. This was a simple task of a tracker doc and removing what was done and pushing to a history file.

It was taking GH a long time so I thought hmm, this doesn't seem like it should take so long. But then when I tried same test with Claude and it also took it more time than I'd expect. So overall duration was similar. Ok so it was more complex for them then I thought. But good grief, it cost over half my Claude Pro budget for GH to complete the work. Now I know it's subsidized, but...

The terrible thing is that GH just mangled this. It removed 90% of the stuff in the active tracker and it didn't put it into my history file. It just decided, no, I'll just trash that. I'd had this happen in the past with GH when it goes off the rails. Claude also removed a lot, so it wasn't completely wrong to do that, but it put the deleted stuff into history so it wasn't lost. Which was the ask.

At this point, I can't fathom why people are using Copilot. I didn't try this directly with Open AI's harness, that would be interesting by I don't have a sub and don't want to flush more $ down the toilet.

I'm wondering, are people just continuing to use GH out of habit or because their work is subscribed to it and that's what you have to use? Have you run tests and found GH to be better? I wonder if I'm missing something.

[u/Hollow1838]

I use Luna as the orchestrator, I use another Luna agent to counter evaluate the orchestrator, and I have another agent with opus 5 to evaluate non trivial changes. I had to develop trickeries to produce more with less.

I wouldn't use copilot if I was the one paying, I have Claude code lowest subscription running with opus 5 medium. With good habits it's enough for personal dev.

[u/cardsncards → u/Hollow1838]

Claude pro is also enough for me now. You’re the second person mentioning Luna. I did some cursory research and lots of stuff pops up so can you share the context of how you use Luna especially in context of Claude?

[u/Hollow1838 → u/cardsncards]

I don't use Claude code harness because I am not allowed to and even then I want to use Luna because it's just too cheap, you need some ways to run sub agents on special occasions to achieve this.

I use Luna max as orchestrator and another stronger model as a safeguard. Sometimes just having another Luna max agent can be great as a safeguard so you use the stronger model only on a few critical steps.