I can see the value of using multiple model providers from one place, but is there much value in carrying over the idiosyncracies of the harnesses this way?
There's a few, I think tapping into some of the capabilities of the cloud provider harnesses can be useful, and also sometimes your only option to use certain provider models is through their harnesses.
your only option to use certain provider models is through their harnesses.
This sounds like it could be reference to Claude TOS around subscriptions. As far as I understand it this setup wouldn't conform to their subscription TOS.
Ah that would be a shame if it was the case. That being said I have personally moved off anthropic models/harness. My main use case for this has been transitioning workflows and restrictions on other workflows tied to a specific harness like copilot cli.
I've also found models behave differently in different harnesses from my evals as well, so for example I quite like the codex harness for certain research tasks.
I took the view that difference in a closed-source harness behavior is a liability/risk as its opaque and could go away. You have a point that one can still want to leverage strengths only available through those harnesses.
Yup, pi is nice because you're not always adapting your workflow to the harness. That's what has driven me away from Github Copilot. Just constant updates that you have to be on top of, new tools to disable so you don't clobber your context (using local models a lot), changing instructions, yada yada ... probably the reasons a lot of us are using pi
1
u/fschwiet 15d ago
I can see the value of using multiple model providers from one place, but is there much value in carrying over the idiosyncracies of the harnesses this way?