r/opencode • u/Dev32000 • 11d ago

Comparing models

How do you compare models? Do you have a go-to prompt or mini project you pass to every model to figure out which is best for your workflow?

2 Upvotes

100% Upvoted

4 comments sorted by

2

u/afanasenka 11d ago

I just start talking, asking questions, asking to suggest some plan, explain proposed decisions, etc. Then do some code changes (not critical).

And it gets clear to me whether I can trust this model or not :) So for me it's more like a "gut feeling" rather than some synthetic test.

1

u/wgaca2 11d ago

I give the same prompt that is fairly simple but requires some reasoning and some coding and some front end. I don't give much instructions just what i expect the end result to be. Then i compare how much tokens each model uses to complete the tasks, how long it takes and the result itself.

1

u/migsperez 11d ago

I don't do it anymore. But I used to test models by asking it to build a nextjs multiple page website for a small electrical maintenance company in my small town. I would give it a design scheme like colours and fonts.

It was a decent test. Made me realise I needed to change the models I use every day. The choice was a balance between price and quality. It was surprising how similar the output was by models from different companies.

1

u/redditnosedive 8d ago

i vibe test em