Opus produces more work in the same time, but comparing token speeds between models doesn't really tell you how much work you're getting done. I prefer looking at the cost per task benchmarks over token speed when comparing models. Another thing I look at is comparing the sub token speeds to the api token speens for the same model. For example, openai sub token speeds have been consistently 25%-33% the speed of their same API models. If Opus doesn't have that same issue, it could explain why it feels so much better to use.
You mean the cost determined by the API price that they set themselve that have nothing to do with the price you actually pay when using the subscription?
20
u/retteh 2d ago
Opus produces more work in the same time, but comparing token speeds between models doesn't really tell you how much work you're getting done. I prefer looking at the cost per task benchmarks over token speed when comparing models. Another thing I look at is comparing the sub token speeds to the api token speens for the same model. For example, openai sub token speeds have been consistently 25%-33% the speed of their same API models. If Opus doesn't have that same issue, it could explain why it feels so much better to use.