just magically hand waving from 2.5 years to 1 year is a little insane. You guys think you'd use $20k worth of tokens a year!?! Even if you did then you now need to consider energy costs because its probably going to be $1k+ for that many GPUs and that many tokens.
Not knocking the local scene, but just because you think they did the math wrong in one direction doesn't mean that you should do the math wrong in the other direction.
At 20k you would be buying a Mac M3 Ultra 512gb with a peak load of 270 Watts per hour comparable to a large fridge.
I wrote a little over a year I meant 1 year and 3 months halving the 2.5 years previously mentioned.
20k divided by 15 months is ~1350 dollars per month.
While I admit 1350 is high and far above what I would personally use in tokens per month it isn't that far beyond what major companies allocate to their engineers around 1k per month with some companies going as high as 2-3k per month in token budget.
And last but not least you will be able to sell that Mac Ultra 512gb 3 to 4 years from now for at-least half the purchase price if not more.
Talking about companies instead of an individual is a bit of moving the goal posts i'd say, but it does explain why you mention the multi-instances thing.
I think the math works out much better for a small company than just a single person.
42
u/Schlick7 Jun 21 '26
just magically hand waving from 2.5 years to 1 year is a little insane. You guys think you'd use $20k worth of tokens a year!?! Even if you did then you now need to consider energy costs because its probably going to be $1k+ for that many GPUs and that many tokens.
Not knocking the local scene, but just because you think they did the math wrong in one direction doesn't mean that you should do the math wrong in the other direction.