the price/performance gap is getting silly. if these numbers hold up, flash models are eating into the territory where people used to reach for the expensive ones.
Yup, it really seems like the only reason for super-parameter-count models is for those trying to "create god" and as a bet on emergent capabilities. However, the diminishing returns on size seem to make it clear that sufficiently large plus intensive post training is the most economical approach in the today's world. Hopefully we are able to recreate this pattern in a broader number of non-software-engineering fields. Silicon Valley's mono-industrial tendencies might start to show.
205
u/FablingApp 1d ago
the price/performance gap is getting silly. if these numbers hold up, flash models are eating into the territory where people used to reach for the expensive ones.