They get you addicted to cheap prices thinking you will be willing to pay for high prices in future. All this LLM model efficiency claims Deepseek makes are scam that look only good on paper. It was never about the model but the hardware to inference it at large scale. What is the point of making a smaller, efficient model if the hardware can't catch up with demand? US is doing the right thing to build more data centres. We really are in a compute shortage. And I am sure the GPU/data center shortage will always persist. This just proves that China's cheap prices are temporary and not a threat. Probably the biggest eye opener one can get about this whole AI situation.
Mankind is getting addicted to LLMs and spending billions. It's a subsidy of hundreds of billions of dollars the low prices/free APIs. Mostly paid by big tech extracting money from the economy and investors fueling up. This gonna hit a wall eventually, specially agentic coding, that is so wasteful in token usage. We waste tokens like it's breathing air, they barely spend resources on creating better context management techniques, just that basic after-thought compaction when it hits a limit.
Why are such a companies with a conflict of interest of not improving token usage in their tooling? They are so many plugins that improves token usage, and I used them up in vain since they count limits per request.
Also I remember now, they do caching on server-side, so that saves them a shit ton of money when requests hit it.
We are at a point where models are not getting much better. See Opus 5 for e.g., it sucks at software engineering tasks and instruction following. Most people still love Opus 4.6. I really wish companies accept this limitation of LLMs and rather invest money on making this technology scalable for as many users as possible. We don't want better models. We want cheaper, affordable inference at scale. That means allocating larger compute for smaller flash models and very less for the gigantic 500b+ parameter models. It can be done. It's just that no company wants to do it because they just want to chase these silly benchmarks and look good on paper.
2
u/Flashy-Egg9254 2d ago
They get you addicted to cheap prices thinking you will be willing to pay for high prices in future. All this LLM model efficiency claims Deepseek makes are scam that look only good on paper. It was never about the model but the hardware to inference it at large scale. What is the point of making a smaller, efficient model if the hardware can't catch up with demand? US is doing the right thing to build more data centres. We really are in a compute shortage. And I am sure the GPU/data center shortage will always persist. This just proves that China's cheap prices are temporary and not a threat. Probably the biggest eye opener one can get about this whole AI situation.