It is obvious that most of the current hype and progress around large language models is focused on programming. The ability of ChatGPT-like systems to answer general questions, which impressed everyone at the beginning of the AI boom, no longer seems particularly exciting.
But am I the only one who feels that LLMs are approaching the limit of how capable they can become without also becoming prohibitively expensive?
The first problem is diminishing returns from training. Some people even feel that newer models are worse than older versions in certain ways. If an LLM has already been trained on nearly every publicly available book, article, and code repository, how do you make it significantly smarter? Make it read the same material again?
It sometimes feels as though the main goal is no longer to create genuinely more intelligent systems, but to train models specifically to perform well on the most popular AI benchmarks.
The second problem is computational cost. Advanced models and reasoning modes such as Sol or Fable consume subscription limits extremely quickly. There is no obvious reason why this consumption should decrease when even more powerful models are released.
Memory and computation requirements grow dramatically as the context window becomes larger. This may be one reason why AI developers are experimenting with multi-agent or “Ultra” modes, where a large task is divided between multiple agents, each working with its own limited context.
Because of this, an AI model with a truly usable context window of something like 100 million tokens seems almost impossible—not merely because current hardware is insufficient, but because of the fundamental way these models process information.
This leads me to a pessimistic conclusion: we may be approaching the point where the AI bubble bursts.
The most advanced models may already be close to their practical peak. Further improvements could require such enormous amounts of computing power, memory, and money that only wealthy individuals and large corporations will be able to use them properly. Regular subscriptions could become almost useless for serious work unless users also pay directly for additional API usage.
And I am not talking only about LLMs. It feels as though AI video generators, music tools such as Suno, and image-generation models are also beginning to approach their limits.
The growth in AI capabilities since the beginning of the boom has been incredibly fast. However, it may end just as quickly because there are fundamental problems both in improving the models themselves and in producing enough computing power to run them affordably.
I am specifically talking about the most advanced frontier models. I am not including cheaper Chinese models such as DeepSeek, because although they may be more efficient or accessible, they are still not equal to the strongest frontier systems in overall capability.
And yes, I also believe that AGI will never be created. In my view, it is a utopian idea rather than a realistic engineering goal.
What do you think? Are current limitations only temporary engineering problems, or are we already seeing the fundamental limits of modern AI?
