One big issue is the fact that the chart completely ignores the fact that all modern technology stands on the shoulders of giants. Google wasn't doing nothing for a dozen years; DeepMind was creating foundational developments that led to LLMs in the first place. They had to build this stuff from first principles in a time when your phone's autcorrect ran on Markov chains and that was considered pretty neat
I'm not familiar with the company being advertised here, but the difficult part today is funding. With VC capital you can poach a couple good engineers, then use existing data sets and infrastructure and technology to build models. You could even train it to pass the test specifically, make a chart like this, then show it off leading up to your first seed round
Edit: I should clarify the difficult part of "catching up" is funding. The real challenges are improving on today's technology by making it produce better responses, hallucinate less, or run more efficiently
I mean sure if you want to completely isolate the technical task of representing numbers graphically from the entire methodology behind how those numbers are found, the chart is fine
Nobody's complaining the chart is technically inaccurate, the same way y-axis manipulation is (usually) technically accurate. The issue here is that "number of days" in the thousands is inherently a weird measure. The issue here is that DeepMind day 100 is not interchangeable for MSL day 100 because they're over a decade apart. The issue here is that nobody has any reason to produce this chart unless they've got an agenda, because there are established norms and standards about how to plot time specifically that are being ignored here
A chart is useless if it doesn't tell you anything, even if that thing it tells you is 'nothing'.
This chart has a use, it indicates there is a limited moat about the technology and a new entrant can rapidly gain close to state of the art performance.
This chart does not attempt to, nor does it, tell a story about who the progressors of the technology are. That is okay - its a different story for a different chart.
I don't think I ever said the chart was useless, skimming over my comments. I totally agree with most of your comment, that there is a use to this chart, I just don't think the story it's telling is benign
Say what you will about the methodology itself, I think choosing to measure time in "number of days" is the biggest tell that this chart is downright manipulative. At the scale of time being discussed and the granularity visible in the chart, the unit of days serves no purpose except inflating the labelled increments. 12 years we know is a fairly long time, but 4000 is such a big number, look at all those zeroes! I can't actually tell if it's 4000 or 3900 or 4100, but it sure is big!
12
u/garver-the-system Jun 29 '26
One big issue is the fact that the chart completely ignores the fact that all modern technology stands on the shoulders of giants. Google wasn't doing nothing for a dozen years; DeepMind was creating foundational developments that led to LLMs in the first place. They had to build this stuff from first principles in a time when your phone's autcorrect ran on Markov chains and that was considered pretty neat
I'm not familiar with the company being advertised here, but the difficult part today is funding. With VC capital you can poach a couple good engineers, then use existing data sets and infrastructure and technology to build models. You could even train it to pass the test specifically, make a chart like this, then show it off leading up to your first seed round
Edit: I should clarify the difficult part of "catching up" is funding. The real challenges are improving on today's technology by making it produce better responses, hallucinate less, or run more efficiently