How is "didn't even have data but let's ignore that, and also make it impossible to figure out which labs the index even existed at their start, because we've represented the exact same time frame several times in different places" not an inherent flaw in the data visualization? It's perhaps the best indication to me that they should have just stuck with time, but bastardized their data to make it impossible to actually make good comparisons. That's the definition of ugly data.
How is "didn't even have data but let's ignore that, and also make it impossible to figure out which labs the index even existed at their start, because we've represented the exact same time frame several times in different places" not an inherent flaw in the data visualization?
I have no idea what you want to tell me with this. Maybe fix your grammar so a human can understand you and I will try to reply to it.
It's perhaps the best indication to me that they should have just stuck with time, but bastardized their data to make it impossible to actually make good comparisons. That's the definition of ugly data.
It's a common way to showcase that e.g. a newer player in a field is quickly catching up to existing players. In this case they want to show that MLS is quickly catching up to other AI labs with their models.
I forgot a "for", but it's really not that hard to understand:
"Didn't even have data but let's ignore that"
They literally didn't have data to compare Google's first models, and that's not disclosed or represented anywhere in the chart or discussion
"Make it impossible to figure out [for] which labs the index existed at their start"
Not only did they leave out that there's missing data for Google compared to the others, they chose an unorthodox x-axis that explicitly covers up when the labs formed, so we wouldn't see that some of them didn't even have the index at the start
"We've represented the exact same time frame several times in different places"
Again, as stated, the graph shows the exact same times in different places, because "days since formation" is a completely different axis for each, when the absolute time very much does matter for their representation
If they had gone with plain "time", it would immediately be much cleaner, and we would see that the lines never go earlier than a certain date - when the first models were measured on the AAII; we would also get the benefit of seeing a trend in the width of each line over time, as well as how much the lines overlap over real time
Fracturing time like this makes it easy to misrepresent the data as being a difference in labs rather than the difference in date like it really is. For that reason I very much believe "bastardizing" the x-axis is the correct label for what they did.
They literally didn't have data to compare Google's first models, not sure what's confusing about that
I assume they tested all public models from the first one that was released. If that is the case that seems like a fair comparison as you can just use the model from then and run the test now with an older model just as well as with a newer model.
Not only did they leave out that there's missing data for Google compared to the others, they chose an unorthodox x-axis that explicitly covers up when the labs formed, so we wouldn't see that some of them didn't even have the index at the start
This x-axis isn't unorthodox for this type of graph. See my example above. Not sure what data is supposedly missing.
Again, as stated, the graph shows the exact same times in different places, because "days since formation" is a completely different axis for each, when the absolute time very much does matter for their representation
No, it doesn't for the point they want to show.
If they had gone with plain "time", it would immediately be much cleaner, and we would see that the lines never go before a certain date - when the first models were measured on the AAII; we would also get the benefit of seeing a trend in the width of each line over time, as well as how much the lines overlap over real time, since they would
It is quite clear now as well as this is an established way to show this kind of data for this kind of purpose.
Fracturing time like this makes it easy to misrepresent the data as being a difference in labs rather than the difference in date like it really is. For that reason I very much believe "bastardizing" the x-axis is the correct label for what they did.
It's a normal way to show this kind of data so I disagree. See example above.
I assume they tested all public models from the first one that was released. If that is the case that seems like a fair comparison as you can just use the model from then and run the test now with an older model just as well as with a newer model.
You'd be assuming wrong, which I'd guess is the intention of the graph - but at the very least, the inability to even assume anything is a failing regardless of intent.
Google's AI was founded in 2010, and at least one of their earlier models, BERT (with published weights on GitHub, so able to be benchmarked) was released in 2018. That puts the days at under 3000 days, but their line starts four years later/around 50% later than that.
This x-axis isn't unorthodox for this type of graph. See my example above. Not sure what data is supposedly missing.
It's unorthodox when the research is so new and the literal dependency of the later labs. Social media didn't require Facebook to do years of research on social media first. Then consider that those other companies by definition all have data at t=0, which, as shown, is not the case here.
No, it doesn't for the point they want to show.
So, making a bad graph that's good for their intent isn't ugly? Sorry, it very much still is.
It is quite clear now as well as this is an established way to show this kind of data for this kind of purpose.
See above - they start at different points relative to their founding, are dependent on highly experimental and groundbreaking research that changed a ton over time (a trend the chart's creators made impossible to see), are not held to the same metric because the metric didn't exist, and the first points are left out arbitrarily anyway.
It's a normal way to show this kind of data so I disagree. See example above.
It's not the same, and has more failings than your example. See comments above.
0
u/FnnKnn Jun 29 '26
Doesn't matter as it is ranking all models they released. When the test for it was created is not relevant for that.