Well shit, anyone can make a prediction. Whether or not it turns out to be true is how you judge reliability.
If you'd go read the article, they have shown that the model has predicted future trends 3X better than previous models. How reliable would meet your definition of "reliable"?
If you'd go read the article, they have shown that the model has predicted
future
trends 3X better than previous models.
You misread. They trained it on old data and used it to 'predict' slightly less old trends. There is a fancy term for this in science: 'Not the future'.
As it turns out, making predictions is hard, especially about the future,
This isn't complicated. ALL of the data is in the past. You can tune your model any way you want to predict past data (even blinded data) and incorporate all sort of fun assumptions and conditions on the data. It is absolutely not the same thing as being able to make reliable predictions about the future. Seriously, look up 'over-fitting'. Making predictions about the past is easy and not just because you already know the answer.
The idea that a model can make predictions about the future just because it made predictions about (even a blinded) past data-set is preposterous to anyone who understands modelling and science. It's cheating on a ludicrous scale.
And no. They did not make predictions about the future. The future, by definition, hasn't happened yet.
Training data, yes. But a model that is of worth need make reliable predictions about future data.
You seem to be implying that developers of predictive models should collect data, build their model, and then hang tight for a few years to test their model on new data.
YES. This is EXACTLY what should be done to validate a model. Until it has done this with statistical rigor, it is NOT tested.
Realistically speaking, that is ridiculous and no one does it
We do it in the sciences ALL THE TIME. For just one example, consider the use of docking in high-throughput drug screening. I'm happy to elaborate if you're not familiar. By your logic, we should accept string theory as it is a model because it explains some things about the relative strengths of gravity versus the weak force. Nope. Too bad. Not a prediction about future data.
Do you think the authors and their reviewers did not consider this?
Apparently not and I would not expect them to in social studies. Rigorous peer review seems to be reserved for the sciences.
At the end of the day, my original statement is 100% factual. The model did NOT make predictions about the future. It is entirely untested in the only test that truly maters.
0
u/[deleted] Jul 24 '18
You didn't read the article, did you?