There are two problems with this article. First is that the "evidence" is from a scatter plot graph, with no statistical measurements attached to it. Stats really aren't something that you can rely on your eyes on. To me, it looks like a weak, positive correlation between previous year's score and current years score. The thing to look for is greater densities in the NE and SW quadrants than the NW and SE. Having a zero-mean "blob" in the data makes it tricky on the eye, but NE and SW points are the ones the model "correctly" predicts, while the NW and SE plots are the ones the models "gets wrong". I think the issue is the eye is looking for a line to process a slope of, but such data doesn't streach into a line. What it should look for is: "given a good score last year, is a good score given this year" or "given a bad score last year, is a bad score given again".
The second issue is that the author does not offer a better solution to the problem. Some information is almost always better than no informantion in decision making. Statistics in human-based samples are always tricky, and its tough to get strong correlations easily. As an aside, this is part of why clinical trials are so tricky and expensive.
(I am not a stats professional, but am a biomedical research scientist with some training in human-population based stats).
Exactly. Charts are useful for eyeballing, but only the use of proper statistical tools is valid for showing trends. Charts can easily be very misleading.
There seem to be a lot of comments about the charts on this page. Arguably basic statistics should be a part of every education - is that happening in the USA?
Can't comment on USA - in Canada we had an intro in high school then about a 1/2 course-credit module per year from second year until grad school in the engineering programs.
The second issue is that the author does not offer a better solution to the problem. Some information is almost always better than no informantion in decision making. Statistics in human-based samples are always tricky, and its tough to get strong correlations easily. As an aside, this is part of why clinical trials are so tricky and expensive.
(I am not a stats professional, but am a biomedical research scientist with some training in human-population based stats).