A slippery slope

Statistics in the geosciences

I signed Bob Kopp’s letter asking for the retraction of Voortman & de Vos (2025). I think retractions should be reserved for papers that cannot be reproduced or are fraudulent. I recently decided not to retract a paper even though the authors were unable to reproduce their results. I do not think Voortman & de Vos are dishonest and they probably have code to replicate their paper. I think the paper is badly done.

Retraction would be too harsh.* Kopp seems just as upset about the bad publicity as about the bad science. So, why did I sign? First of all, I expect the editors to invite us to write a comment instead. Second, strictly, the Voortman & de Vos were dishonest: They claim to share their code, but they do not. Third, the statistical analysis is really terrible.

Retracting a paper for incompetent statistics is a slippery slope, one I am prepared to glide down. The standard of statistical analysis in the geosciences is typically abysmal. I have yet to meet a geoscientist who knows Haavelmo (1944). More down to Earth, Yule (1927) and Engle & Granger (1987) appear to have passed them by.

In the case of Voortman & de Vos, the applicable test for a trend break is Zivot & Andrews (1992). Instead, they correct for autocorrelation following Bence (1992), apparently oblivious to the bias due to higher-order autocorrelation and the singularity due to a unit root.

So, part of me hopes against hope that calling out Voortman & de Vos will lead to some reflection on statistical standards in the geosciences. For instance, the recent paper by Wang et al. (2025) would not pass as an undergraduate thesis in econometrics: The estimator is inefficient, unnecessarily so, and key tests are missing. It is not enough to test a linear trend against no trend. You need to test trend stationarity against non-stationarity. Wang study sea level acceleration and should have tested their non-linear trend against second-order integration and against trend stationarity with one or more breaks. They did not do this so their results are inconclusive.

*UPDATE: The special issue editors have disowned the paper.