"Magnitude based-inference”: A statistical review

The presenter of this AIS Smart Talk gave an overview of the recently published (ahead of print) paper, entitled the same as this presentation, in Medicine and Science in sport and Exercise (MSSE). Over the last decade, “magnitude-based inference” has been developed and promoted in sport science as a new method of analysing data, however it has so far attracted little scrutiny by statisticians. The authors consider "magnitude-based inference" and its interpretation by examining in detail its use in the problem of comparing two means.

They extract from the spreadsheets which are provided to users of the analysis (http://www.sportsci.org/) a precise description of how "magnitude-based inference" is implemented. They compared the implemented version of the method with general descriptions of it and interpret the method in familiar statistical terms.

They show that "magnitude-based inference" is not a progressive improvement on modern statistics. The additional probabilities introduced are not directly related to the confidence interval but rather are interpretable either as p-values for two different, nonstandard tests (for different null hypotheses) or as approximate Bayesian calculations which also lead to a type of test. The authors also discussed sample size calculations associated with "magnitude-based inference" and show that the substantial reduction in sample sizes claimed for the method (30% of the sample size obtained from standard frequentist calculations) is not justifiable so the sample size calculations should not be used. Rather than use "magnitude-based inference", a better solution is to be realistic about the limitations of the data and use either confidence intervals or a fully Bayesian analysis.

Biography

Professor Alan Welsh obtained a BSc with a University Medal in Mathematical Statistics from the University of Sydney in 1982 and obtained a PhD from the Australian National University (ANU) in 1985. He was an Assistant Professor at the University of Chicago in the USA from 1984 to 1987 before he became a lecturer at the ANU. He held the Chair of Statistics at the University of Southampton in the UK from 2001 to 2003, and is currently at the ANU in the Mathematical Sciences Institute. He was awarded the Moran Medal in 1990 and is a Fellow of the Australian Academy of Science, the Institute of Mathematical Statistics and The American Statistical Association. Professor Welsh has broad interests within statistics including Statistical Inference, Statistical Modelling, Robustness, Nonparametric methods, and Analysis of Sample Surveys.

Dr Emma Knight joined the AIS in March 2012. At the AIS, Emma enjoys collaborating with researchers to ensure appropriate and informative statistical practice. Previously Emma worked as a statistician at the Australian National University and the Bureau of Rural Sciences in the Department of Agriculture, Fisheries and Forestry. She has also lectured at Charles Sturt University. Emma has a BSc (1997) (First Class Honours, Statistics, 1998) and PhD (2008) from the University of Adelaide.