Most journals require a statistical analysis section, in the spirit of affording your colleagues the opportunity to reproduce your results. The first part of the methods section describes how you obtained your data. The statistical analysis section describes how you manipulated those data to get your results.
Kids, don’t try this at home!
Unless you are a statistician, you should probably collaborate with one before writing your paper. You should certainly consult a statistician before writing your statistical methods section. Choice of appropriate statistical tests is essential to writing an excellent paper. Choosing the wrong statistical analyses can mean your article is rejected and you are sent back to the proverbial drawing board. There’s no substitute for expertise.
Anatomy of a statistical analysis section
The format of the statistical analysis section is fairly standard. Readers expect to encounter information in a particular order. Whereas a certain amount of flexibility is tolerable, it is best to stick with the formula (no pun intended).
Begin by stating what kinds of data do you have, categorical or numerical. Then explain how you expressed those data. For example: ‘Weights of widgets were expressed as means ± standard deviation. Categorical data (widget class) were expressed as frequencies’.
Next, you should describe the tests for normality that you performed. Yes, your data need to be tested for normality (Gaussian distribution); just ask the statistician! You need to indicate which test of normality you used (e.g., Kolmogorov–Smirnov test, Shapiro–Wilk test). You also need to state which statistical analyses you used to analyse all your data.
The power of the power calculation
The best statistical analysis sections include power calculations. These tell your readers that you thought carefully about statistics before you started the study. How many subjects would you need to obtain a significant result? How many experiments would you need to perform to demonstrate a significant difference between groups? Were the groups large enough to detect the expected effect? A power calculation is particularly important if you are performing a controlled trial involving patients. It is essential to demonstrate that you knew how many patients you needed to recruit to obtain valid results.
I have to p-value
Virtually every statistical analysis section includes a description of the level of statistical significance. An alpha value 0.05 (or 5%) is generally accepted for most hypothesis-driven studies. Therefore, you should say ‘An alpha value of 0.05 was considered significant’. Instead, most authors write ‘P < 0.05 was considered statistically significant’. Whereas this is acceptable, it’s best to show the world that you understand what ‘P < 0.05’ means.
Finally, state which statistical software package you used, including version and manufacturer.
A plea for action
Finally, please remember that you are doing the statistical analysis, not a set of equations or a software package. It is preferable to say ‘We analysed the data using SSPS, version 17.0 (IBM, Armonk, NY)’ rather than saying ‘SPSS analysed the data…’. Use active voice! Doing so will remind your readers that software packages don’t do science; scientists do science.