Reporting mean differences
Why do I need to do this?A lot of biology involves (1) doing an experiment where you make an intervention and (2) measuring what effect the intervention has.If you are living in the Dark Age ofP, you assume your intervention has had absolutely no effect, and then calculate the probability of seeing your data (or more extreme data), under that assumption ofzeroeffect. Since you likely did the experiment because you thought there would be an effect, it seems super weird and slightly depressing to then go assuming zero effect when you start analyzing your data.If you useestimation, a far more sensible method, you want to estimate the size of the effect of your intervention. The most straightforward effect size is the difference between the control mean and the intervention mean (‘mean difference’), along with its confidence interval.I’ve got confidence interval error bars, is that enough?It’s a good start to have CI error bars onyour observed data plots. However, you alsoneed to be able to say things like:“when we made the intervention to flies their behavior X increased by +54% [95CI +35, +78],P= 0.01.”You need (1) know what your delta variable is and (2) know how to get these numbers.What statistics are essential?The three essential statistics are:Mean difference (your ‘delta variable’)Confidence interval of the mean differenceSample sizes (N)Figure out how to calculate these numbers, and work them into your writing, especially in the figure legend, and also in the main text when you want to highlight an effect. It’s simply not enough to say “behavior X increased,” or worse, “behavior X increased significantly” without putting an effect size number on that statement, either immediately in the Results section or in the relevant Figure legend. If you have this you can import it into a Google Spreadsheet and share with your co-authors, one sheet per Figure or Panel.What statistics are nice to have?Hedges’gPIt’s nice to have Hedges’g,because this effect size is in units of standard deviations, which is sort of a universal currency. Most scientists will know what it means for an intervention to produce a 1 SD shift in the mean, and have an intuition that this is a large effect. ReportPforpro formapurposes only and state “no significance tests were conducted,Pwas reportedpro forma.” This will satisfy reviewers wanting to seePwhile avoiding significance testing.What text style should I follow?The text format we are using for mean difference with its confidence interval is:-1.5 [95CI -1.2, -1.8]or+1.5 [95CI +1.2, +1.8]Note the use of the +/- signs to denote that this is a measure of thechangein the variable, not a measure of the variable itself. I prefer ‘95CI’ instead of ‘95%CI’ as I find the latter to be cluttered.Often it is useful to write something like this.∆weight = +1.5 µg [95CI +1.2, +1.8]to remind the reader of the change variable (∆weight)and the units (µg). The confidence interval bounds are contained within square brackets [], and this generally follows the mean difference closely in text. It is then followed by the supporting statistics. For example:∆VO2 = -1.5 µl/fly/min [95CI -1.2, -1.9],N= 65, 62or∆VO2 = -1.5 µl/fly/min [95CI -1.2, -1.9],g= 0.56,P= 0.01,N= 65, 62Thegstatistic was invented by Larry Hedges, so Hedges’guses a possessive apostrophe after the ‘s.’ The’g’is italicized.Pshould be italicized and capitalized by default.What graphical style should I follow in charts?Instead of the little stars, you can put the effect size right next to your difference marker. In this case, simplicity is a virtue. You can write ‘∆ = -1.5’ next to the first marker, then just the numbers without the ‘∆ =’ for the rest of the markers in that Figure.