Statistical Power: When is it useful?

About this episode

Statistical power is one of the most important ideas in statistics—and one of the most misunderstood. In this episode, we explain what statistical power really means, why researchers need to think about it before they collect their data, and what goes wrong when they calculate “post hoc power” after the study is over. Along the way, we hunt for UFOs, test Kristin’s psychic abilities, explore the ethics of underpowered studies, and discover why post hoc power is really just a p-value wearing a fake mustache. Then statistician and powerlifter Andrew Althouse joins us for the inside story of a years-long scientific feud over post hoc power, where we discuss what happens when being wrong about statistics turns into doubling down, angry letters to the editor, and debates about how science corrects itself. Statistical topics • Effect size • False negatives • P-values • Post hoc power • Power calculations • Research ethics • Sample size • Scientific controversy • Sensitivity analysis • Statistical power • Statistical significance • Study design • Type II error Methodologic Morals • “Big questions need studies powerful enough to answer them.” • “Post hoc power is circular reasoning