Math
]
Statistical Power and Effect Size
Definition
Power refers to what percentage of the time we would reject the null hypothesis if the null hypothesis were false.
Mathematically, the power is equal to $1-\beta$, which is $1- Type II error$.
That is, power is the probability of not making a Type II error/the probability of detecting an effect when there is an effect. The higher the statistical power for a given experiment, the lower the probability of making a Type II error.
Why It matters
Below is a quote from a wonderful article:
I’ve spent years working with social sector organizations, helping them answer important questions with data. And the idea of statistical power is one that’s often thought of too late — or not at all.
Why does it matter? If you hope to understand how likely it is that your research noticed an effect or trend actually occurring in the real world, you need to understand the concept of statistical power. Evidence-based decision- and policy-making is gaining popularity. If you can wrap your head around the idea of statistical power, you’ll better understand your chances of making the right choice based on your data.
Cautions
post-hoc power analyses; that is, calculating the power associated with a sample or full dataset that you already collected. This type of analysis is uninformative (since we cannot say with any certainty whether our result came from the null distribution or a specific alternative distribution), and can be misleading (see Hoenig & Heisey, 2001).