I'm not a medical (/ medical research) professional, but I know my probability theory and a bit of statistics. When leafing through some meta-study texts (such as this one), I'm encountering use of the terms "powered sample" and "unpowered sample". What do these mean? Is it that the sample sizes are insufficient for reasonable levels of confidence?
1 Answer
Yes, these are referring to statistical power.
Statistical power is the probability of correctly rejecting a null hypothesis given a certain sample size and expected magnitude of effect (i.e., defining the alternative hypothesis). It should be calculated prior to doing a study. Although it is technically possible to calculate post-hoc power (using an observed effect size), doing so is totally useless.
In the paper you link, they seem to be using the term "unpowered" to mean the study was not designed with power in mind (that is, the sample size seems to have been chosen arbitrarily). One could also refer to studies being "underpowered" if they have too small a sample size to detect a given effect. In my experience "underpowered" is a much more typical term though I've seen the terms used interchangeably in some cases. It's like the difference between saying "we ran out of milk" versus "we don't have milk": the first phrase tells you more about the process leading to the lack of milk, but both convey the same information about the current state: no milk.
Frustratingly, though, the paper you link has this sentence:
The study used a fairly large sample (n = 80) but was underpowered to detect any statistically significant effects.
This sentence is unfortunately referring to the aforementioned (totally useless) post-hoc power analysis. The study they are referring to in fact was powered to measure a particular effect, the only rationale for describing it as "underpowered" is because the effect measured was not significant. Calling it underpowered is effectively assuming that the null hypothesis should have been rejected; by that argument, every single study that does not show a significant result is "underpowered" - this is bad logic.
I will say, though, that in the context of meta analysis, one could plausibly mark studies as underpowered post-hoc by applying some universal expected effect rather than actual measured effects.
As far as the phrase "powered sample size" - I have not encountered this phrase, I think it's a bit of a language quirk perhaps by translation from a more common phrase in another language. The authors here are clearly using it to mean "they based their sample size on a (reasonable) power analysis," however.
answered May 31, 2019 at 16:08
