Frequentist statistics are really… silly in a way. And this coming from someone who has to teach it. Sure, p is less than 5%, but you sampled 100,000 people-- an effect size of 0.05 would be significant at this rate. “bUt ItS sIgNiFiCaNt”… Oy.
I get very suspicious if a paper samples multiple groups and still uses p. You would use q in that case, and the fact that they didn’t suggests that nothing came up positive.
Still, in my opinion it’s generally OK if they only use the screen as a starting point and do follow-up experiments afterwards
Yeah, I used to work in a field with huge samples so significance wasn’t really all that useful. I usually just report significant coefficients and try to make clear what changes by model. For instance, if a type of curriculum showed improvements on test scores, you simply say how much and, possibly, illustrate it by saying if a person went from 50th percentile to 55th percentile.
Every field varies, though. I find it crazy how much psychologists I’ve worked with cared about r-squared. To each their own, I guess.
Frequentist statistics are really… silly in a way. And this coming from someone who has to teach it. Sure, p is less than 5%, but you sampled 100,000 people-- an effect size of 0.05 would be significant at this rate. “bUt ItS sIgNiFiCaNt”… Oy.
I get very suspicious if a paper samples multiple groups and still uses p. You would use q in that case, and the fact that they didn’t suggests that nothing came up positive.
Still, in my opinion it’s generally OK if they only use the screen as a starting point and do follow-up experiments afterwards
Yeah, I used to work in a field with huge samples so significance wasn’t really all that useful. I usually just report significant coefficients and try to make clear what changes by model. For instance, if a type of curriculum showed improvements on test scores, you simply say how much and, possibly, illustrate it by saying if a person went from 50th percentile to 55th percentile.
Every field varies, though. I find it crazy how much psychologists I’ve worked with cared about r-squared. To each their own, I guess.