Results from Class survey…
I can see the NEJM article title now…
"Being born on odd days predisposes you to alcoholism and laziness, but makes you a better med student."
Results from Class survey…
Assuming that this difference can't be explained by astrology, it's obviously an artifact!
What's going on …
Results from Class survey…
After the odd/even day question, I asked you 25 other questions…
I ran 25 statistical tests (comparing the outcome variable between odd-day born people and even-day born people).
So, there was a high chance of finding at least one false positive!
P-value distribution for the 25 tests…
Recall: Under the null hypothesis of no associations (which we'll assume is true here!), p-values follow a uniform distribution…
My significant p-values!
Compare with…
Next, I generated 25 "p-values" from a random number generator (uniform distribution). These were the results from two runs…
In the medical literature…
Hypothetical example:
Researchers wanted to compare nutrient intakes between women who had fractured and women who had not fractured.
They used a food-frequency questionnaire and a food diary to capture food intake.
From these two instruments, they calculated daily intakes of all the vitamins, minerals, macronutrients, antioxidants, etc.
Then they compared fracturers to non-fracturers on all nutrients from both questionnaires.
They found a statistically significant difference in vitamin K between the two groups (p<.05).
They had a lovely explanation of the role of vitamin K in injury repair, bone, clotting, etc.
In the medical literature…
Hypothetical example:
Of course, they found the association only on the FFQ, not the food diary.
What's going on Almost certainly artifactual (false positive!).
Factors indicative of chance findings
Adjustment for multiple comparisons can help control the study-wide false positive rate.

上一页下一页