MINI LESSON 7: PValue Hacking: a simplified lecture.
 Video
 May 23, 2021
 #Probability
Video
Channel:
N N Taleb's Probability Moocs
We saw that 1) many metrics are stochastic, 2) what is stochastic can be hacked. This is the simplification of my work showing that "pvalues are not pvalues", i.e. highly sample dependent, with a skewed distribution. For instance for a "true" P value of .11, 53% of observations will show less than...
Show More

You might also be interested in

 Article
 Jan 4, 2023

 Tweet

by
 Jan 1, 2023

 Article
 Oct 30, 2022
We saw that 1) many metrics are stochastic, 2) what is stochastic can be hacked. This is the simplification of my work showing that "pvalues are not pvalues", i.e. highly sample dependent, with a skewed distribution. For instance for a "true" P value of .11, 53% of observations will show less than .05. This allows for hacking: in a few trials a researcher can get a fake pvalue of .01.
Paper is here and in Chapter 19 of SCOFT (Statistical Conseq of Fat Tails):
https://arxiv.org/pdf/1603.07532.pdf