- Jonathan Whitmore, PhD
- Data Scientist, SVDS
- @jbwhitmore
- Blinding statsmodels output: https://gist.github.com/dmargala/be3ef4c7cd676eac07e3dc8add4603d2
- http://pdg.lbl.gov/2005/reviews/historyrpp.pdf
- http://news.berkeley.edu/2015/10/08/blind-analysis-could-reduce-bias-in-social-science-research/
- https://www.slac.stanford.edu/econf/C030908/papers/TUIT001.pdf
- BLIND ANALYSIS IN NUCLEAR AND PARTICLE PHYSICS Joshua R. Klein and Aaron Roodman Annual Review of Nuclear and Particle Science, Vol. 55: 141 -163 (Volume publication date December 2005)
- https://peerj.com/preprints/1733v1/
- Terrell J, Kofink A, Middleton J, Rainear C, Murphy-Hill E, Parnin C. (2016) Gender bias in open source: Pull request acceptance of women versus men. PeerJ PrePrints 4:e1733v1 https://doi.org/10.7287/peerj.preprints.1733v1
- https://peerj.com/preprints/1733/
- https://peerj.com/questions/2002-do-you-have-data-on-the-gender-of-the-users-that/#annotation-2002-replies
- Some Criticism
- http://slatestarcodex.com/2016/02/12/before-you-get-too-excited-about-that-github-study/
- Silberzahn, R. & Uhlmann, E. L. Psychol. Sci. 24, 2437–2444 (2013).
- Silberzahn, Raphael et al. “Many Analysts, One Dataset: Making Transparent How Variations in Analytical Choices Affect Results.” Open Science Framework, 20 Aug. 2015. Web.
- Final Manuscript: https://osf.io/j5v8f/
- Data: https://osf.io/47tnc/
- http://fivethirtyeight.com/features/science-isnt-broken/#part2
-
Source: The reusable holdout: Preserving validity in adaptive data analysis, Dwork, C., Feldman, V., Hardt, M., Pitassi, T., Reingold, O., Roth, A, Science, 07 Aug 2015: 636-638.
-
https://research.googleblog.com/2015/08/the-reusable-holdout-preserving.html
-
https://www.oreilly.com/ideas/3-ideas-to-add-to-your-data-science-toolkit
- http://www.nature.com/news/how-scientists-fool-themselves-and-how-they-can-stop-1.18517
- http://www.nature.com/news/blind-analysis-hide-results-to-seek-the-truth-1.18510
- http://www.nature.com/news/let-s-think-about-cognitive-bias-1.18520
- http://www.nature.com/news/crowdsourced-research-many-hands-make-tight-work-1.18508
-
Gelman, Andrew, and Eric Loken. "The garden of forking paths: Why multiple comparisons can be a problem, even when there is no “fishing expedition” or “p-hacking” and the research hypothesis was posited ahead of time." Department of Statistics, Columbia University (2013).
-
Considering the opposite: A corrective strategy for social judgment. Lord, Charles G.; Lepper, Mark R.; Preston, Elizabeth Journal of Personality and Social Psychology, Vol 47(6), Dec 1984, 1231-1243.
-
Kerr, Norbert L. "HARKing: Hypothesizing after the results are known." Personality and Social Psychology Review 2.3 (1998): 196-217.
-
The reusable holdout: Preserving validity in adaptive data analysis, Dwork, C., Feldman, V., Hardt, M., Pitassi, T., Reingold, O., Roth, A, Science, 07 Aug 2015: 636-638.