Research, Rigor & Reproducibility: The Seven Deadly Selection Biases (3.5.18)

Speaker: Xiao-Li Meng, PhD, Dean, Graduate School of Arts and Sciences (GSAS, on sabbatical for the 2017–2018 academic yearr) and the Whipple V. N. Jones Professor of Statistics, Harvard University

Description: This talk provides a statistical perspective on the roles the seven S’s (sins?) play in increasing the amount of irreproducible research, in medical and life sciences and beyond:

1. Selections in hypotheses (e.g., subgroup analysis);
2. Selections in data (e.g., deleting “outliers” or only using “complete cases”);
3. Selections in methodologies (e.g., for goodness of fit);
4. Selections in due diligence and debugging (e.g., triple checking only when the outcome seems undesirable);
5. Selections in publication (e.g., only when p-value <0.05);
6. Selections in reporting/summary (e.g., suppressing caveats);
7. Selections in understanding and interpretation (e.g., our preference for deterministic, “common sense” interpretation).

The Big Data Paradox and Simpson’s Paradox will be used to demonstrate that the problem of irreproducible research is getting BIGGER with Big Data. A cocktail treatment approach together with a selfish/blowfish test is suggested to combat this problem.
This event is part of the Research, Rigor & Reproducibility series

Bio: Xiao-Li Meng is the Dean of Harvard University’s Graduate School of Arts and Sciences (GSAS), currently on sabbatical for the 2017–2018 academic year, Whipple V. N. Jones Professor and former chair of Statistics at Harvard, an Honorary Professor of the University of Hong Kong, and a faculty affiliate at the Center of Health Statistics at the University of Chicago. He is well known for his depth and breadth in research, his innovation and passion in pedagogy, and his vision and effectiveness in administration, as well as for his engaging and entertaining style as a speaker and a writer. Full bio here.