Examining the Replication Crisis

Friday, April 8, 2016 - 12:30pm to 2:00pm

The ability to replicate research findings is an essential component of the scientific process.  However, the scientific process itself has come under considerable scrutiny due to recent evidence that the results of many studies and experiments are difficult, if not impossible, to replicate.[1]  Many trace the roots of the problem to an academic merit system that rewards original research and positive results with publication, while discounting studies that involve replication or show insignificant findings.  Others argue that conflicting and contradictory findings are a natural part of the process of discovery where knowledge does not hinge on a single experiment, but rather evolves over the course of multiple studies.  The inability to replicate may simply show that that context matters – results found in one setting may very well differ in another.

This panel discussion will explore the causes, controversies, and consequences of the “Replication Debate” featuring methodologists and researchers from across the social and computational sciences. We will also explore the pros and cons of several proposed remedial actions, such as expecting more stringent statistical tests, abandoning the use of threshold p values, requiring larger samples, and requiring publishing scholars to provide open access to their datasets and code. The event is hosted by ISSR and the Computational Social Science Institute.

Featured Speakers

  • Emery Berger (Information and Computer Science)
  • Thomas Herndon (Economics)
  • David Jensen (Information and Computer Science)
  • Caren Rotello (Psychology)
  • Adrian Staub (Psychology)

[1] Some recent examples include:

Chang and Li (2015). “Is Economics Research Replicable? Sixty Published Papers from Thirteen Journals Say 'Usually Not',” Finance and Economics Discussion Series 2015-083. Washington: Board of Governors of the Federal Reserve System.

Herdon, Ash and Pollin (2013). “Does High Public Debt Consistently Stifle Economic Growth?  A Critique of Reinhart and Rogoff.” Cambridge Journal of Economics.

Open Science Collaboration (2015). "Estimating the Reproducibility of Psychological Science." Science 349: 6251.