Reproducibility: A Primer on Semantics and Implications for Research

Science is allegedly in the midst of a reproducibility crisis, but questions of reproducibility and related principles date back nearly 80 years. Numerous controversies have arisen, especially since 2010, in a wide array of disciplines that stem from the failure to reproduce studies or their findings: biology, biomedical and preclinical research, business and organizational studies, computational sciences, drug discovery, economics, education epidemiology and statistics, genetics, immunology, policy research, political science, psychology, and sociology.

This book defines terms and constructs related to reproducible research and examines considers key considerations and challenges in reaching key goals of credible scientific investigations. It clarifies reproducible research, with its attendant (and confusing or even conflicting) lexicon, and provides useful background, definitions, and practical guidance for all readers.

The authors conclude that researchers must become better educated about these issues, so as to communicate more clearly within their own fields and, more importantly, across multiple disciplines. Scientists should embrace these concepts as part of their responsibilities as good stewards of research funding and as providers of credible information for public policy decision-making in many areas of public policy. The goal is achieving the most rigorous, high-quality science possible given limitations on time, funding, or other resources.

Edo D. Pellizzari, AB in biology, PhD in biochemistry, Lead Fellow (Emeritus) at RTI International, has nearly 40 years of research experience in biochemistry, analytical chemistry, and environmental health. His research focused on exposure of children and adults to organic and inorganic chemicals in air, house dust, food, drinking water, and human biological media, and PM10 and PM2.5 particles in air. Dr. Pellizzari developed and applied methods for exposure assessment to 14 probability-based human population studies throughout the United States and Canada. For more information, visit www.rti.org/experts.

Kathleen N. Lohr, PhD, MPhil, MA, Distinguished Fellow at RTI International, has more than 40 years of experience in health care and health policy research. She founded and is now senior advisor to the RTI–University of North Carolina Evidence-based Practice Center. In 2005, she was awarded the Avedis Donabedian Outcomes Research Lifetime Achievement Award by the International Society of Pharmacoeconomics and Outcomes Research. Dr. Lohr has been a Research Professor and adjunct faculty in Health Policy and Management in the University of North Carolina Gillings School of Global Public Health.

Alan Blatecky, MBA, MA, THM, MDiv, is a Visiting Fellow at RTI International; he previously directed the NSF Office of Cyberinfrastructure at the National Science Foundation. He helped co-found the Renaissance Computing Institute and was also the Executive Director of Research and Programs at the San Diego Supercomputing Center. As Vice President of Information Technology at MCNC, he played a central role in establishing the NC Research and Education Network and the North Carolina Supercomputing Center.

Darryl V. Creel, MS, is a Senior Research Statistician at RTI International. He has 19 years of experience working in the statistical field with a focus on survey sampling and biostatistical research. His general statistical experience includes survey sampling, data visualization, data analysis, and statistical learning. His statistical experience has been applied to a wide spectrum of research projects, ranging from large national surveys to relatively small surveys of highly specific populations.