Reproducibility efforts as a teaching tool: A pilot study.

Karathanasis, Nestoras; Hwang, Daniel; Heng, Vibol; Abhimannyu, Rimal; Slogoff-Sevilla, Phillip; Buchel, Gina; Frisbie, Victoria; Li, Peiyao; Kryoneriti, Dafni; Rigoutsos, Isidore

The "replication crisis" is a methodological problem in which many scientific research findings have been difficult or impossible to replicate. Because the reproducibility of empirical results is an essential aspect of the scientific method, such failures endanger the credibility of theories based on them and possibly significant portions of scientific knowledge. An instance of the replication crisis, analytic replication, pertains to reproducing published results through computational reanalysis of the authors' original data. However, direct replications are costly, time-consuming, and unrewarded in today's publishing standards. We propose that bioinformatics and computational biology students replicate recent discoveries as part of their curriculum. Considering the above, we performed a pilot study in one of the graduate-level courses we developed and taught at our University. The course is entitled Intro to R Programming and is meant for students in our Master's and PhD programs who have little to no programming skills. As the course emphasized real-world data analysis, we thought it would be an appropriate setting to carry out this study. The primary objective was to expose the students to real biological data analysis problems. These include locating and downloading the needed datasets, understanding any underlying conventions and annotations, understanding the analytical methods, and regenerating multiple graphs from their assigned article. The secondary goal was to determine whether the assigned articles contained sufficient information for a graduate-level student to replicate its figures. Overall, the students successfully reproduced 39% of the figures. The main obstacles were the need for more advanced programming skills and the incomplete documentation of the applied methods. Students were engaged, enthusiastic, and focused throughout the semester. We believe that this teaching approach will allow students to make fundamental scientific contributions under appropriate supervision. It will teach them about the scientific process, the importance of reporting standards, and the importance of openness. Author summary: Many scientific findings have been difficult or impossible to reproduce, resulting in the "replication crisis." Because the repeatability of empirical data is a vital component of the scientific process, such errors imperil the validity of hypotheses dependent on them and potentially significant amounts of scientific knowledge. Nevertheless, direct replications are expensive, time-consuming, and unrewarded in today's publishing standards. Thus, we suggest that bioinformatics and computational biology students reproduce current findings as part of their coursework. In one of the graduate-level courses, Intro to R Programming, we conducted a pilot study in which we asked the students to reproduce figures from published articles. The course is for students in our Master's and PhD programs who have little to no programming experience and focus on the difficulties of analyzing real-world data. They recreated 39% out of the assigned figures, with the most significant barrier being the requirement for more sophisticated programming abilities and a lack of documentation for the methods used. We noticed that students find it fascinating and enjoyable to replicate scientific results. Properly supervised, it will allow them to make essential scientific contributions. It will also teach them about the scientific process, the relevance of reporting standards, and the value of transparency.

SCIENTIFIC knowledge; PILOT projects; COMPUTATIONAL biology; BIOLOGY students; SCIENTIFIC method; PUBLISHED articles

PLoS Computational Biology, 2022, Vol 18, Issue 11, p1

1553-734X

Academic Journal

10.1371/journal.pcbi.1010615