ABSTRACT

For science to reliably support new discoveries, its results must be reproducible. This has proven to be a challenge in many fields including, most notably, fields that rely on computational studies as a means for supporting new discoveries. Reproducibility in these studies is particularly difficult because they require open sharing of data and models and careful control by the original researcher. This is to ensure that products can be run on later generations of hardware and software and produce consistent results. This project will develop software that helps support computational reproducibility and makes it easier and more efficient for geoscientists to preserve, share, repeat and replicate scientific computations. The Broader Impacts of this project include a collaboration between computer scientists, hydrologists and the Consortium of Universities for the Advancement of Hydrologic Science Inc. (CUAHSI) for the hydrology research community. With over 3500 users, and holding over 8000 model and data resources, this collaboration will bring improved tools and best practices to a broad and diverse community of geoscientists. Beyond hydrology, the methods and tools developed as part of this project have the potential to be extended to the solid Earth and space science geoscience domains. They also have the potential to inform the reproducibility evaluation process as currently undertaken by journals and publishers. The projct will also conduct workshops to train researchers and be used in the classroom at Utah Sate Universtiy, DePaul University and the University of Virginia.

Emphasis on the importance of research reproducibility is steadily rising, however many studies still continue to not be reproducible. Reproducibility in computational studies is particularly difficult because of the challenges involved in completely documenting the data, models and procedures used together with the underlying hardware and software dependencies. The reproducibility workbench software (ReproBench) developed in this project will address reproducibility questions by establishing a container-based reproducible workflow that will make it easy and efficient for geoscientists to verify scientific results. Automation and documentation are two key methods for improving verification and, in general, the conduct of reproducible science. This project will build-from past investments: (I) automated containerization methods, through the Sciunit project, and (II) well-documented, community-adopted interfaces, through HydroShare, and bring these investments together to establish a novel, robust, and reproducible workflow. By applying this workflow to water-related science use cases, this project will demonstrate how to preserve, share, repeat, and replicate scientific results. The interfaces can become an exemplar for other community cyberinfrastructure that, akin to Hydrology, aims to share data and models at a large scale. In establishing this workflow, the ReproBench project team combines expertise in cyberinfrastructure, domain science, and reproducible computational data science. By leveraging Sciunit, ReproBench brings formal methods for the conduct of reproducible computational science into the geosciences.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

Please report errors in award information by writing to: awardsearch@nsf.gov.