To provide reliable information on the adoption of Open Science practices, the Public Library of Science (PLOS) defined underlying principles and a framework for measuring Open Science practices across the scholarly literature. PLOS then partnered with DataSeer to develop a solution, Open Science Indicators (OSI), which uses natural language processing and artificial intelligence in its methodology. The OSI results show increasing adoption of data sharing, code sharing and preprint posting from 2019 to 2022 in a cohort of 78,744 analyzed research articles, New results will be released quarterly in 2023. OSI data and methods are shared openly to enable their reuse, and to support the work of other organizations and researchers who seek better evidence on the adoption of Open Science.

We surveyed 485 biology researchers who have served on committees for grant review or hiring and promotion decisions, to understand how they assess the credibility of research outputs in these contexts. We found that assessment of credibility is very important to researchers serving in these committees but researchers are dissatisfied with their ability to judge credibility and often use inappropriate proxies, such as journal reputation and impact factor, to do this. Non-traditional research outputs associated with Open Science practices are particularly hard to assess, despite their potential to signal and support credibility assessments. Current policy may be overly focused on assessment of impact, and there are opportunities to provide better solutions to enable researchers to judge the credibility of research.