Revised A survey of how biology researchers assess credibility when serving on grant and hiring committees

We surveyed 485 biology researchers who have served on committees for grant review or hiring and promotion decisions, to understand how they assess the credibility of research outputs in these contexts. We found that assessment of credibility is very important to researchers serving in these committees but researchers are dissatisfied with their ability to judge credibility, and often use inappropriate proxies, such as journal reputation and Journal Impact Factor (JIF), to do this. Non-traditional research outputs associated with Open Science practices are particularly hard to assess, despite their potential to signal trustworthiness and support credibility assessments. Our results imply there are opportunities to provide better solutions to enable researchers to judge the credibility of research in time-constrained, research assessment contexts. We infer that current policy and the available tools may be overly focused on assessment of impact.