Diversity, equity and inclusion: how funding agencies are addressing inequalities in research

Yohanna Juk¹, Sergio Salles-Filho², Karen E. F. Pinto³, Ana Carolina Spatti⁴, Evandro Coggo Cristofoletti⁵, Gabriela Araujo Tetzner⁶ and Emily Campgnolli⁷

¹yjuk@unicamp.br, ORCID: 0000-0002-8960-3371

² sallesfi@unicamp.br; ³karenefp@unicamp.br; ⁵evcoggo@unicamp.br; ⁶gabrielatetzner@gmail.com; ⁷emilymaciel2003@gmail.com

²0000-0002-8727-4500; ³0000-0002-7217-2632; ⁵0000-0001-5178-6451; ⁶0000-0002-3702-0086

Department of Science and Technology Policy, University of Campinas, Brazil

⁴spatti@unicamp.br, 0000-0001-8839-8276

University of Campinas, School of Applied Science (Brazil)

Abstract: Recent events have reinforced the need to implement responsible research assessments to address different forms of inequalities in science (e. g. gender, race, language). Funding agencies have a central role in mitigating or perpetuating inequities and can influence societal priorities and policies. Considering this background, we investigated how six FAs around the world are addressing EDI in their funding activities. The following aspects were considered: if more types of diversities are being addressed besides gender approaches; if FAs hold specific departments to address EDI issues; if internal guidelines to eliminate EDI-related bias in peer review processes, grant and projects evaluations we implemented; if FA conduct systematic evaluation of diversity data; and finally, if FAs makes diversity data available. Our findings indicate that FAs are considering EDI in their activities and are advancing in a more intersectional approach to addressing EDI.

1. Introduction

The beginning of the 21st century brought a growing interest in meta-research and research integrity. The debate accompanies, both the need to generate evaluations and metrics of research impact (Wilsdon et al., 2015) usually influenced by best practices conducted by funding agencies, and a growing understanding of the strong relationship between Responsible Research and Innovation (RRI) and Responsible Research Assessment (RRA) in the dynamics of production and dissemination of knowledge and innovation. Those are relevant topics for illuminating the debates on various forms of inequality (e.g., gender, race, language) in science (Burget et al., 2017).

More recently, the discussion on RRA has been advancing worldwide, particularly as a topic of interest in several scientific initiatives, mainly the Research Excellence Framework in the UK, the San Francisco Declaration on Research Assessments in 2012 (DORA, 2021), the Metric Tide in 2014 (Wilsdon et al., 2015), revisited in 2022, the Leiden Manifesto in 2015 (Hicks et al., 2015), and the most recent European Agreement of Reforming Research Assessment (Science Europe, 2022). These efforts have encouraged funders, research institutions, publishers, and other stakeholders to focus on the fundamental aspects of research assessment, such as methodologies, systems, mechanisms of incentives and recognition related to research projects, researchers, and research teams (Curry et al., 2020).

Curry et al. (2020) defend that RRA is an umbrella concept that comprehends the principles of equity, diversity and inclusion (EDI) in research. Although there is still no consensus among researchers and institutions of what EDI actually implies, we can use Dewidar et al. (2022) definition that states that diversity refers to the presence of people of diverse identities in the research fundings, publications and editorial teams; equity refers to recognizing the existence of identity-based advantages and barriers as well as working to correct and address imbalances; and inclusion refers to creating an environment where all those with diverse identities are welcomed and valued (Dewidar et al., 2022).

According to Hunt et al. (2022) and Henry et al. (2021), funding agencies have ample room to improve their policies and, encouraged by social movements and scientists, have begun to implement policies that consider the many diversities in science into grant proposal processes “where these factors have been shown to play a role” (p. 1492).

To effectively address this diversity gap, funding agencies must examine its readiness to confront underlying structural biases as a cause of the lack of EDI in science. That is why we are seeing many funding agencies around the world developing their own ways to implement policies and mechanisms that integrate elements of RRI, RRA and EDI in their initiatives, which necessarily involve pluralized data sources, diversity of stakeholders, and open policy frames (Rafols; 2019).

Our research aims to present key evidence about how funding agencies are addressing, promoting and assessing EDI in science. Our guide-research question is: How are EDI-related initiatives being implemented and assessed by funding agencies around the world?

We therefore conducted a literature review on the topics of RRA and EDI, providing an updated picture of how this topic is being approached in recent literature. We conducted an empirical analysis on secondary data from the selected funding agencies, analysing their recent strategic plans and initiatives that address EDI-related elements and finally we compared the initiatives, common approaches, and innovative activities from the selected funding agencies.

2. Conceptual literature

The beginning of the 21st century brought a vivid interest in meta-research, research integrity, and bibliometrics and raised awareness by researchers that research was vulnerable to misconduct and inaccuracies and that the way assessments were carried out could be adversely affecting the research environment. The debate has intensified across the international research community by addressing the issues mentioned above (Jong et al., 2021; Langfeldt et al., 2020; Ràfols, 2019). This debate aims to find answers about how research can be better funded and practiced and how research cultures can be made more open, inclusive, and impactful (Curry et al., 2020).

Funding agencies currently focus not only on research excellence and scientific relevance but also on ethics, integrity, and reproducibility, interdisciplinarity, collaboration, team science, and the need for greater diversity and inclusion (Aiello et al., 2021; Sandes-Guimarães, Velho & Plonski, 2022; Milat, Bauman & Redman, 2015; Curry et al., 2020). A recent study by Salles-Filho et al. (2022) investigated funding agency trends by analysing nine case studies in six countries. Among the identified trends, authors highlighted the commitment to using alternative assessment techniques (such as artificial intelligence or multicriteria mathematical approaches) that can complement usual methods (traditional peer review) precisely to minimize biases and promote equity, diversity, and inclusion.

There has been an increased engagement on RRA over the past decade, prompted by a wave of initiatives. The DORA Declaration in 2012 (DORA, 2021), The Metric Tide in 2014 and now the metric Tide revisited in 2022 (Wilsdon, 2015), and the Leiden Manifesto in 2015 (Hicks et al., 2015) were among the first vital documents to specifically address and raise awareness on the faults of the current assessment. The focus of discussion based on these initiatives to address EDI has been in analysing or identifying the gender gap in science. We can identify specific actions promoted by the abovementioned initiatives, such as the European Commission that already in 2003 endorsed that FAs should question systematically whether, and in what sense, sex and gender are relevant in the objectives and methodology of projects (Hunt et al., 2023). The Agreement on Reforming Research Assessment, published in 2022, is the most hopeful sign of real change by developing a model agreement to create more inclusive an equitable research ecosystem.

Moreover, finally, Curry et al. (2020) defend that RRA is an umbrella concept that comprehends within itself the principles of EDI. EDI is an essential element of the Global Research Council agenda on RRA. Likewise, Ruzycki and Ahmed (2022) propose specific recommendations on how EDI can be considered at each research project step to make the subject actionable.

We observed growing attention to EDI in recent years. However, we still need a consensual understanding of what is, in fact, equity, diversity, and inclusion in research and how we can implement and measure these elements. A consequence of this lack of consensus translates in many studies that varies their understanding of EDI, or that focus in only one type of diversity.

Gender inequalities across scientific careers and disciplines were addressed by Huang et al. (2019). Bibliometrics indicators that can be assessed by authorship of published papers were investigated in Larivière et al. (2013), Ni et al. (2021) and Raman et al. (2022). Ni et al. (2021) combine authorship information with information from an international survey to investigate women’s perceptions of authorship communication, disagreement, and fairness. The Leiden Ranking is also a relevant source for bibliometrics indicators on scientific impact, gender diversity, among many others.

Gender disparities in innovation and patenting were investigated by Sugimoto et al. (2015). Similarly, Koning et al. (2021) examined US biomedical patents and found that although fewer women engage in commercial patenting compared with men, their patents are more likely to focus on women's health. Hoftra et al. (2020) also advance on that matter by investigating the diversity–innovation paradox in science. The study finds that demographically underrepresented students innovate at higher rates than the majority of students, but their novel contributions are discounted and less likely to earn them academic positions. The value of diversity in academia was also addressed, for example, analyzing the structure of academic collaboration via co-authorships, which frequently involve scientists from different locations, disciplines, and backgrounds (AlShebli, Rahwan, & Woon, 2018). This ascending discussion suggests that there are scientific and societal benefits to increasing diversity in science (Kozlowski et al., 2022).

More recently, Kozlowski et al. (2022) extended the investigation of gender inequalities in science by accounting for the intersection of race, gender, and research topics. Intersectionality can be an exciting approach to studies of this nature by considering how different kinds of inequalities are interrelated and mutually shaping each other.

An interesting approach concerns the Sex, Gender and Diversity Analysis (SG&DA) proposed by Hunt et al. (2022) to evaluate how FAs integrate EDI policies. This approach covers intersectional characteristics such as age or life course, indigeneity, race and ethnicity, sexuality, socioeconomic status, and other axes of inequality. The study identified a trend that is in line with Kozlowski et al. (2022) approach of intersectional analysis. In other words, there is a tendency of broadening the sex and gender analysis to incorporate other social conditions, “in the past, EDI has typically focused on “who” is doing the research, not on “how” research is done. This means that special care will be needed to expand EDI to include research methodologies.”

3. Methods

The methodological approach was based on collecting secondary data of the identified funding agencies employing literature review, desk review of official documents and websites from those agencies.

The main topics investigated were: i) does the FA explicitly consider more types of diversities; ii) does the FA created a specific department to address EDI issues in their funding activities; iii) does the FA developed internal guidelines to eliminate EDI-related bias in peer review processes, grant and projects evaluations; iv) does de FA open specific call for EDI-related projects; v) does the FA conduct systematic evaluation of diversity data; vi) does de FA makes diversity data available?

3.1 Case studies

The funding agencies analysed in this manuscript were chosen considering their saliency as a research funding agency within each country’s research community. The following agencies were selected:

-The French Agence Nationale de la Recherche – ANR, a traditional, broad-ranging funding body that is highly committed to the implementation of development policies.

-The South African National Research Foundation – NRF, a funding agency from a less developed country that started a new model, based on planning and evaluation procedures.

-VINNOVA: a world reference for innovation agency.

-In the UK, the UK Research and Innovation – The UKRI, the largest and most important funding body in the UK.

-The National Science Fountation in the United States, a world-leading funding agency.

-The São Paulo Research Foundation FAPESP, one of the largest and most important funding agencies in Brazil

4. Preliminary results

Given the common environment in which different STI funding bodies operate, it is possible to observe common trends across FAs. The main findings are indicated in Table 1.

Table 1 - How funding agencies are addressing EDI

Agencies have increasingly incorporated EDI into their activities, programs, and internal guidelines. Our results are in line with Hunt et al. (2022) findings that indicated that agencies are now including other elements to their gender approach which points to a more intersectional analysis considering many types of diversity (gender, race, ethnicity, age, language, among many others). The NRF approach is particularly interesting as it now adopts gender-, age- and race-specific elements to increase the participation of women and young researchers of black origin in research teams (SA-NRF, 2020). The ANR and Vinnova still specifically focus on a gender dimension in their research and funding instruments.

Many agencies are not creating specific departments to address and conduct EDI activities and programs. This is the case of FAPESP that in November 2022 created the Coordination Programm for matters related to EDI, which provides a set of actions to expand the diversity of the list of students and scientists funded and aims to improve internal processes and remove obstacles associated to gender, ethnicity, or origin. Similarly, UKRI has an advisory group for EDI. The group provides advice and challenge, working to identify and prioritise areas that still need attention. EDI has also been incorporated in strategy and action plan aiming for a more diverse and inclusive research and innovation system in UKRI. While a specific department is not the case of many FAs, NSF for instance has its own Special Emphasis Programs that allow the agency to ensure affirmative steps are made towards providing equal opportunity to minorities, women, and people with disabilities in all areas of employment. Other programs such as ADVANCE and NSF INCLUDES aims to address this issue. NSF also uses an interesting concept of “missing millions” that considers that there is a vast untapped talent pool that exists as a result of the underrepresentation of women and many communities of color in the Science & Engineering workforce. Similarly NRF developed the Leading Researchers and Scholars Programme to support exceptional researchers to transition to globally leading researchers and scholars, with a deliberate focus on Black South African women. ANR announced a work plan that is part of the European GenderSMART project favouring workplace equality. Vinnova has had an action plan since 2019 to achieve gender equality in four years in their funded projects.

Internal guidelines to address inequalities to eliminate EDI-related bias in peer review processes, grant and projects evaluation are specifically incorporated in ANR. To promote gender equality in research, ANR’s calls for proposals are now written in a way that takes the dimension of sex and/or gender equality into account. In 2021, FAPESP updated its Curriculum Summary model by recommending the insertion of information on career interruptions resulting from maternity, paternity, or third-party care leave for those filling out the form and also guided evaluators how to consider this information. NSF has a concern through their research training activities to invest in diversity of STEM talent, aiming to advance racial equity and a STEM workforce that is more representative of the population.

Launching specific calls to approve EDI related projects is a unanimous approach of FAs. ANR has a concern that their calls for proposals must meet an egalitarian drafting requirement by targeting both men and women and incorporating the sex and/or gender dimension in the description of the scientific areas covered by the calls. The same can be said by Vinnova that promoted specific calls for projects that consider gender equality, together with UKRI.

A challenge reported in recent literature is related to ways to measure and evaluate EDI initiatives, policies, and programs. This can also be observed when investigating if FAs are reportedly evaluating or at least monitoring diversity data among their grantees. Indicators of progress have been reported by NSF when evaluating the progress of their proposals to observe in participation of minority groups. NRF claims also to adopt gender-, age- and race-specific indicators to increase the participation of women and young researchers of black origin in research teams. ANR is applying some measures to identify potential biases in evaluation using a quantitative and qualitative methodology as part of the GenderSmart project team. And UKRI have conducted many investigations to benchmark EDI metrics in other countries and organizations to develop their own EDI related policies and programs.

Finally, diversity data availability was only reported by UKRI. This initiative allows the exchange of methodologies and metrics and also allows investigators to identify trends on the diversity of applicants and recipients of funding for each council making up the UKRI.

5. Preliminary conclusions

Despite the growing attention on the matter of EDI in recent years, discussion still seem to be focusing on narratives, principles, and best practices. Investigating how FAs, a core element of the research system, are addressing this issue can be an important starting point to see if real change is actually taking place. FAs seek different ways to support the progress of science, this heterogeneity can also be observed in the six dimensions investigated related to EDI initiatives and approaches. It is interesting to see an intersectional approach more preoccupied with how different types of diversities can be addressed concomitantly and how some funding instruments can be helpful in promoting a more equitable and inclusive research.

Finally, UKRI’s initiatives of making diversity data and methodology publicly available is an innovative and important step towards developing EDI metrics. FAs and universities are the main organizations with hard to-find data and information that could allow researchers to conduct intersectional analysis and expand EDI studies from focusing solely on gender base approaches.

A limitation of this study is that some information could not be found within FAs official reports, strategic plans, and official websites. That could mean that it is just unavailable information but could still be part of internal practices. A further step would be to increase the number of investigating FAs, in order to have a more complete view of trends and practices, and also to obtain primary data from the investigating FAs.

A possible outcome of this research is to inform and inspire different FAs of the most common and innovate practices toward more diverse research, and also to assist FAs and other organizations to achieving develop instruments and EDI policies.

Open science practices

The data used in this research is publicly available in the official websites and official documents, all of public nature, from the selected funding agencies.

Author contributions

Yohanna Juk - conceptualization, investigation, formal analysis, methodology, writing original draft, writing review and editing

Sergio Salles-Filho - conceptualization, supervision, Writing – review & editing

Karen Esteves - conceptualization, investigation, writing original draft, writing review and editing

Ana Carolina Spatti - writing original draft, methodology, writing review and editing

Evandro Coggo Christofoletti - literature review, writing review and editing

Gabriela Tetzner Araujo - literature review, writing review and editing

Emily literature review, writing review and editing

Competing interests

The authors declare that they have no conflict of interest.

Funding information

The authors acknowledge funding from the São Paulo Research Foundation - FAPESP ( Grant # 2021/06285-3, # 2022/05545-4, #2022/05288-1, #2021/05938-3)

References

Aiello, E., Donovan, C., Duque, E., Fabrizio, S.; Flecha, R., Holm, P., Molina, S., Oliver, E., & Reale, E. (2021). Effective strategies that enhance the social impact of social sciences and humanities research. Evidence & Policy, 17(1), 131-146. doi: https://doi.org/10.1332/174426420X15834126054137

AlShebli, B. K., Rahwan, T., & Woon, W. L. (2018). The preeminence of ethnic diversity in scientific collaboration. Nature communications, 9(1), 5163. doi: https://doi.org/10.1038/s41467-018-07634-8

Burget, M., Bardone, E., & Pedaste, M. (2017). Definitions and conceptual dimensions of responsible research and innovation: A literature review. Science and engineering ethics, 23(1), 1-19. doi: https://doi.org/10.1007/s11948-016-9782-1

Curry, S., de Rijcke, S., Hatch, A., Pillay, D., van der Weijden, I., & Wilsdon, J. (2020). The changing role of funders in responsible research assessment: progress, obstacles and the way ahead. (RoRI Working Paper No.3). Research on Research Institute. doi: https://doi.org/10.6084/m9.figshare.13227914.v1

Dewidar, O., Elmestekawy, N., & Welch, V. (2022). Improving equity, diversity, and inclusion in academia. Research Integrity and Peer Review, 7(4). doi: https://doi.org/10.1186/s41073-022-00123-z

DORA. (2021). San Francisco Declaration on Research Assessment. The American Society for Cell Biology (ASCB). Retrieved January 10, 2023, from https://sfdora.org/read/

Editors' Association of Canada. About Editors Canada. Retrieved January 03, 2023, from https://www.editors.ca/about-editors-canada

European Commission, Directorate-General for Research and Innovation, Spaini, C., Peter, V., Bloch, C., et al. (2018). Monitoring the evolution and benefits of responsible Research and Innovation - The evolution of responsible Research and Innovation : the indicators report. Publications Office. doi: https://data.europa.eu/doi/10.2777/728232

Fochler, M., Felt, U. & Müller, R. (2016). Unsustainable Growth, Hyper-Competition, and

Government of Canada. (2021). Best Practices in Equity, Diversity and Inclusion in Research. Retrieved January 10, 2023, from https://www.sshrc-crsh.gc.ca/funding-financement/nfrf-fnfr/edi-eng.aspx#2

Henry, B. V., Chen, H., Edwards, M. A., Faber, L., & Freischlag, J. A. (2021). A new look at an old problem: Improving diversity, equity, and inclusion in scientific research. The American Surgeon, 87(11), 1722-1726.

Hicks, D., Wouters, P., Waltman, L., de Rijcke, S., & Rafols, I. (2015). Bibliometrics: The Leiden Manifesto for research metrics. Nature, 520, 429–431. doi: https://doi.org/10.1038/520429a

Hofstra, et al. (2020). The diversity–innovation paradox in science. Proc. Natl. Acad. Sci. U.S.A. 117, 9284–9291 (2020).

Huang, J., Gates, A. J., Sinatra, R., & Barabási, A. L. (2020). Historical comparison of gender inequality in scientific careers across countries and disciplines. PNAS, 117(9), 4609-4616. doi: https://doi.org/10.1073/pnas.1914221117

Hunt, L.; Nielsen, M. W.; Schiebinger, L. (2022). A framework for sex, gender, and diversity analysis in research: funding agencies have ample room to improve their policies. Science, 377(6614), 1492-1495.

Jong, L., Franssen, T., & Pinfield, S. (2021). Excellence’ in the research ecosystem: a literature review. (RoRI Working Paper No.5). Research on Research Institute. doi: https://doi.org/10.6084/m9.figshare.16669834.v2

Koning, S. Samila, J.-P. Ferguson. (2021). Who do we invent for? Patents by women focus more on women’s health, but few women get to invent. Science, 372, 1345–1348.

Kozlowski, D.; Lariviere, V.; Sugimoto, C. ; Monroe-White, T.. Intersectional inequalities in science. PNAS, 119 (2), p. 1-8. Doi: https://doi.org/10.1073/pnas.2113067119

Langfeldt, L., Nedeva, M., Sörlin, S., & Thomas, D. A. (2020). Co-existing Notions of Research Quality: A Framework to Study Context-specific Understandings of Good Research. Minerva, 58, 115–137. doi: https://doi.org/10.1007/s11024-019-09385-2

Larivière, V., Ni, C., Gingras, Y., Cronin, B., & Sugimoto, C. R. (2013). Bibliometrics: Global gender disparities in science. Nature, 504, 211–213. doi: https://doi.org/10.1038/504211a

Milat., A. J., Bauman, A. E., & Redman, S. (2015). A narrative review of research impact assessment models and methods. Health Research Policy and Systems, 13(18). doi: https://doi.org/10.1186/s12961-015-0003-1

Nature. (2022). Support Europe’s bold vision for responsible research assessment. Nature Editorial, 607, 636. doi: https://doi.org/10.1038/d41586-022-02037-8

Ni, C., Smith, E., Yuan, H.,Larivière, V., & Sugimoto, C. R. (2021). The gendered nature of authorship. Science Advances, 7(36). doi: DOI: 10.1126/sciadv.abe4639

Østergaard, C. R., Timmermans, B., & Kristinsson, K. (2011). Does a different view create something new? The effect of employee diversity on innovation. Research policy, 40(3), 500-509. doi: https://doi.org/10.1016/j.respol.2010.11.004

Ràfols, I. (2019). S&T indicators in the wild: Contextualization and participation for responsible metrics. Research Evaluation, 28(1), 7–22. doi:  https://doi.org/10.1093/reseval/rvy030

Raman, R.; Subramaniam, N.; Nair, V.K.; Shivdas, A.; Achuthan, K.; Nedungadi, P. Women Entrepreneurship and Sustainable Development: Bibliometric Analysis and Emerging Research Trends. Sustainability 2022, 14, 9160. https://doi.org/10.3390/su14159160

Ruzycki, S. M., & Ahmed, S. B. (2022). Equity, diversity and inclusion are foundational research skills. Nature Human Behaviour, 6, 910–912. doi: https://doi.org/10.1038/s41562-022-01406-7

Salles-filho, S. et al. (2022). Trends in STI Funding Agencies. Report prepared for the São Paulo Research Foundation (Fapesp).

Sandes-Guimarães, L. V., Velho, R., & Plonski, G. A. (2022). Interdisciplinary research and policy impacts: Assessing the significance of knowledge coproduction. Research Evaluation, 31(3), 344-354. doi: https://doi.org/10.1093/reseval/rvac008

Science Europe. (2022). Agreement on Reforming Research Assessment. Retrieved January 10, 2023, from https://www.scienceeurope.org/our-resources/agreement-reforming-research-assessment/

Sugimoto, C. R., Ni, C., West, J. D., & Larivière, V. (2015) The Academic Advantage: Gender Disparities in Patenting. PLOS ONE, 10(5). doi: https://doi.org/10.1371/journal.pone.0128000

Wilsdon, J., Allen, L., Belfiore, E., et al. (2015). The Metric Tide: Report of the independent review of the role of metrics in research assessment and management. doi: 10.13140/RG.2.1.4929.1363