Roadmap for Open Science:
An analysis of open science action plans by Canadian federal science departments

Chantal Ripp^* and Stefanie Haustein^**

^*chantal.ripp@uottawa.ca

https://orcid.org/0000-0003-3544-8158

University of Ottawa, School of Information Studies, Scholarly Communications Lab, Canada

^**stefanie.haustein@uottawa.ca

https://orcid.org/0000-0003-0157-1430

University of Ottawa, School of Information Studies, Scholarly Communications Lab, Canada

Université du Québec à Montréal, Observatoire des Sciences et des Technologies (OST), Centre

Interuniversitaire de Recherche sur la Science et la Technologie (CIRST), Canada

Abstract

This research-in-progress paper examines the implementation of open science (OS) in Canada by federal science-based departments and agencies (SBDAs), following the release of the Roadmap for Open Science (ROS) in February 2020. The paper presents initial findings of a study that explores how SBDAs have engaged in open access (OA) before the ROS came into force in January 2022, the approaches and rationales SBDAs adopt to meet the ROS requirements in their respective OS action plans. The paper employs bibliometric analysis to contextualize the findings of the content analysis on SBDA OS action plans. The results are examined by ROS recommendations that directly affect SBDAs, including consultations with federal scientists, OA publication, FAIR data, and research assessment. The paper provides valuable insights into how the government understands and implements OS activities in Canada and how it intends to address obstacles impeding access to federal research.

1. Introduction

To tackle the complex scientific and societal challenges today, many funders, governments and international organizations argue that science needs to be more transparent, accessible and inclusive. The practice of open science is firmly secured in the values of sharing and collaboration to advance opportunities for innovation and participation in the co-creation of knowledge (UNESCO, 2021). The European Commission, the Organisation for Economic Cooperation and Development (OECD), the United Nations, the World Bank, and the United Nations Educational, Scientific and Cultural Organization (UNESCO), among many more recognize the importance of open science and have released formal statements and recommendations to support its adoption (OECD Council, 2021; UNESCO, 2021).

At the same time, national governments and public funding agencies have gradually adopted policies that promote open science (OS) principles, including open access (OA) to scientific articles and access to data from research. Most recently, the White House announced that all US federal funded research must be published immediately OA and underlying data be made available at the time of publication (White House Office of Science and Technology Policy, 2022).

Within the Canadian context, the federal funding bodies have OA and research data management policies in place, while the Quebec provincial funder joined Plan S in 2021 (Birsan, 2021). In February 2020, Canada's Chief Science Advisor released the federal Roadmap for Open Science (ROS) with the overarching goal of making research conducted by federal scientists openly available (Office of the Chief Science Advisory, 2020). The Roadmap’s primary recommendations for federal science-based departments and agencies (SBDAs) are:

• To engage their departmental scientific community to identify challenges and opportunities of OS [ROS recommendation 2];

• Make scholarly articles authored or co-authored by federal scientists published in scholarly journals OA without an embargo period [ROS recommendation 4]; and

• Develop strategies to ensure that scientific data is FAIR (Findable, Accessible, Interoperable, and Reusable) [ROS recommendation 5];

• Develop an OS Action Plan describing how they intend to meet the recommendations set out in the Roadmap [ROS recommendation 3];

The ROS and individual departmental action plans provide an opportunity to critically examine how the government understands and implements OS activities in Canada and how it intends to address obstacles impeding access to federal research. This research-in-progress paper presents initial results that explores the following questions: (1) how have SBDAs engaged in OA prior to ROS coming into force in January 2022?; (2) what are the approaches, rationales and instruments that SBDAs plan to adopt to satisfy the ROS requirements?; and (3) how do they differ between SBDAs?

2. Methodology

This study uses a sequential mixed-methods approach to investigate current publication behaviours of federal scientists and how we may interpret potential impacts of a newly implemented OS policy. We first used bibliometric analysis to ground our understanding of current SBDA publication patterns and then carried out a content analysis of SBDA action plans. The results of the study will establish a framework for future analyses of the policy change and its effect on OS practices by Canadian federal scientists.

2.1 Bibliometric and co-authorship analysis

Bibliographic metadata was retrieved from Web of Science (WoS), by searching organizational affiliations of the ten SBDAs that produced an action plan for the years 2017 to 2021. OA status, including Gold, Hybrid, and Green OA, is provided at the article-level in WoS based on information provided by Unpaywall. Within the OA status, article-level OA of all versions are identified when multiple versions of an article are available (e.g., gold and green, hybrid and green).

2.2 Content analysis of SBDA action plans

The study analyzes the ten publicly available SBDA action plans available at the time of this inquiry using content analysis. The documents were collected from the Office of the Chief Science Advisory website (Government of Canada, 2022). The codebook (Table 1) was developed inductively based on the ROS recommendations and then expanded deductively by reading action plans and examining publication patterns from the bibliometric analysis (Ripp, 2023). Some discrepancies were observed with respect to terminology used to differentiate between types of OA. Within the plans, preprints focus on early communication prior to peer review and are therefore described as distinct from Green OA. The decision was to use the code ‘publisher route’ to denote instances of Gold or Hybrid OA routes in SBDA action plans.

Table 1. Codebook used in content analysis

3. Initial findings

We first present the bibliometric results to contextualize the findings of the content analysis. The content analysis results are subdivided by recommendations that directly affected SBDAs, namely consultations with federal scientists [ROS recommendation 2], OA publication [ROS recommendation 4], and FAIR data [ROS recommendation 5] in addition to findings on research assessment.

3.1 Bibliometric analysis

The Canadian federal government performs science and technology activities and works closely with cross-sector partners, often in shared collaborative facilities, in Canada and abroad (Government of Canada, 2023), which is confirmed by the high collaboration rates observed for all ten SBDAs (Table 2).

Table 2. Publication output 2017-2021, OA status and recommended priority OA route from action plan per SBDA.

3.2 Content analysis

3.2.1 Consultation

The ROS asked SBDAs to consult their scientists to identify challenges, concerns and benefits of OS to inform the development of their action plans. The majority of departments used surveys to gain an understanding of challenges and opportunities faced by their scientists, while a handful held more in-depth focus groups. AAFC provided a reference to the consultation data, however, it was restricted to federal employees only. Concerns about OS (Figure 2) included cost of article processing charges (APCs) for publishing in OA journals, lack of infrastructure to deposit publications and research data, as well as language concerns. Six action plans specifically reference Canada's Language Act, or concerns around Official languages, as a barrier to making federal articles more widely available.

Figure 2. Number of action plans mentioning barriers to OS as reported through consultations

3.2.2 Open access to publications

Many of the action plans confirmed that ultimately the decision on where to publish and whether to make publications OA falls solely with the author. As such, the majority of the action plans (8) identified rectifying current publication policies in place to align with OA, develop support and guidance materials and emphasizing training to support the adoption of OA.

SBDAs have distinguished between three paths to OA, the publisher route (Gold & Hybrid), deposits in a repository (Green), and making manuscripts available before peer review via preprints. The dominant OA type across departments is the repository route (Table 1). Departments that identified the publisher route emphasized the desire to publish in prestigious journals. These plans also identified pilots or intentions to fund costs of publishing in OA journals through centralized funding or transformative agreements with publishers. At the same time, the same plans also identified exploring preprints to share pre-peer reviewed manuscripts as an option for meeting OA targets.

Many of the SBDAs are working collaboratively to explore opportunities to develop shared tools and infrastructure to meet OA targets. Six SBDAs reported developing a new repository for federal science publications, while others have invested in solutions to track and facilitate the discovery of published articles.

3.2.3 FAIR data

Many SBDAs produce data as part of their scientific activities, these include but are not limited to, observational, monitoring, operational, modelling and simulation, risk assessment, survey and surveillance, and diagnostics and targeted survey. Some departments and agencies are dedicated to scientific research, including the National Research Council Canada, while other organizations conduct scientific research within a larger framework, such as Public Health Agency of Canada or Statistics Canada.

How SBDAs interpreted the application of FAIR principles to scientific and research data differed within the action plans. Open Data has been an integral component of the Government of Canada’s commitment to openness since the release of the Directive on Open Government (Government of Canada, 2014). Seven SBDA action plans identified making scientific datasets as collected as part of their mandate open and adhering to the FAIR principles. Conversely, five SBDA action plans focused on enabling the data producer (i.e., researchers) to adhere to OS practices and FAIR principles. The distinction between FAIR principles for departmental collections versus those applied by researchers to promote reproducible results is captured in Figure 3.

Figure 3. Conceptual distinction of scientific and research data concepts

The Roadmap acknowledges that a prerequisite to implementing FAIR data principles are strong data management practices. While seven plans discussed data management broadly, only five offered details and identified the use of data management plans, metadata standards, and curation in support of implementing FAIR principles.

3.2.4 Research assessment

While not explicitly identified as a ROS recommendation, the current metrics-based assessment of researchers is argued to impede progress towards OS (COARA, 2022). Institutions play an important role in terms of providing support and incentives that incite behavioural changes towards more open practices (Huang et al., 202). Plans whose focus was on enabling its researchers to adapt OS practices, for both publications and research data, were more likely to have identified the need to review the evaluation criteria for career progression. Three plans included statements on career progression criteria and recognized that to foster a culture of change and incentivize practices, researchers' efforts need to be rewarded and fostered.

4. Discussion

The Roadmap is founded on the principle that collaboration with intramural and extramural partners is imperative to scientific production, which is corroborated by the findings from the bibliometric analysis. SBDA action plans emphasize guidance and tools to support OA decision making; however, these plans fail to articulate how collaborative outputs and shared decision making in publication venues will be made open in keeping with the ambitions of the Roadmap.

As reported through departmental consultations, the cost of APCs was identified as a barrier to engaging in OA. The bibliometric data showed that federal researchers make their publications open via Green OA. Several SBDAs are piloting and deploying a new repository to support the Green OA route for federally authored papers. As the majority of academic institutions also host institutional repositories, the creation of a federal repository creates another siloed infrastructure within the Canadian research landscape. As advocated for in the development of a Canadian national PID strategy, the creation of an information-sharing solution to connect PID APIs, funder systems, and institutions, would reduce the burden of duplication, lower costs, promote adoption of best practices, and ensure equitable access to support services (Brown et al., 2022). Several countries are developing national research evaluation systems founded on interoperable infrastructure capable of leveraging existing metadata solutions (Bracco et al., 2022; Moreira et al., 2022).

The Roadmap itself calls upon SBDAs to "develop strategies and tools to implement FAIR data principles to ensure interoperability of scientific and research data” (p.8, 2021). It is particularly important, as seen via publications, that SBDA collaborative relationships may influence the extent to which data is made open. Challenges around data ownership, intellectual property, licensing and data use agreements need to be explored, and many of the plans failed to surface these particular constraints in their action plans. Shared understandings of what data should be prioritized to be made ‘FAIR’ are more likely to be implemented within and between departments, at national and international levels, and to foster a culture change towards more open practices. As PHAC addressed in its action plan “federal interoperability of scientific and research data will depend on adopting a common understanding of data definitions across governmental departments (p.9, 2021)”.

While the priority areas for the Roadmap have been discussed separately, they are inheritably interconnected. OA to federal scientific articles and FAIR data can only be adopted as an intrinsic part of the research lifecycle if they receive the appropriate recognition. At the same time, research assessment systems that are transparent can support the advancement of an OS agenda. Research evaluation criteria and metrics need to be reviewed in parallel with evolving trends and new policy initiatives to ensure open science efforts are recognized and fostered. The Career Progression Management Framework for Federal Researchers, referenced in a couple plans, is an internal document to the Government of Canada that provides guidance for career advancement for federal scientist (2006). The Framework requires SBDAs to track research outputs and their impacts. For government scientists, how their department interprets the Framework will greatly influence what researchers see as important and where they may invest their efforts, such as enacting FAIR data principles. As only three action plans identified reviewing the guidelines for career progression, there remains opportunities to update the government researchers’ career progression frameworks to align with the principles and objectives as set out in the Roadmap.

5. Conclusion

The study reported the results of a content analysis of SDBA action plans to satisfy the recommendations set forth in the Government of Canada’s Roadmap on Open Science. The results highlight extensive variation across a number of critical areas, including modes of OA, infrastructure developments, incentivizing culture change, and understanding of data outputs to be rendered FAIR. As evident by the share of co-published research, SBDAs place a strong emphasis on research collaboration to advance scientific objectives across a range of disciplines. Given the integrated position of SBDAs in the public research landscape, there’s need for a common approach to a national open science strategy for Canadian science more broadly, as advocated for in the Roadmap.

Open science practices

As Web of Science data is proprietary, the raw data for the bibliometric analysis cannot be shared. However, as the current analysis is intended to establish a framework for future analyses of the current and anticipated activities of SBDAs to satisfy the recommendations of the Roadmap, the methods could be replicated using open bibliographic metadata databases. The data dictionary and coded SBDA action plans are available through the repository Zenodo to support transparency and replication of results of the content [DOI: 10.5281/zenodo.7844158].

Author contributions

Chantal Ripp: Conceptualization, methodology, formal analysis, investigation, visualization, writing - original draft, writing – review & editing

Stefanie Haustein: Supervision, methodology, writing - review & editing

Competing interests

The authors have no competing interests.

Funding information

The research has been supported in part by funding from Coalition Publica's Scholarship Program.

References

Agriculture and Agri-Food Canada. (2021). Agriculture and Agri-Food Canada’s Open Science Action Plan 2021-2022 to 2025-2026 – Enable and Equip. https://agriculture.canada.ca/en/about-our-department/agriculture-and-agri-food-canadas-open-science-action-plan-2021-2022-2025-2026-enable-and-equip

Birsan, G. (2021, June 1). The Fonds de recherche du Québec support open science by joining cOAlition S. Fonds de Recherche Du Québec. https://frq.gouv.qc.ca/en/les-fonds-de-recherche-du-quebec-appuient-la-science-ouverte-en-joignant-la-coalition-s/

Bracco, L., L’Hôte, A., Jeangirard, E., & Torny, D. (2022). Extending the open monitoring of open science. https://hal.science/hal-03651518

Brown, J., Jones, P., Meadows, A., & Murphy, F. (2022). Towards a national PID strategy for Canada—Vers une stratégie nationale sur les PID pour le Canada. Zenodo. https://doi.org/10.5281/zenodo.7217469

Government of Canada, I. (2022, June 30). Departmental open science action plans. https://science.gc.ca/site/science/en/open-science-helping-make-science-accessible-all-canadians/departmental-open-science-action-plans

Government of Canada, I. (2023, April 13). Chief Science Advisor Annual Report 2020-21. https://science.gc.ca/site/science/en/office-chief-science-advisor/annual-reports/chief-science-advisor-annual-report-2020-21

Moreira, J. M., Pardelha, F., Laranjeira, C., Pereira, F., Novais, J., Lopes, P., Antunes, A., & Cardoso, E. (2022). Development of Plan S monitoring and compliance tool in the context of PTCRIS for Portuguese National Science Foundation. 211(C), 151–161. https://doi.org/10.1016/j.procs.2022.10.187

OECD Council. (2021, January 20). Recommendation of the OECD Council concerning Access to Research Data from Public Funding. https://www.oecd.org/sti/recommendation-access-to-research-data-from-public-funding.htm

Office of the Chief Science Advisory. (2020). Roadmap for Open Science. https://science.gc.ca/eic/site/063.nsf/eng/h_97992.html

Research Community Advisory Committee. (2006). Career Progression Management Framework for Federal Researchers. Application for the SE-RES Community. Government of Canada.

Ripp, C. (2023). Content Analysis of Canada’s Science Based Departments and Agencies Open Science Action Plans [Data set]. Zenodo. https://doi.org/10.5281/zenodo.7844158

UNESCO. (2021, November). UNESCO Recommendation on Open Science. https://en.unesco.org/science-sustainable-future/open-science/recommendation

White House Office of Science and Technology Policy (OSTP). (2022). Ensuring Free, Immediate, and Equitable Access to Federally Funded Research Memorandum. https://www.whitehouse.gov/wp-content/uploads/2022/08/08-2022-OSTP-Public-Access-Memo.pdf