Implementing Citizen Science within Open Science: Identifying Extra-Academic Skills, Collaborations, Rewards and Recognitions in the Context of a University

within the context of TU Delft University and its open science programme. The insights from this case-study resulted in a number of tips and tricks for researchers to help them improve the results of their research with an open citizen science project. It shows that various stakeholders (i.e., researchers, citizens, civil servants, and NGO’s) can benefit from open citizen science, it can be of great value to universities and open science in general.

opportunities.This has strong impacts on core academic processes like research, education, and innovation (NPOS, 2017).It is, for instance, easier to replicate an experiment if the relevant data sets are digitally available to any person who wishes to corroborate a researcher's findings.Furthermore, OS increases the opportunity for citizens or lay-people to interact or influence science.Citizen Science (hereafter: 'CS'), a form of research where non-professional scientists are actively involved in one or more parts of a scientific research effort, is often presented as an 'enabler' of openness in science (Suman & Pierce, 2018).It is therefore often recommended to consider CS and OS jointly (DITOs consortium, 2017;Wehn et al., 2020).The Citizen Science & Open Science Community of Practice (CoP), in the context of the UNESCO Recommendation on Open Science, identified the development of OS together with CS as mutually beneficial.Developing them together could be an important window of opportunity for laying the foundations of science in the future (Wehn et al., 2020).This CoP argued that there are shared characteristics between both CS and OS and recommended acknowledging CS as an important pillar of OS.This recommendation on OS has recently been adopted by UNESCO (International Science Council, 2021).
For CS to play an important role in OS, universities should take part in CS activities (Hecker et al., 2018).CS is connected to all aspects of OS and is recognised as an important aspect of OS in general (NPOS, 2020;Wehn et al., 2020;Kunst et al., 2021).However, CS is more than making methods, data, and results openly available.It invites everybody (e.g., society, citizens) to participate in the process of practicing science.This involvement can vary from participating in data collection, to being part of shaping the research question, analysing data, and publishing results (Hecker et al., 2018; 1. Introduction Open Science (hereafter: 'OS') aims to bring about socio-cultural and technological change to the scientific research process based on openness, reproducibility, and connectivity.OS is already a requirement in many leading universities in Europe and is becoming a norm worldwide (Morais et al., 2021).OS aims to change the way research is designed, performed, captured, and assessed by opening up the entire research process (Vicente-Saez & Martinez-Fuentes, 2018).It does not have a strict definition, but is defined by Vicente-Saez & Martinez-Fuentes (2018) as "transparent and accessible knowledge that is shared and developed through collaborative networks," while Gomez-Diaz & Recio (2020) define OS as "the political and legal framework where research outputs are shared and disseminated to be visible, accessible, and reusable." An argument for more openness of scientific research processes and results is, among others, that it brings an opportunity to accelerate innovation.Furthermore, it is the responsible thing to do, since most research is publiclyfunded, and transparency is considered to increase the reproducibility and reliability of results (Fecher & Friesike, 2014;Laine, 2017).Open Science is creating new forms of scientific interaction that generate new and unique NPOS, 2020).Involving citizens in all these steps requires additions and changes to conventional research processes and OS guidelines, which are not always supported by all universities, and researchers may lack the knowledge and skills to implement CS effectively and correctly.
As a result, university engagement in CS, especially when the goal is to implement it within OS guiding principles, faces several practical challenges (DITOs consortium, 2017).There have been several Communities of Practice and consortia trying to underline the synergies and future paths for CS within OS (DITOs consortium, 2017;Wehn et al., 2020).However, to the extent of our knowledge, there are no studies on the supporting mechanisms and tools for the adoption of CS in an OS and university context.The lack of literature and knowledge on this matter was also encountered by the Delft University of Technology (TU Delft) when setting up their OS programme.Working in a new and undiscovered territory, the positioning and implications of CS within the OS framework of TU Delft were explored in this paper.We aim to identify the implications of applying the OS guiding principles to a citizen science project executed by a university.Our main goal is to determine what a university could do to enable and support its research staff in good citizen science practices, especially within an OS context.Effectively, we want to find out what types of skills, collaborations, rewards, and recognitions are vital for practicing good and sound citizen science, and which are needed to deliver on the OS aspects within a citizen science project.

OS
Programme.We start by setting the scene explaining how the TU Delft OS Programme is built, and how citizen science is a part of this programme.Subsequently, we give insight into the CS project Delft Measures Rain that we have used as a casestudy to implement and reflect on the TU Delft They range from educational materials and software to a publishing platform.All outputs of the programme will be as 'FAIR' as possible (OpenAire, 2016).In addition to these six However, the European Citizen Science Association (ECSA) has proposed 10 principles for Citizen Science and added characteristics that attempt to represent a wide range of interpretations in an inclusive way (ECSA, 2015).This loose definition or description allows the inclusion of different types of participatory projects and programmes, where contextspecific criteria can be set for each project (ECSA, 2020).Like the approach taken by ECSA, the goal of the TU Delft OS Programme was not to give an enclosed definition of CS or even to define a universal set of rules for exclusion or inclusion, for fear of limiting the advancement of the field.Instead, the citizen science exploration at TU Delft developed a working definition specifically for the TU Delft OS programme, based on the ECSA 10 principles (Kunst et al., 2021) (Fig. 1).
projects, the programme includes three important cross-cutting themes as preconditions for the successful implementation of each project.These themes are: 1) ensuring appropriate rewards and recognition, 2) facilitating fruitful collaboration with third parties, and 3) gathering relevant skills for open science.

Operationalization of citizen science within the TU Delft Open Science Programme
CS is, like OS, an emerging movement within science, and there is no common, agreed-upon definition.
Figure 1.The working definition development by Kunst et al., (2021).This serves to advance the development of citizens sciencerelated activities at TU Delft (Kunst et al. 2021, reprinted with permission).

A Citizens Science Project: Delft Measures Rain
The  2020).The rain gauge was analogue and submissions were validated with an additional check based on a submitted picture for each measurement.The results would immediately be visible on an online, open data map, making it possible for citizen scientists to keep track of their own submitted data and that of others.Subsequently, the data were analysed by an undergraduate student under the supervision of researchers at the Water Management department.Updates and news were shared with the citizen scientists every two weeks via newsletters and when the project finished, the results were shared via infographics and a webinar.

Methodology
To determine what a university can do to enable successful CS practices within an OS context for its research staff and students, we followed a three-step process.We applied that process on the CS case-study DMR within the TU Delft OS Programme.
First, we needed to establish how the CS case study DMR performed according to the OS working definition of CS (as stated in Fig. 1).So, we reflected on the case study according to the following questions, where in principle the answer should hold a 'yes': 1. Did either citizens and/or scientists define the problem?
2. Did the project generate new knowledge or understanding, while using method?
the scientific 3. Was there a clear attribution of roles between the participants?
4. Did all participants in the project benefit from participation and where they acknowledged for their efforts?contained no electronic sensor, and measurements were submitted manually through an online data collection form.Data 5. Did all participants benefit from the results of the project?6. Were communication and tools tailor-made to the project?7. Was the project evaluated for its scientific output and data quality, the participant experience, and wider societal or policy impact?

Requirements of a CS project within OS
In Table 1 the results for the qualitative check performed on the DMR project are demonstrated.The project applies to all CS requirements as stated in the TU Delft working definition.
Second, during and after the DMR project took place, we evaluated in what manner the project implements the OS guiding principles.While setting up and executing DMR, we listed the deliverables that were achieved for each of the seven TU Delft OS projects.We also listed the deliverables that were missing to fully commit to these OS requirements.However, as said, CS goes further then only OS, and therefore we listed deliverables that go beyond OS needs, but can be essential for CS implementation.

4.
Did either citizens and/or scientists define the problem?

Yes
The problem definition came from a scientist, Dr. Ir. Marie-Claire ten Veldhuis, who investigates rain variability in urban settings.She saw the potential for establishing a dense network of rainfall measurements that could provide added information to that of existing rain radars.Did the project generate new knowledge or understanding?

Yes
Because DMR measures with a higher spatial representativeness than the KNMI (Royal Netherlands Meteorological Institute), the citizen science measurements are a good addition for measuring rainfall disparities within cities.As the data collected by citizens was accurate, it was concluded that there is an identifiable difference between the amount of rainfall between different parts of the city, with the north-western part of the city receiving slightly more rain than the rest of Delft (Timori, 2020).Besides delivering reliable data, the project has been a good test case for collecting and visualizing data in a more open environment.From a scientific point of view, DMR was a success and showed promising results for the application of citizen science in identifying high-resolution rainfall patterns Were scientific methods used to generate this new knowledge or understanding?

Yes
Generally applied and accepted manual rainfall measurement methods were used.Thereby accepting that there is an evaporation loss and errors when daily rainfall is below a threshold of 4 mm, because the goal was to focus on weather events that cause larger rainfall amounts, which are above this 4 mm threshold.WaterLab ensured logistics, communication, and management, the citizens took daily measurements and made observations, and students, supervised by the scientist, analysed the data Was there a clear attribution of roles between the participants?

Yes
The roles were divided such that the scientist defined the research question, S4W established and validated the measurement method.
Did all participants in the project benefit from participation and were they acknowledged for it?
Yes Acknowledgement took place in the form of this paper, in the other publications and during the webinar held at the end of the project.
Did all participants benefit from the results of the project?

Yes
We aimed to share the results with the citizen scientist in a way that was useful to them.The result of the project was shared among the citizens via a webinar and in the form of infographics.

Contributions needed to deliver
The deliverables listed in Table 2 were analysed according to the actors that ensured or created them, the skills these actors needed for that, the

Deliverables of DMR according to the TU Delft OS pillars
We list and structure the deliverables of DMR according to each TU Delft OS project in the table below (table 2).The table also demonstrates deliverables or items that were still missing in the project and that will need adjustment to enable full OS guiding principles to be implemented.
established collaborations, and finally rewards and recognitions that motivated achievement of these deliverables.These demonstrated in Figure 2.
the the are Were communication and tools tailor-made to the project?Yes A manual was adapted from existing methodologies to fit the project.Additionally, a targeted newsletter was provided on a twoweekly basis to communicate about the project.The recorded data was visible on an online data map and finally, all necessary tools to start measuring were provided to the citizens personally via mail.
Was the project evaluated for its scientific output and data quality, the participant experience and wider societal or policy impact?

Yes
The results of DMR were promising: the data submitted by the citizen scientists were of equal quality in comparison to data from digital sensors from the KNMI (Timori, 2020).
Since the evaluation, the project has also proven to be valuable for the citizen scientists.Many agree that the project was valuable (76.7%), interesting (63.9%), meaningful (78.3%), and fun (74.2%) (n=63).In addition, 65.1% of the respondents to the evaluation indicated that they would like to join a similar project again (based on scoring a 6 or higher on a 1-10 scale of "certainly not" to "definitely would").Furthermore, the data was shared with the municipality of Delft to increasing the value of the data collected.overall success of the project from a scientific and OS perspective.The needs and successes as

Discussion
Now that we have identified which actors, skills, collaborations, and rewards & recognitions contributed to effective implementation of this CS project within the TU Delft OS Programme, we should place them in perspective.The context of DMR is not generally applicable to all CS projects, and the way the TU Delft organises its OS programme might not be (fully) comparable to other universities.It is important to note here that this paper still primarily focuses on the interpreted by the participating citizens do not necessarily fully overlap with these goals.

Actors' contributions -roles in citizen science
and is an important factor to take into consideration when planning a project.To add to that, we would also like to emphasize that funding is an important aspect.With more budget, more time can be spent on training and acquiring new skills required for CS projects, engaging citizens, communication, and maing the project and materials look attractive.We even defined finding funding as one of the skills needed specifically for citizen science, due to the special needs CS has, which are often not yet covered within regular research funds.
First, we identified who acted and contributed to create each deliverable of DMR.Some deliverables are logical to expect from a specific actor.For this case study, researchers from TU Delft formulated the research question, decided on the general methodology and performed the data analysis.The extra-academic skills and elements were primarily executed by WaterLab.
It is important to mention that we have only looked at the needed skills and infrastructure, but not the amount of time needed to acquire those skills or build these infrastructures.The time needed for that can vary between projects It's worthwhile to ask the question: if we take time constraints into account, should we want and/or expect scientists to execute all the aspects of a CS project by themselves?Is that even feasible?As Hecker et al., (2018) mention, universities might be able to play a role to help researchers receive funding for acquiring these additional skills.We believe that a dedicated unit within a university can organise teams of people who could provide guidance and expertise along the research project, with training, support, or executing some of the tasks within a citizen science project where and when necessary.Additionally, we endorse the recommendations of When et al., (2020), aimed at maximising effective contribution of citizen science to OS, to "1) … [draw] on the vast practical experience within its communities, … 2) [foster] greater and enhanced cooperation, synergies, and crosspollination of practitioners among and between Citizen Science and Open Science communities, benefits, and drawbacks of CS.If CS indeed seems to fit the research question, the scientific methodology often needs to be adapted to be usable and applicable by non-experts, without losing scientific rigor.For instance, by swapping out highly exclusive and often costly sensors or equipment for more accessible and affordable methods and materials that provide similar or otherwise reliable results.In the DMR project, this resulted in the use of DIY-rain gauges, knowing and accepting that this would increase the measurement error and decrease its reliability with lower measurement values.
Additionally, data collection by citizens requires specialised infrastructure and software that is publicly accessible, usable, GDPR-proof (General Data Protection Regulation), and user-friendly.Relevant skills for doing CS research include knowledge and skills in engaging and motivating and 3) [ensure] global access to supporting technical networks." infrastructures, including participants, science communication and infrastructures, and community (Wehn et al., 2020, p. 1).
education, and know-how in visual design and relevant software (see Table 2 for more elaboration).Within the framework of DMR, the main role of the citizens has been to contribute to the data collection and recording, but we invited citizens to take part in more than that: five of the participants have provided input about their experiences and directly contributed to this paper as authors.Those are not the only roles a citizen can or should play in citizen science as there are many more options (Franzoni, Poetz & Sauermann, 2021).Possibly, when citizens take up other or more roles, the needed skills and associated tools might change.

Collaborations for citizen science projects
When realising that the DMR project needed partners that have skills, connections, or tools that the scientists and WaterLab did not have to successfully implement the project, we looked for third-party collaborations.To make sure all relevant parties were considered, and the best fitting parties were invited for collaboration, we performed a stakeholder analysis.We described all relevant stakeholders within the city of Delft and the benefits, drawbacks, and opportunities

Extra-academic skills for citizen science
With regard to an extension of the standard research skills towards extra-academic, it is important that a CS-project team should be able to decide whether the research question can be answered satisfactorily using a CS methodology.That requires understanding of the possibilities, for all parties involved.Ultimately, we collaborated with multiple parties from different parts of society: governmental (municipality of Delft), an NGO (S4W) and different departments within the university (communications, the library, and alumni relations).As can be seen in Figure 2, third-party collaborations could improve both the quality and reach of the DMR project.In this way, for many CS projects such collaborations will provide value and may even collaborations are usually interpreted as collaborating with businesses, start-ups, or other (profit-oriented) industry parties (Haslinger et al., 2019).However, in CS, the nature of the collaborating parties (e.g., often governments or non-profit parties) and their motivations for participating are often different.It is also important to notice that 'third party' in this case does not refer to a hierarchy, but to refer to collaboration with groups other than university-increase the long-term sustainability project. of the

and recognitions
As different skills, deliverables, and parties are required in CS, this requires different rewards and recognition mechanisms than standard research projects.For DMR, we paid attention to giving participating citizens the appropriate recognition by providing regular feedback on their data and intermediary results, thanking participants for their efforts and data, and acknowledging them in all publications about the project.Additionally, students involved with the projects were given the opportunity to share to-university collaborations.
Nevertheless, collaborations, data collection, and data analysis with other parties in general can have implications with regard to GDPR-regulations, data-ownership, or intellectual property rights.In the DMR project, we certainly came across them, especially in the form of the EU privacy regulations described in the GDPR and the strict limits on sharing data, even among collaborative partners.As queried by Suman and Pierce (2018), the data processing requirements under the GDPR represent a possible hindrance to the advancement of CS and OS, as they can make their results and experiences with the participants during a public webinar.Next to the specific OS goals, the researchers also obtained rewards for their efforts by being able to connect to new networks and by broadening their professional network via scientific conferences, like OS-and CS-focused symposia.Furthermore, the researchers got unexpected insights into the increased interest and awareness of their sharing data and information among collaborating partners more complicated or cumbersome.While respect and protect and privacy, it may it is very important to participants' information produce an undesirable disincentive to engage the public in research and broadly share research data, resulting in a hindrance to the progress of OS and CS.We want to emphasize that this should not discourage researchers to engage with citizen science.However, it is important that universities support their researchers on these aspects as well, as they would for other research collaborations.
research topics with the participants, indirectly increases the relevance of rainfall research.which urban

Ethical and legal challenges
While in this paper we focused on the practical challenges that occur when implementing CS within OS at a university, we would like to stress that there are also ethical and legal challenges to face (DITOs consortium, 2017).It is important that a researcher or project manager keeps the ethical and legal aspects of collaborations with other groups from outside the university, such as citizens, NGOs, or governments at heart.Within the TU Delft OS Programme, third party

The way forward
Universities can contribute to practicing sound citizen science by providing professional infrastructure, knowledge and skills, ethical and legal background, educational facilities for citizen scientists, sustainable teaching, and funding (Hecker et al., 2018).Although there are many challenges in moving to an open science environment for universities, the most difficult change needed seems to be a cultural change to seeing CS not as an exception, but one of many variations in academic research methods.A programme of change management needs to accompany and support any move to OS to decide which mix of policies, measures, support systems, and engagements best supports their missions and implementation strategies (Ignat & Ayris, 2020).By adopting and supporting CS and implementing OS guiding principles while doing so, universities in turn gain breadth and strength in research, which consolidates their position and recognition in society, brings new resources, and increases public trust in universities (Hecker et al., 2018;Wehn et al., 2020;Kunst et al., 2021, Hall et al, 2022).Currently, many tools and information sources to make scientists more familiar and knowledgeable about general OS practices are being developed at TU Delft.Also researchers within the university to perform effective and open citizen science: 1. Sufficient (additional) time and budget to implement these elements in the citizen science project, or support in finding additional budget suitable for science projects;  2018;European Commission, 2021).Since CS is recognised as an integral part of OS, it is vital that these tools and sources are also applicable to CS.

7.
Training in data-validation methods for data collected with a citizen science methodology; 8. Support in logistics (e.g., assembling and sending out research kits); Taking the results of our study, we would therefore like to encourage the TU Delft and other universities to further develop support, infrastructure, and tools for scientists and research departments when it comes to good CS and OS practices.Through the DMR project, we 9. Support in finding suitable 3rd party collaborations, especially outside the university environment; The Evolving Scholar 16

Conclusion
We used the CS project Delft Measures Rain as a case-study to reflect on TU Delft OS Programme and the UNESCO OS guiding principles in order to establish what is needed to practice CS successfully and in accordance with the OS goals and guidelines of TU Delft.We established that DMR reflected the TU Delft CS definition and requirements.
We identified which (extra)academic skills, rewards, recognitions, and collaborations were needed to make the CS project deliver on both the CS and OS goals.A good understanding of the communicational, educational, and other requirements for CS, as well as a supporting toolkit and infrastructure can aid researchers in setting up and executing citizen science projects.Additional rewards and recognitions on top of the standard academic rewards (such as a published article in reviewed journal) are needed to motivate researchers to invest their time and efforts.It is needed to provide them with the appropriate amount of time and budget to make the project work, while at the same time acknowledging the efforts and input from the participating citizens.Based on our findings, we provide a list of 10 recommendations for universities as to how they can support practicing good, legal, and open CS by their researchers.We recommend universities that are interested in developing their CS and OS programme to start working on providing and developing the supporting tools, infrastructures, rewards, and recognitions to embrace CS as a valuable asset to their research and public engagement programmes.

Data availability statement
Not applicable, no dataset was created for this paper.

Contribution statement
Description of the author contribution and all types of contribution Marit Bogert: executing DMR project (project management, communication, education).Part of the citizen science research team in the Open Science programme at TU Delft, supplying content to this paper, writing paper.Sandra de Vries: executing DMR project (project management, developing methodology, set-up and uphold contact with collaborative partners), supplying content to this paper, writing paper.
Sabine Kunst: member of the citizen science research team in the Open Science programme at TU Delft, writing content, providing input, and comments to this paper.
Nicoleta Nastase: member of the citizen science research team in the Open Science programme at TU Delft, writing content, providing input, and comments to this paper.
Winke Kloosterman: Citizen scientist in Delft Measures Rain 2020 and 2021, collecting data for DMR, providing input, and comments to this paper.
Karen Knols: Citizen scientist in Delft Measures Rain 2020 and 2021, collecting data for DMR, providing input, and comments to this paper.
Science includes guiding principles for OS to provide a framework for enabling conditions and practices within which OS values are endorsed, and the ideals of open science are made a reality.These ideals are (a) transparency, scrutiny, critique, and reproducibility; (b) equality of opportunities; (c) responsibility, respect, and accountability; (d) collaboration, participation, and inclusion; (e) flexibility; and (f) sustainability (UNESCO, 2021).
added later as a 7th project after an additional exploration of the subject(Kunst et al., 2021).The projects aim at creating and disseminating various types of resources for the benefit of TU Delft researchers, teachers, and students, as well as the public (Haslinger, 2019).
2020 project Delft Measures Rain (DMR) was developed by the Citizen Science platform WaterLab in collaboration with TU Delft scientist Dr. Ir. Marie-Claire ten Veldhuis and external partners, including Smartphones4Water (S4W), the municipality of Delft, and several internal departments within TU Delft.WaterLab is an external consortium consisting of Science Centre Delft (science museum), IHE Delft, the local waterboard Delfland, and PulsAqua, a small consultancy company.WaterLab has plenty of practical experience in citizen science projects concerning water quality and quantity, using the distinct expertise from all the four partners (WaterLab 2022).Likewise, S4W already had experience with developing and coordinating citizen science projects.Citizens of the city of Delft were encouraged to participate and work together with scientists and students from the TU Delft Water Management department to investigate rainfall patterns within the city.The DMR project answered two main research questions, 1) What is the quality of the data collected from the citizen rain gauges?and 2) How is rainfall spatially distributed over the city of Delft?In total, 95 participants across Delft participated in the project, submitting a total of 1991 measurements between July 17 and September 14, 2020.Each participant received a kit with a manual on how to build their own rain gauge and instructions for taking and submitting their measurements.The data collection method was based on the methodology developed and internationally applied by S4W, with rain gauges made from accessible materials like (used) soda bottles and concrete (Smartphones4Water, Finally, each deliverable was then analysed to identify what ensured a successful creation and execution required to create the deliverables;distinguishing between skills that are already part of the general academic skillset that is used in research, and skills that we define as extra-academic.Collaborations playing a role in creating deliverables.Rewards and recognitions needed to motivate different actors.3.

© 2021
Bogert, de Vries, Kunst, Nastase, Kloosterman, Knol, Schleiss & Werkman.This work is licensed under a Ceative Commons Atribution 4.0 International (CC BY 4.0) license.Not yet achieved:1.Most of the information is now only findable in Dutch.English is still largely missing.2. The dataset should be stored in the TU Delft Research Data repository for accessibility -Use of an open-source application (ODK collect) -Reusable: the online forms used for data collection (developed by S4W) are open for use by anyone who wants to contribute.Open Hardware -Citizens made their own rain gauge, based on open available instructions on the S4W website.S4W website -Citizens used their own smartphone to record data.-Improvements to the rain gauges and data collection were made, based on the methodology developed by S4W, and therefore directly shared with S4W Citizen Science -Target group analysis.-Recruitment and engagement using flyers, press releases, and social media posts.-Visualization of data and results.-Project evaluation.-Development and distribution of measurement kit.-Budget to implement the project -Helpdesk during the project -Cooperation with third parties

Figure 2 .
Figure 2. Actors roles, extra-academic skills, collaborations, rewards and recognitions identified according to deliverables of the DMR project 10. Training and/or support in correctly rewarding and recognizing the efforts of the citizen scientists.have identified multiple extra-academic elements that are essential to good, open, citizen science practice, but are not yet covered by general TU Delft OS practices and developments.From these, we can deduce what training, support (e.g., infrastructure or tools) and facilities are needed to stimulate and support At this moment, the need for and content of legal and ethical protocols, guidelines, tools, training, and/or support for CS applications are being explored by the OS citizen science team at TU Delft.

Table 1 .
Qualitative check performed to indicate if the DMR project applied to all the requirements in the TU Delft CS working definition.

Table 2 .
Overview of the deliverables of Delft Measures Rain for each pillar of the TU Delft Open Science Program