Funding projects for Spanish public universities in research, development, and innovation related areas

Thamyres T. Choji^*, J. A. Moral-Munoz ^**, and M. J. Cobo ^***

^*choji@uca.es

ORCID 0000-0002-8158-6124

Department of Nursing and Physiotherapy, University of Cadiz, Spain

^** joseantonio.moral@uca.es

ORCID 0000-0002-6465-982X

Department of Nursing and Physiotherapy, University of Cadiz, Spain

Institute of Research and Innovation in Biomedical Sciences of the Province of Cadiz (INiBICA), Cadiz (Spain)

^*** mjcobo@decsai.ugr.es

ORCID 0000-0001-6575-803X

Department of Computer Science and Artificial Intelligence, Andalusian Research Institute in Data Science and Computational Intelligence (DaSCI), University of Granada, Spain

This contribution examines the integration of Science of Science and data visualization to acquire insight into the scientific landscape of funding projects for Spanish public universities, as well as to detect the research, development and innovation related areas and analyse its development by universities. We analyse 18,423 Spanish funding, specifically in public universities. The performance analysis was based on granted projects, budget, paper production and the relative specialisation index. The findings show that Chemical Sciences and Technologies and Biomedicine receive the highest number of projects, budget and have high production rates, while Computer-based Technologies also showed high production, although does not receive as much funding. In contrast, Gender and Women studies demonstrate low investment and research output. Our analysis contributes to broaden knowledge about the national scientific production, and it could be used to detect strengths and weaknesses in the development of science in Spain.

1. Introduction

The measure of scientific knowledge is often based on its papers production and its impact, and the analysis of different fields at a global or local level is a common approach to understand how knowledge is organized (Fortunato et al., 2018). This information, related to the topics covered by each field, is useful in understanding how knowledge is configured and structured at a global level. At the local level, it represents an understanding of the country’s participation in the construction of knowledge. Likewise, with the development of science itself, its production increase in number and complexity and the Science of Science (SoS) becomes fundamental to understand, quantify and predict this complex system (Zeng et al., 2017).

In fact, the Nature publishing group annually publishes a study based on the scientific production of the previous year. The analysis focuses on the main institutions responsible for the production of papers and the areas of knowledge which are the highest production in each country (“Nature Index,” s.f.). This knowledge allows understanding at a large-scale level and determines who and where handles scientific production in a country. Then, the SoS is used to study scientific knowledge from different perspectives. Given this potential, it is essential to consider this field as valuable for understanding the science landscape and supporting the policymakers in strategic decisions, such as developing resource allocation policies and understanding the impact of specific policies for example. However, due to the complexity of the science system, accurate measurement and assessment of scientific knowledge transfer is challenging. To address this complexity, various indicators have been developed to increase the accuracy of measurements (Donovan & Butler, 2007). However, any method of measurement has inherent potential limitations. For instance, metrics based solely on paper production and impact are not suitable for comparing academics from different fields, this is due to potential variations in the behavior of these variables within each research field, and which could imply an unfair comparison between them (Zeng et al., 2017). Therefore, it is recommended to consider multiple variables, in addition to those mentioned above, as well as a thorough understanding of what needs to be measured, in order to select or develop an approach that provides a more accurate assessment of scientific research.

The complexity of the evaluation process has been addressed by several authors from different perspectives. On the one hand, evaluation can be analysed by accessing different research fields, analyzing and/or developing indicators specifically tailored to examine the landscape of each field (Butler, 2008). On the other hand, evaluation can be performed by analyzing the funding projects awarded in different research fields and their impact on each area that received a grant (Cattaneo, Meoli, & Signori, 2016; Thomas, Nedeva, Tirado, & Jacob, 2020).

To the best of our knowledge, there is no previous analysis of the funding projects in Spain. Therefore, in order to address one of these perspectives of research evaluation, with this contribution, we propose merging SoS and data visualization to analyse funding projects related to research and development (R&D) and research, development, and innovation (R&D&I), identifying fields of knowledge and its behaviour according to the number of research projects awarded, papers produced, budget received, investment per paper, and specialisation rate in Spanish public universities. The focus of the study was public universities, given the relevance of understanding what is at the core of their investment and production, identifying strengths and weakness.

2. Methodology

We propose a methodology based on four phases: i) Data acquisition, ii) Data filtering, iii) Performance analysis, and iv) Visualisation.

Data acquisition

In order to analyse funding projects for Spanish public universities in R&D&I related areas, we retrieved projects related to three calls issued by the Spanish State Research Agency: i) excellence/knowledge generation in R&D, ii) knowledge generation and research challenges in R&D&I, and iii) research challenges in R&D&I. These data were collected in the Spanish State Research Agency, a Spanish agency focused on promoting the development of high-impact scientific research. The query was performed in March 2023, to retrieve funding information for projects granted in the period between 2013-2021 and retrieved 26.066 unique projects that contained information about the projects call, the institution that received the funding, the funding identification, the field of knowledge covered by the project, the autonomous community where the institution that received the funding is based, and the amount received. To enrich the further analysis, the total of papers related to each funding project was retrieved through a query performed in the Scopus database using the grant number supporting the work.

Data filtering

To ensure that observations are as precise as possible, we focused our analysis on Spanish public universities that fulfil analogous scopes of work and activities. After data collection, the public universities in Spain were identified based on the list of recognised universities by the Spanish authorities (Ministerio de Educación, Cultura y Deporte, 2008). The dataset used is can be downloaded at DOI: 10.6084/m9.figshare.22643872.v1 (Choji, Moral-Munoz, & Cobo, 2023a).

Performance analysis

In order to analyse the performance of the public universities in Spain, our research is based on the number of national projects received by each university, the amount of funding represented by these projects, the number of papers produced in each field, the average cost of each paper, and the relative specialisation index (RSI). The RSI is an indicator used to measure the research profile of a country in a field of knowledge and compare its specialization rates in production with worldwide production (Aksnes, van Leeuwen, & Sivertsen, 2014). In the current research, we adapted the RSI using the metrics related to each university and the total production of all the public universities. This adjustment allows us to standardize the criteria for comparison in the degree of specialization of the different public universities at the national level and among universities.

All these metrics were calculated for each public university to provide a more specific view of each field of knowledge in the national scenario. With these metrics, we can evaluate and compare the scientific production in different fields and link this information with the granted projects, estimating each paper’s costs and detecting the specialisation rate for each university in different fields. The RSI calculation is based on the Thematic Specialization Index (TSI):

\[TSI = \ \frac{a/b}{c/d}\]

Where a = number of publications of university X in area Y; b = number of publications of university X in all areas; c = number of publications from all universities in area Y and d = number of publications of all universities in all areas. Then,

\[RSI = \ \left( \frac{TSI - 1}{TSI + 1\ } \right)\]

The RSI range from -1 to 1, with 0 representing the mean output of universities in the field. The RSI lower than 0 means that the output rate is lower than the average, while RSI greater than 1 represents that the output rate is higher than the average.

Visualisation

To display the relationship between the areas of knowledge and the public universities that received funding in the computed metrics, a set of five heatmaps was generated. A heatmap is a type of graph that uses colour to represent numerical values in a matrix or table, representing multivariate data and aiding in the visualization of the relation between variables (Gehlenborg & Wong, 2012). By using colour to represent values, we can quickly identify areas of high and low activity and compare them across different dimensions. Each column in the heatmap represents the areas of knowledge for which the project was funded, while each row represents the public university that received funding for such projects. Using three colours gradient allows to highlight the low and high values and makes it more visible to the reader. Here, the colour green indicates high values, the yellow represents intermediate values and the grey the low ones.

3. Results

To create comprehensive analysis of Spanish scientific research in the public university scenario, this study analysed 18,423 funding projects. Projects that did not meet the criteria of being from a public university were excluded to prevent bias. Therefore, the funding projects were awarded to 48 different universities and focused on 39 different fields of knowledge. The Spanish government invested €1,932 billion between the years 2013 and 2021.

As previously indicated, this study was conducted to construct and comprehend the national landscape of investment in R&D&I. Accordingly, the number of projects, research papers, investments, investment per paper, and RSI were computed and are presented in Figures 1-5. To view them with high-resolution, please visit DOI: 10.6084/m9.figshare.22656988.v1 (Choji, Moral-Munoz, & Cobo, 2023b).

It is possible to observe that investments occur in different areas of knowledge, and the papers production varies across each university. The areas with the highest number of national projects in R&D&I are Chemical Sciences and Technologies, Biomedicine, Law, and Environmental Sciences and Technologies. Chemical Sciences and Technologies receive the second highest amount of funding for research and have the highest paper production in the country. Biomedicine is the area that receives the highest amount of funding, although the cost of investment per paper is one of the most expensive, close to papers produced in Energy and Transport, and in Bioscience and Biotechnology. Computer-based Technologies is the area with the second highest papers production in the country and the number of projects received is at the mean compared to other areas. However, in this area, the amount of investment earned and the investment per paper are considerably lower compared to other universities.

The areas of knowledge with a lower number of projects and with lower investment are Gender and Women Studies, followed by Sports science and Astronomy and Astrophysics. The areas with lower paper production are Gender and Women studies, Culture: Philology, Literature and Art and Law.

At the university level, the Universidad de Barcelona received the highest number of projects, budget, and paper production. Regarding the RSI, the highlighted areas are Biomedicine, and Astronomy and Astrophysics, although the conceived projects and papers production were mainly in Biomedicine and Chemical Sciences and Technologies. In the sequence, we observe the Universidad Complutense de Madrid, in which the highest production is in Chemical Sciences and Technologies, Biomedicine, Computer-based Technologies and Materials. In the same university, the areas with the highest RSI were Gender and women studies, Physics science and Particle physics and accelerators. On the other hand, the Universidad Politecnica de Cartagena is one of those that received the lowest number of R&D&I projects. Nevertheless, it demonstrated the highest investment per paper, highlighting the cost of paper in Space Research and Materials Sciences and Technology. In contrast, the Universidad de Burgos, the Universidad de la Rioja, the Universidad de Huelva and the Universidad Politecnica de Cartagena received the lowest number of projects. The first three presented the lowest budget and the lowest paper production, with less than 600 articles.

4. Discussion

In this section, we discuss the results and present insights on research funding projects in Spanish public universities. Our aim was to combine SoS with data visualization to identify the research fields and public universities that received grant projects in R&D and R&D&I from a national perspective. This study reveals highlighted areas of knowledge based on the number of funding projects received, budget, production, and RSI. Our heat maps enable comparisons between the same research fields across different universities, providing a comprehensive view of national R&D&I projects and enhancing our understanding of funding distribution.

The areas Chemical Sciences and Technologies, Biomedicine, and Biotechnology have shown high income for funding projects. Chemical Sciences and Technologies achieved one of the highest papers productions in Spain. These results align with the findings of 2022 year achieved by the Nature publishing group (“Nature Index|Country/territory outputs|Spain,” s.f.). According to Yan E. (2015), Chemical Engineering-related fields demonstrate the fastest growth in scientific impact, achieving recognition and diffusion in a short period. Moreover, our study revealed that Electronic and Communication Technologies and Computer-based Technologies had the highest production and RSI. Both areas are considered to be Science, Technology, Engineering and Mathematics (STEM) and have significant potential for application and growth. At university level, it is interesting to note that even in cases of relatively low investment, such as the Universidad de Burgos, Universidad de la Rioja and Universidad de Huelva, Chemical Sciences and Technologies was the field that received the highest number of funded projects and budget allocation. In all three cases, the budget allocated to this specific field was the highest compared to other fields, and the paper production in this area was higher than in other fields explored within each respective university.

In contrast, Gender and Women Studies showed been underexplored by Spanish public universities. Only twelve Spanish universities received funding in this research field, which could explain the low investment, RSI, and paper production in projects related to R&D&I in this area.

Regarding areas with low RSI, such as Law and Mind, Language and Thought, they predominantly belong to humanities-related fields. Some studies have reported significant differences in the pattern of production and citation within this field compared to others, and the metrics used for evaluation need to fit to the analysis performed (Ochsner, Hug, & Daniel, 2016). These differences may be due to the fact that knowledge production in different fields contributes to different publication categories, including books, book chapters, journal articles, and conference papers (Butler, 2008). Therefore, to improve the assessment of different research fields one of the perspectives is to study the knowledge transfer between them. This suggests that using of the RSI to directly compare Formal, Natural and Applied-sciences to Social-sciences could be imprecise and may not represent the reality. Additionally, when the analysis is focused on citations rates, it is essential to consider that the works achieve the highest citation level approximately 7 years after the publication (Yan, 2016). Likewise, the time could affect similarly in the project’s papers production, which means that the progress of funding projects awarded between 2020-2021 may not have been fully observed yet, as it may take some time for scientific articles to be published. However, this potential delay applies to all areas and projects analysed and does not significantly impact the findings of the analysis.

Therefore, it is as important to study the complexity of both the scientific structure and the potential impact of funding projects as it is to develop methodologies to evaluate them (Kenkmann, Schumacher, Freiburg, Eisenmann, & Muckenfuss, 2020). This is because different circumstances can lead to different results. For example, when evaluating the effects of specific policies across different universities, Cattaneo et al. (2014) demonstrated that universities with high legitimacy perceived greater benefits in terms of output production and performance in response to funding mechanisms compared to other universities (Cattaneo et al., 2016). Regarding the management of funding projects, Butler et al. (2001) analysed their application in terms of the reward system and observed an increase in scientific output despite a decrease in quality in the Australian context (Butler, 2003). These observations may have implications for the performance of both the funded project and the universities themselves. It is therefore essential to develop an evaluation infrastructure that considers the complexity of the fields and the behavior of universities, with the aim of the most accurate evaluation results possible.

Our approach and findings have the potential to benefit both researchers directly involved in the projects and individuals not directly involved in the research. On the one hand, the use of heat maps improves the efficiency of the benchmarking process, thereby facilitating the identification of potential collaborations. On the other hand, our findings can be used to support policymakers in their resource allocation decisions, to serve as a data source for critical studies assessing the impact of specific policies, and to provide insights into the expertise of different fields and universities in a country-level perspective (Thomas et al., 2020), among others.

Although these findings are relevant, it is necessary to report and discuss some limitations. Our study included some measurements based on research output, but science is marked by inequality, random chance, and other factors that make it difficult to measure the impact of research (Bornmann, 2017). In addition, research output can be influenced by factors, such as gender, age, academic qualification, academic rank, and other characteristics (Armijos Valdivieso, Avolio Alecchi, & Arévalo-Avecillas, 2022). In this respect, the metrics used to compare different areas may have a negative impact on the humanities-related fields, and the evaluation criteria used in this contribution may not accurately represent achievement in those areas. Additionally, we were unable to conduct a gender analysis of the principal researcher due to a lack of information. In future research, we aim to cover the impact of the projects by looking at the citations received by the papers in order to better understand the performance achieved in each field and to identify the gender balance in funding projects in Spain.

5. Conclusions

This contribution analyses the funding projects for Spanish public universities in areas related to R&D&I research, from 2013 to 2021. As demonstrated, the highest budget, granted projects and production were found in Chemical Sciences and Technologies and Biomedicine, while Computer-based technologies also has higher production, although low funding. In contrast, the area Gender and Women studies received few projects, investment, and just a few Spanish universities conducting research in R&D&I in this area. Regarding the metrics presented in this contribution, it is relevant to acknowledge that these indicators are imprecise for analysing the performance of humanities-related fields and, that comparing disciplines within Formal, Natural and Applied Sciences to those within Social Sciences can be challenging due to differences in production patterns. The findings highlight the potential of integrating SoS and data visualization as a powerful tool to improve understanding to the Spanish funding landscape. The implications of these results are diverse, such as providing a valuable resource for identifying potential collaborations based on research fields within the country, initiating discussions on the impact of funding projects, and potentially assisting policymakers and government agencies in formulating science policies.

Open science practices

The data used and Figures presented are available in the supplementary material (dataset: https://doi.org/10.6084/m9.figshare.22643872.v1; Figures: https://doi.org/10.6084/m9.figshare.22656988.v1)

Author contributions

Thamyres T. Choji: Data curation, Investigation, Methodology, Validation, Visualisation, Writing – original draft, Writing – review & editing.

Jose A. Moral-Munoz: Conceptualisation, Funding acquisition, Investigation, Methodology, Validation, Visualisation, Writing – review & editing.

Manuel J. Cobo: Conceptualisation, Funding acquisition, Investigation, Methodology, Validation, Visualisation, Writing – review & editing.

Funding information

This contribution is part of the Spanish State Research Agency through the project PID2019-105381GA-I00/AEI/10.13039/501100011033 (iScience) and is part of the first author (Choji, Thamyres T.) thesis. In addition, the authors are thankful for the funding received by the University of Cadiz and University of Granada for conference attendance as part of the "Plan propio 2022-2023".

References

Aksnes, D. W., van Leeuwen, T. N., & Sivertsen, G. (2014). The effect of booming countries on changes in the relative specialization index (RSI) on country level. Scientometrics, 101(2), 1391–1401.

Armijos Valdivieso, P., Avolio Alecchi, B., & Arévalo-Avecillas, D. (2022). Factors that Influence the Individual Research Output of University Professors: The Case of Ecuador, Peru, and Colombia. Journal of Hispanic Higher Education, 21(4), 450–468.

Bornmann, L. (2017). Measuring impact in research evaluations: A thorough discussion of methods for, effects of and problems with impact measurements. Higher Education, 73(5), 775–787.

Butler, L. (2003). Explaining Australia’s increased share of ISI publications—The effects of a funding formula based on publication counts. Research Policy, 32(1), 143–155.

Butler, L. (2008). ICT assessment: Moving beyond journal outputs. Scientometrics, 74(1), 39–55.

Cattaneo, M., Meoli, M., & Signori, A. (2016). Performance-based funding and university research productivity: The moderating effect of university legitimacy. The Journal of Technology Transfer, 41(1), 85–104.

Choji, T. T., Moral-Munoz, J. A., & Cobo, M. J. (2023a). Data funding projects—Spanish public universities. Retrieved from https://figshare.com/articles/dataset/Data_funding_projects_-_Spanish_public_universities/22643872

Choji, T. T., Moral-Munoz, J. A., & Cobo, M. J. (2023b). Heat Maps—Spanish public universities. Retrieved from https://figshare.com/articles/dataset/Heat_Maps_-_Spanish_public_universities/22656988

Donovan, C., & Butler, L. (2007). Testing novel quantitative indicators of research “quality”, esteem and “user engagement”: An economics pilot study. Research Evaluation, 16(4), 231–242.

Fortunato, S., Bergstrom, C. T., Börner, K., Evans, J. A., Helbing, D., Milojević, S., Petersen, A. M., et al. (2018). Science of science. Science, 359(6379), eaao0185.

Gehlenborg, N., & Wong, B. (2012). Heat maps. Nature Methods, 9(3), 213–213.

Kenkmann, T., Schumacher, K., Freiburg, O.-I., Eisenmann, L., & Muckenfuss, L. (2020). Municipalities in focus: Evaluating the Local Authorities Guideline (LAG) within the National Climate Initiative (NCI) of Germany: Challenges and findings.

Ministerio de Educación, Cultura y Deporte. (2008, September 25). Registro de Universidades, Centros y Títulos (RUCT). Retrieved April 17, 2023, from https://www.educacion.gob.es/ruct/home

Nature Index. (s.f.). . Retrieved April 18, 2023, from https://www.nature.com/nature-index/

Nature Index|Country/territory outputs|Spain. (s.f.). . Retrieved April 18, 2023, from https://www.nature.com/nature-index/country-outputs/Spain

Ochsner, M., Hug, S. E., & Daniel, H.-D. (Eds.). (2016). Research Assessment in the Humanities. Cham: Springer International Publishing. Retrieved from http://link.springer.com/10.1007/978-3-319-29016-4

Thomas, D. A., Nedeva, M., Tirado, M. M., & Jacob, M. (2020). Changing research on research evaluation: A critical literature review to revisit the agenda. Research Evaluation, 29(3), 275–288.

Yan, E. (2016). Disciplinary knowledge production and diffusion in science. Journal of the Association for Information Science and Technology, 67(9), 2223–2245.

Zeng, A., Shen, Z., Zhou, J., Wu, J., Fan, Y., Wang, Y., & Stanley, H. E. (2017). The science of science: From the perspective of complex systems. Physics Reports, 714–715, 1–73.