Scientific leadership and collaboration can predict the academic popularity of highly-cited university’s publications indexed in WoS from an Andean country

Scientific leadership and collaboration can predict the academic popularity of highly-cited university’s publications indexed in WoS from an Andean country

Carlos Vílchez-Román^*, Alejandra Manco^**

^* cvilchez@pucp.edu.pe

https://orcid.org/0000-0002-6802-053X

CENTRUM Católica Business School / Pontificia Universidad Católica del Perú (PUCP), Peru

^** alejandra.manco-vega@univ-lyon1.fr

https://orcid.org/0000-0002-8168-4074

Université Claude Bernard Lyon 1 (UCBL), France

Abstract

We examined if collaboration, scientific leadership, and expert knowledge predict citation using WoS-extracted data from Andean country universities. We analyzed data for the last six years (2018-2022) for the three main WoS indexes: SCIE, SSCI, and AHCI. Given the over-dispersion of the predicted variable, we worked with a negative binomial regression and found that scientific leadership is a primary citation predictor, followed by collaboration. Scientific leadership is a relevant issue if research teams from Andean countries plan to have some control over research agenda-setting and provide significant contributions to knowledge diffusion and expansion.

Keywords: collaboration, scientific leadership, citation, responsible metrics, Peruvian universities

Corresponding author:

Carlos Vílchez-Román. Phone: +511 992 726 099. E-mail address: cvilchez@pucp.edu.pe

1. Introduction

From an open science perspective, responsible research assessment has become a predominant topic because of its emphasis on accountability, transparency, and replicability (Das & Dutta, 2020; Garfinkel, 2021). The premise behind this logic is that research output contributes to development agendas or knowledge expansion. Given that examining research institutions’ contribution to development agendas initially requires defining the notion of development agenda, studying knowledge expansion drivers seems a more attainable goal.

Despite several definitions of knowledge diffusion (Abramo et al., 2020; Yang et al., 2022), it is generally accepted that studies published in prestigious journals are those academic works that contribute to knowledge diffusion and expansion. Therefore, the reasoning seems straightforward: the higher the scientific output published in Scopus or Web of Science (WoS) first-quartile journals, the higher the contribution to knowledge diffusion and expansion. From a global north countries researchers’ perspective, this relationship admits no doubt. However, from a global south countries researchers’ approach, there is a different landscape since local research teams only sometimes control research agenda-setting (Kreimer, 2006). The reason for this lack of control is the already known research gap between the global north and south countries, which makes it visible through an intense international collaboration with global north countries, and to a lesser degree with global south ones (Jamil & Haque, 2016). Due to the nature of scientific and technical advancements, there are significant gaps between North and South universities and research organizations. Such disparities and inequalities seriously hinder effective and productive collaboration in joint research and knowledge-generation initiatives.

This study examines whether scientific leadership, collaboration, and expert knowledge can predict academic popularity, measured through citation counting in WoS (SCIE, SSCI, and AHCI indexes). To test the hypothesis, we analyzed article-based production from an Andean country that had ended its 2014-2015 university reform.

2. Conceptual model

From a Bourdieusian perspective, scientific research communities conduct studies to increase their symbolic capital, which has several dimensions, like knowledge diffusion and expansion. However, this increase is not automatic; it requires members of those scholar structures to mobilize their social capital, a product of their position within a research community social structure, for example, positioned in the core zone or the periphery (Bourdieu, 1988). To achieve this goal, research communities’ members interchange shares of symbolic power deployed into roles like the designated corresponding author, given that this role usually belongs to the scholar who controls the research agenda-setting. For this reason, it is primary to examine the effect of scientific leadership, among other factors like collaboration (Kato & Ando, 2013; Ni & An, 2018; Van Raan, 1997) and expert knowledge (Bornmann et al., 2012; Larivière & Gingras, 2010; Oppenheim, 1995) on the academic popularity measured by citation counting.

Regarding the definition of scientific leadership, operationally, this concept translates into the number of documents in which the corresponding author has an institutional affiliation with a university or another research institution. It is important to note that scientific leadership can be examined at different levels (e.g., individual, institutional, country) or publication outlets: journals or articles (Azadi & Rasuli, 2022; Moya-Anegón, 2012). Since scientific leaders usually come from the global north universities and countries (Editor, 2022; Kahalon et al., 2022; O’Grady, 2022; Purnell, 2023), the measurement of this leadership becomes essential to understand the research gap among countries. Furthermore, recent studies reveal that scientific leadership could be one of the drivers explaining the research capacity gap between the global north and south countries (Aegina et al., 2020; Huang & Yue, 2022).

Regarding the other variables of the conceptual model, universities’ scientific output is a well-studied issue, especially output from Asian and European universities (Agasisti et al., 2021; Demeter et al., 2022; Kifor et al., 2023). There is a similar interest in Latin American and Caribbean (LAC) universities. However, even though several LAC countries increased their scientific output in the last years, Argentina, Brazil, and Mexico concentrated the highest volumes of accumulated scientific output (Maldonado-Maldonado & Cortes-Velasco, 2021; Nunes et al., 2022; Pozzo et al., 2022).

In LAC, the primary source for measuring scientific output is institutional affiliation rather than scientific leadership. Peruvian research officers apply the same criterion. However, the university affiliation does not necessarily reflect the university’s contribution to knowledge diffusion and expansion. The primary limitation of institutional affiliation data is that this indicator is one of the most accessible indicators to obtain. Still, it does not necessarily reflect whether the university controls the research agenda, at least from the perspective of the global south countries. As mentioned, the university scientific output is obtained from the institutional affiliation field of authors listed in a study (Bachelet et al., 2019; Hottenrott & Lawson, 2017, 2022) instead of the corresponding author data. Unless scientific leadership data is not registered appropriately, it is hard to determine the university’s contribution to knowledge expansion.

We selected Peru as a case study since the government implemented a university reform to improve higher education quality and research capacity (Benavides et al., 2016; Gallegos, 2022). The reform implementation concentrated in 2014-2015 and created a legal framework that ensured Peruvian universities met quality conditions to operate as licensed universities.

Following this literature review, the research hypothesis guiding this study is: Scientific leadership, collaboration, and expert knowledge predict the citation-counting-based popularity of Peruvian universities’ recent publications (2018-2022).

3. Methods

3.1 Information source and data collection

We examined data from Peruvian universities’ publications indexed in WoS for the last five years (2018-2022). First, we selected WoS primary indexes: Science Citation Index Expanded (SCIE), Social Sciences Citation Index (SSCI), and Arts & Humanities Citation Index (AHCI) because –besides Scopus– they constitute the science mainstream. Then, we executed a basic search command [PY=(2018-2022) AND (CU=Peru)] and filtered by document type article. In the end, we collected data from Peruvian universities’ publications (n = 182) with at least 46 citations in WoS.

We operationalized the three predictor variables this way: collaboration as the number of co-authors mentioned in the co-authorship field, scientific leadership as a dichotomous variable with two values (0 = university’s researcher was not designated as the corresponding author, 1 = researcher was designated as the corresponding author), and the expert knowledge as the number of references cited in the study indexed in WoS. Given that some publications had a very long list of co-authors, we also used a feature available in Scopus to obtain the number of co-authors when the list of co-authors was larger than 15. Additionally, we registered the digital object identifier (DOI) for each publication. Finally, we recorded WoS data on a Google Drive spreadsheet.

3.2 Data analysis

We obtained descriptive statistics and verified if the variables had a normal distribution. Besides visual inspection, we calculated the Shapiro-Wilk test and used a threshold = 0.05 to determine the variables’ normal distribution (Yap & Sim, 2011). Before hypothesis testing, we obtained a correlation matrix for quantitative variables and a contingency table for scientific leadership. Given that we aimed to predict citation in WoS, we used a negative binomial regression, the recommended approach when the predicted variable shows a power-law distribution (Hilbe, 2011, 2014).

4. Preliminary results

Citation, collaboration, and expert knowledge in highly-cited Peruvian universities’ publications indexed in WoS showed a variability higher than expected (see Table 1 and Figures 1-2). However, disaggregated data by scientific leadership maintained the high data dispersion and the absence of variables’ normal distribution (see Table 2).

Table 1. Descriptive statistics for highly-cited Peruvian universities’ publications indexed in WoS

Figure 1: Boxplots for hypothesis variables

Figure 2: Violin plots for hypothesis variables

Table 2. Statistics for highly-cited Peruvian universities’ WoS papers according to leadership

Regarding citation counting, we did not only observe high data variability but also the presence of several outliers. The contingency table confirmed an association between citation-counting-based popularity and scientific leadership (χ² = 13,446, p < .001), collaboration (χ² = 26,464, p < .001), and expert knowledge (χ² = 7,189, p < .001). Spearman’s rho displayed a moderated and significant correlation between citation and collaboration (rho = 0.246, p < .001), but no for expert knowledge.

Finally, we found partial support for the conceptual model introduced in this study because two of the three hypothesized variables predict citation-counting-based academic popularity, scientific leadership being the prominent one. According to the results, the citation of Peruvian universities’ publications increases by 1.000 times for each additional co-author. In contrast, it decreases 1/0.591 = 1.692 times when a Peruvian university had the scientific leadership (see Table 3 and Figure 3).

Table 3. Hypothesis testing with negative binomial regression

Note: Akaike Information Criterion = 2,070.858, Deviance = 198.836. Loglikelihood ratio tests with one degree of freedom = scientific leadership (χ² = 10.662, p = 0.001), collaboration (χ² = 8.678, p = 0.003), an expert knowledge (χ² = 0.284, p = 0.594). S.E. = Standard error.

Figure 3: Effect of predictor variables on citation-count-based popularity

5. Discussion

The conceptual model introduced in the study got partial support because two of the three predictor variables attained statistical significance: negative effect on scientific leadership and positive impact on collaboration. This result means that the scientific leadership of a Peruvian university (public or private) reduces academic popularity, whereas it increases when the university lacks scientific leadership. Traces of this collaboration phenomena have been largely described by Kreimer in his works about Latin American science (2006, 2019), but this work presents this results in a statistical way. Since this dynamic results from the research capacity gap between the global north and south countries, it ultimately reproduce Matthew’s effect: countries with a higher quote of scientific leadership publish more citation-counting-based popular papers.

On the contrary, countries with a lower proportion of scientific leadership will publish works with less academic popularity. However, collaboration has a positive effect because the higher the number of co-authors, the higher citation-counting-based academic popularity. Although it is a fact already established in bibliometrics, this effect raises a question: Is the relationship between citation and collaboration explained only by the number of co-authors, or does a long list of co-authors anticipate a higher research quality because more scholars review the study and suggest improvements? We can only spot the issue now but do not have enough data for an in-depth analysis.

From a different approach, even though LAC researchers examined several dimensions of scientific leadership in Cuban journals or universities (Corrales-Reyes et al., 2020, 2021; Zacca-González et al., 2015), to our knowledge, this investigation is one of the first studies analyzing the interaction between the citation, collaboration, scientific leadership, and expert knowledge. Therefore, we did not compare our results with previous findings.

5.1 Theoretical and practical implications

From a global south perspective, the study of scientific leadership is relevant because it is a strong predictor of citation-counting-based academic popularity than collaboration. Although we worked with a small dataset, we found this inverse relationship between both variables. In that sense, our finding contributes to understanding this scientific publication dimension. Regarding the practical implications, rather than insisting on the “publish or perish” approach, research managers and officers should focus on capacity building; so research teams from the global south countries’ universities can control the research agenda-setting. This way, universities’ research output could contribute not only to knowledge diffusion and expansion but also to the economic and social development in the global south countries.

5.2. Limitations and future research

This study has two limitations: information source and absence of thematic coverage. Regarding the first limitation, given that we extracted data from WoS to improve information coverage, in future studies, it is advisable to extract data from databases without bias towards English publications (that occur with WoS) like Emerging Sources Citation Index (ESCI) or Scopus. Concerning thematic coverage, it would be advisable to disaggregate analysis considering publications' subject areas. In that sense, it would be helpful to classify publications with open-access taxonomies like the one provided by OpenAlex, one of the largest multidisciplinary academic databases with an acceptable coverage of publications from the global south countries' universities.

6. References

Abramo, G., D’Angelo, C. A. & Di Costa, F. (2020). The role of geographical proximity in knowledge diffusion, measured by citations to scientific literature. Journal of Informetrics, 14, 101010. https://doi.org/10.1016/j.joi.2020.101010

Adegnika, A. A., Amuasi, J. H., Basinga, P., et al. (2020). Embed capacity development within all global health research. BMJ Global Health, 6, e004692. https://doi.org/10.1136/bmjgh-2020-004692

Agasisti, T., Yang, G.-L., Song, Y.-Y. & Tran, C.-T., T. D. (2021). Evaluating the higher education productivity of Chinese and European “elite” universities using a meta-frontier approach. Scientometrics, 126, 5819-5883. https://doi.org/10.1007/s11192-021-03978-z

Azadi Ahmadabadi, G. & Rasuli, B. (2022). A multidimensional framework for measuring scientific leadership. International Journal of Information Science and Management (IJISM), 20, 311-326. https://ijism.ricest.ac.ir/article_698433.html

Bachelet, V., Uribe, F. A., Díaz, R. A., et al. (2019). Misrepresentation of institutional affiliations: The results from an exploratory case study of Chilean authors. Learned Publishing, 32, 335-344. https://doi.org/10.1002/leap.1257

Benavides, M., Chávez, C. & Arellano, A. (2016). “La construcción política e institucional de la reforma universitaria: los casos del Perú y Ecuador”. In Cueto, S. (Ed.), Innovación y Calidad en Educación en América Latina. Lima: Grupo de Análisis para el Desarrollo (GRADE), Iniciativa Latinoamericana de Investigación para las Políticas Públicas (ILAIPP).

Bornmann, L., Schier, H., Marx, W. & Daniel, H.-D. (2012). What factors determine citation counts of publications in chemistry besides their quality. Journal of Informetrics, 6, 11-18. https://doi.org/10.1016/j.joi.2011.08.004

Bourdieu, P. (1988). Homo Academicus. Stanford: Stanford Academic Press.

Corrales-Reyes, I., Fornaris-Cedeño, Y., Dorta-Contreras, A. & Pacheco-Mendoza, J. (2020). Visibility and impact of the Cuban dental scientific output, Scopus 1995-2017. Journal of Oral Research, 9, 129-141. https://doi.org/10.17126/joralres.2020.020

Corrales-Reyes, I., Hernández-García, F., Vitón-Castillo, A. A. & Mejía, C. R. (2021). Visibility, collaboration and impact of the Cuban scientific output on COVID-19 in Scopus. Heliyon, 7, e08258. https://doi.org/10.1016/j.heliyon.2021.e08258

Das, A. K. & Dutta, B. (2020). Open science in addressing responsible research and innovation: Evidence from India and other countries. Journal of Scientometric Research, 9, S14-S123. https://doi.org/10.5530/JSCIRES.9.2S.33

Demeter, M., Jele, A. & Major, Z. B. (2022). The model of maximum productivity for research universities SciVal author ranks, productivity, university rankings, and their implications. Scientometrics, 127, 4335-4361. https://doi.org/10.1007/s11192-022-04432-4

Editor (2022). Nature addresses helicopter research and ethics dumpling. Nature, 606, 7. https://doi.org/10.1038/d41586-022-01423-6

Gallegos, J. (2022, January 17). La reforma universitaria ha favorecido la producción académica en el país. La República. https://larepublica.pe/domingo/2022/01/17/la-reforma-universitaria-ha-favorecido-la-produccion-academica-en-el-pais.

Garfinkel, M. (2021). Towards the right standards: the intersection of open science, responsible research and innovation, and standards. Metode, 2021, 99-103. https://doi.org/10.7203/metode.11.16103

Hilbe, J. M. (2011). Negative Binomial Regression. 2nd ed. Cambridge: Cambridge University Press.

Hilbe, J. M. (2014). Modeling Count Data. Cambridge: Cambridge University Press.

Hottenrott, H. & Lawson, C. (2017). A first look at multiple institutional affiliations: a study of authors in Germany, Japan and the UK. Scientometrics, 111, 285-295. https://doi.org/10.1007/s11192-017-2257-6

Hottenrott, H. & Lawson, C. (2022). What is behind multiple institutional affiliations in academia? Science and Public Policy, 49, 382-402. https://10.1093/scipol/scab086

Huang, T. Y. & Yue, T. (2022). How does the research capacity gap among institutions change over time? Journal of Data and Information Science, 7, 3-4. https://doi.org/10.2478/jdis-2022-0022

Jamil, I., & Haque, S. T. M. (2016). “Knowledge generation through joint research: What can North and South learn from each other?” In Halvorsen, T. & Nossum, J. (Eds.), North–South Knowledge Networks: Towards Equitable Collaboration Between Academics, Donors and Universities. Cape Town: African Minds.

Kahalon, R., Klein, V., Wright, S. C., et al. (2022). Mentioning the sample's country in the article's title leads to bias in research evaluation. Social Psychological and Personality Science, 13(2), 352-361. https://doi.org/10.1177/19485506211024036

Kato, M. & Ando, A. (2013). The relationship between research performance and international collaboration in chemistry. Scientometrics, 97, 535-553. https.//doi.org/10.1007/s11192-013-1011-y

Kreimer, P. (2006). ¿Dependientes o integrados? La ciencia latinoamericana y la nueva división internacional del trabajo. Nómadas (Col), 24, 199-212. https://www.redalyc.org/articulo.oa?id=105116598017

Kreimer, P. (2019). Science and Society in Latin America: Peripheral Modernities. New York: Routledge.

Larivière, V. & Gingras, Y. (2010). On the relationship between interdisciplinary and scientific impact. Journal of the American Society for Information Science and Technology, 61, 126-131. https://doi.org/10.1002/asi.21226

Maldonado-Maldonado, A. & Cortes-Velasco, C. (2021). “Why research matters: Latin America facing world-class universities and rankings”. In Hazelkorn, E. & Mihut, G. (Eds.), Research Handbook on University Ranking. Elgar, pp. 473-485. https://doi.org/10.4337/9781788974981.00047

Moya-Anegón, F. de (2012). Liderazgo y excelencia de la ciencia española. Profesional de la Información, 21, 125-128. https://doi.org/10.3145/epi.2012.mar.01

Ni, P. & An, X. (2018). Relationship between international collaboration papers and their citation from an economic perspective. Scientometrics, 116, 863-877. https://doi.org/10.1007/s11192-018-2784-9

Nunes Pozzo, D., Gutierrez Roa, I., Gonzales Beleño, C. A., et al., (2022). Mapping innovation systems in Latin America: a bibliometric review of current trends. Procedia Computer Science, 203, 496-501. https://doi.org/10.1016/j.procs.2022.07.069

O’Grady, C. (2022). ‘Helicopter research’ comes under fire at Cape Town conference. Science, 376, 1144. https://doi.org/10.1126/science.add3544

Oppenheim, C. (1995). The correlation between citation counts and the 1992 research assessment exercise ratings for British library and information science university departments. Journal of Documentation, 51, 18-27. https://doi.org/10.1108/eb026940

Pozzo, M. I., Evangelista, I., Nilsson, K. S., et al. (2022). Virtual instrument systems in reality (VISIR) in Latin America from a bibliometric perspective: looking back to plan ahead. 2021 World Engineering Education Forum/Global Engineering Deans Council (WEEF/GEDC), Madrid, Spain, pp. 632-636. https://doi.org/10.1109/WEEF/GEDC53299.2021.9657423

Purnell, P. J. (2023). Geodiversity of hunger research: country focus and its regional and international actors. arXiv. https://arxiv.org/ftp/arxiv/papers/2302/2302.00659.pdf.

Van Raan, A. F. J. (1997). Science as an international enterprise. Science and Public Policy, 24, 290-300. https://doi.org/10.1093/spp/24.5.290

Yang, J. & Liu, Z. (2022). The effect of citation behavior on knowledge diffusion and intellectual structure. Journal of Informetrics, 16, 101225. https://doi.org/10.1016/j.joi.2021.101225

Yap, B. W. & Sim, C. H. (2011). Comparisons of various types of normality tests. Journal of Statistical Computation and Simulation, 81, 2141-2155. https://doi.org/10.1080/00949655.2010.520163

Zacca-González, G., Chinchilla-Rodríguez, Z., Vargas-Quesada, B. & Moya-Anegón, F. de (2015). Patterns of communication and impact of the Cuban scientific output in public health. Revista Cubana de Salud Pública, 41, 200-216.

Open science practices

The authors will make publicly available the dataset used for this study by uploading the Excel spreadsheet into Zenodo.org. In addition, to guarantee replicability, the authors conducted statistical analysis in Jamovi, an open-source software for data analysis and visualization.

Author contributions

Carlos Vílchez-Román contributed with conceptualization, formal analysis, investigation, methodology, and writing – original draft. Alejandra Manco contributed with investigation, and writing – review and editing.

Funding information

For this article, Carlos Vílchez-Román received funding from CENTRUM Católica Graduate Business School (CCGBS).

Competing interests

Authors declare no conflict of interest.