Social Sciences and Humanities research funded under the European Union Seventh Framework Programme (2007-2013): the challenge of retrieving its scholarly outputs

Jordi Ardanuy^*, Llorenç Arguimbau^**, Ángel Borrego^* and Andreu Sulé^*

^* jordi_ardanuy@ub.edu; borrego@ub.edu; sule@ub.edu

0000-0001-8453-586X; 0000-0002-6462-3966; 0000-0002-2467-3678

Departament de Biblioteconomia, Documentació i Comunicació Audiovisual & Centre de Recerca en Informació, Comunicació i Cultura (CRICC), Universitat de Barcelona, Spain

^** info@contextid.es

0000-0002-3484-9016

ContextI+D, Cerdanyola del Vallès (Spain)

Abstract

This study follows suit of a former analysis of the performance of Social Sciences and Humanities in the European Union’s 6th Framework Programme by analysing its presence in the 7th Framework Programme that was active between 2007 and 2013. The results show that economics and political science continued to be the disciplines with the largest share of funded projects. As in the former study, the high level of collaboration in project consortia did not extend to the scholarly outputs resulting from those projects, which showed low levels of institutional co-authorship. Most scholarly outputs resulting from projects were articles published in journals covered by Scopus. The study illustrates the difficulties in comprehensibly retrieving these outputs. Some of the scholarly outputs self-reported by the beneficiaries in CORDIS were not covered by OpenAIRE and/or Scopus. In addition, a comparison of the list of self-reported scholarly outputs in CORDIS with those retrieved from Scopus when searching by funding showed little overlap. Some of the outputs in CORDIS were not retrieved from Scopus whereas a large share of the articles indexed in Scopus acknowledging FP7 support had not been self-reported by the beneficiaries in CORDIS. Our results suggest that neither source (CORDIS, OpenAIRE or Scopus) is comprehensive enough as to guarantee a thorough retrieval of the scholarly outputs resulting from Social Sciences and Humanities research projects.

1. Introduction

Research assessment in the Social Sciences and Humanities (SSH) has been problematic for a long time. Performance indicators are frequently impregnated by the features of science, technology, engineering and mathematics (STEM) disciplines and are less sensitive to the singularities of the SSH. These latter disciplines are characterised by a pronounced national and regional orientation; fewer publications in journals and more in books; a slower pace of theoretical development; a single-scholar approach rather than team research; and a greater share of publications directed at the non-scholarly public (Nederhof, 2006; van Leeuwen, 2013).

Several studies confirm these features in the SSH, although they note differences between academic fields and a certain homogenization with the STEM disciplines in the choice of journal articles in English as an increasingly relevant publication venue. Certain social sciences, such as economics or psychology, show a greater degree of homogenisation with the STEM, whereas in other fields scholars tend to publish more articles in national journals and monographs, including books addressed to a non-scholarly audience. Changes in scholars’ behaviour in relation to knowledge generation and dissemination frequently result from the requirements of evaluation systems that tend to rely on the assessment procedures employed in STEM (Chi, 2014 and 2015; Engels, Ossenblok and Spruyt, 2012; Guns, Eyckens and Engels, 2019; Hammarfelt and Haddow, 2018; Kulczycki et al., 2018 and 2020; Ossenblok, Engels and Sivertsen, 2012).

The Framework Programmes (FP) for Research and Technological Development are funding programmes created by the European Union (EU) to support and foster research in the European Research Area. Through thematic calls for projects, they constitute the main funding instrument to support research in the EU since 1984. Previous research (Ardanuy, Arguimbau and Borrego, 2022) has explored how the EU contributed to the development of research in the fields of SSH through the Sixth Framework Programme (FP6), which assembled a collection of actions to fund and promote research between 2002 and 2006. Results showed that most funded projects were in the fields of economics and political sciences, in order to enhance economic development and the integration process in Europe. Research teams showed a high level of collaboration, although the large size and diversity of consortia did not translate into a large number of co-authored scholarly journal articles. The results showed that research funds in the SSH may have long-term effects, with some outputs acknowledging funding being published more than a decade after the end of the project.

This study follows suit by investigating the participation of the SSH in the Seventh Framework Programme (FP7) that was active between 2007 and 2013. The study aims to:

a) describe the participation of SSH in FP7

b) analyse the amount and type of scholarly outputs resulting from those projects

c) assess the coverage of those scholarly outputs in CORDIS, OpenAIRE and Scopus.

2. Methods

2.1. SSH projects funded under FP7

According to the ex-post evaluation report (Fresco et al., 2015), FP7 funded more than 25,000 projects around four main programmes. FP7-Cooperation promoted international cooperation within ten thematic priorities, SSH being one of them. FP7‐Ideas was newly introduced to increase research excellence in Europe and Europe’s attractiveness for world-renowned researchers. Developing human potential and strengthening research infrastructure were addressed by FP7‐People and FP7-Capacities, respectively.

We obtained a list of the 25,782 FP7 funded projects from the CORDIS website (https://cordis.europa.eu/projects). Our analysis was not limited to the 253 FP7-Cooperation-SSH projects, but covered all projects in the four FP7 programmes that included SSH features. A total of 1,290 projects devoted to SSH were identified plus 272 projects combining SSH and STEM features (see examples in Table 1), adding 1,562 projects that are those analysed in this paper.

Projects were classified by disciplines and issues. Disciplines were based on those listed in a report describing the integration of SSH on Horizon 2020 (Kania, Lemaire and Swinnen, 2019, p. 14). Issues were manually assigned based on title and abstract keywords.

Table 1. Examples of SSH projects funded under FP7 in programmes different from Cooperation-SSH. (a) a project funded under FP7-Security programme classified as SSH. (b) a project funded under FP7-Information and Communication Technologies (ICT) combining SSH and STEM features.

2.2. Publications resulting from FP7-Cooperation-SSH projects

In order to explore a homogeneous set of projects, the analysis of the publications resulting from FP7-SSH projects was limited to the scholarly outputs resulting from the 253 projects funded under the FP7-Cooperation-SSH programme. According to the evaluation report (Fresco et al., 2015), these 253 projects produced 914 publications, with an average of four publications per project, below the eleven publications per project for the whole FP7-Cooperation programme.

In order to analyse the scholarly outputs resulting from these 253 FP7-Cooperation-SSH projects, we compared four information sources:

1. CORDIS: a public dataset, updated on 7 September 2021, contained a list of publications linked to FP7 projects self-reported by beneficiaries. The beneficiaries declared a total of 3,098 scholarly outputs. However, the list included 266 duplicates that were removed resulting in 2,832 outputs.

Source: https://data.europa.eu/data/datasets/cordisfp7projects?locale=en

2. OpenAIRE: although OpenAIRE allows filtering by “Funding stream”, limiting results to FP7-SSH only retrieved 353 records. Therefore, we performed individual searches for each of the 2,832 publication listed in CORDIS. Searches were performed by DOI or, if unavailable, combining title and authors.

Source: https://explore.openaire.eu

3. Scopus (search 1): we searched the 2,832 publications listed in CORDIS in Scopus, using the same strategy that in b), i.e. by DOI or, if unavailable, combining title and authors.

4. Scopus (search 2): we retrieved, in February 2023, the scholarly outputs that acknowledged funding from any of the 253 FP7-Cooperation-SSH projects. The search equation (Table 2) retrieved 2,003 records. The purpose was to retrieve any additional scholarly outputs acknowledging FP7 support that had not been self-reported by the beneficiaries in CORDIS.

Table 2. Scopus search strategy to retrieve scholarly outputs acknowledging funding from FP7-Cooperation-SSH projects.

3. Results

3.1 FP7 SSH research projects

The analysis of the research projects funded under the four FP7 programmes resulted in the identification of 1,562 projects related to the SSH (Table 3).

Table 3. SSH projects funded under FP7 programmes.

By discipline, most of the projects were in economics, sociology and political science (Table 4). FP7-Cooperation funded a relatively small number of projects in history and anthropology, but these disciplines benefited from a large number of projects funded under FP7-Ideas and FP7-People programmes.

Table 4. Disciplines of FP7 projects in SSH (total adds more than 1,562 projects since some projects were classified in several disciplines).

The projects dealt with a wide range of issues, with those more frequently researched being migration (73 projects), the development of tools for policy decision-making (70 projects) and historical time periods, e.g. Neolithic, ancient period, medieval period, late modern period, etc. (Figure 1).

Figure 1. Issues of FP7 projects in SSH.

In terms of funding, the size of the awarded grants varied across FP7 programmes. SSH projects received less funds than the average except for those in the Capacities programme (Table 5).

Table 5. Average funding of FP7 projects.

Research collaboration varied as a result of the different requirements of the four FP7 programmes (Table 6). The analysis of the countries and institutions involved in the projects showed a high level of collaboration in FP7-Capacities (on average, each project involved 8.2 countries) and FP7-Cooperation (8.1 countries per project).

Table 6. Country collaboration in FP7 SSH projects.

The United Kingdom (737 projects) and Germany (407 projects) were the two countries involved in the largest number of projects. Up to 98 countries worldwide were involved in FP7-SSH projects (Figure 2).

Figure 2. (top) EU member countries participation in FP7 SSH projects. Labels refer to the total number of projects participated by the top five countries either as a coordinator or as a participant. Data include the current 27 EU member countries plus the United Kingdom. (down) Worldwide participation in FP7 SSH projects.

3.2 Scholarly outputs resulting from FP7 SSH projects

This section focuses on the scholarly outputs resulting from the 253 projects funded under FP7-Cooperation-SSH programme. According to the CORDIS dataset, which contained the references of the publications self-reported by the beneficiaries, 184 projects (73%) had produced at least one publication by the time the information was compiled. These projects declared a total of 2,832 outputs. On average, each project reported 15.4 publications (median = 9). Table 7 shows the distribution of scholarly outputs by document type.

Table 7. Scholarly outputs self-reported by beneficiaries of FP7-Cooperation-SSH projects.

When we searched for these 2,832 outputs in OpenAIRE, we retrieved 78% of them. This figure raised up to 86% for journal articles, but was lower for working papers (65%), book chapters (46%) and books (39%). Searches were hampered by variations in article titles, translations, changes in journal titles, etc. In general terms, searches were much more successful when a DOI was available. The results were similar for Scopus (search 1) where we retrieved 71% of the outputs listed in CORDIS. Again, the figure for journal articles was higher (82%) than for book chapters (21%) and books (19%) (Figure 4, left).

In order to supplement the data self-reported by beneficiaries, we also searched the scholarly outputs that acknowledged funding from any of these 253 FP7-Cooperation-SSH projects in Scopus (search 2) using an equation that combined the funding acronym, the funding sponsor and the funding number (Table 2). As a result, we retrieved 2,003 records (Table 8).

Table 8. Scholarly outputs of FP7-Cooperation-SSH projects retrieved from Scopus.

Although FP7 was active between 2007 and 2013, scholarly outputs continued to be published long afterwards with annual numbers of publications growing until 2018 and starting then a gradual decline (Figure 3).

Figure 3. Scholarly outputs of FP7-Cooperation-SSH projects retrieved from Scopus (search 2) by year of publication (shadowed in grey the years while FP7 was active).

Regarding collaboration, Table 6 showed that FP7-Cooperation projects involved an average of 8 countries per project. However, the scholarly outputs resulting from SSH projects showed a much lower level of co-authorship. Thus, out of the 2,003 records retrieved from Scopus, 31% had authors affiliated within a single affiliation. The average number of affiliations per scholarly output was 2.8.

The comparison of the list of scholarly outputs declared by beneficiaries in the CORDIS dataset with that of scholarly outputs retrieved from Scopus (search 2) showed little overlap (Figure 4, right). Just 497 documents (18% of the records in the CORDIS dataset) were retrieved from Scopus. The difference between both sources was partially explained by the fact that some of the scholarly outputs listed in CORDIS were not covered by Scopus. Moreover, when published, some of the publications listed in the CORDIS dataset did not acknowledge funding from the FP7-SSH programme or, if they did, this information was not correctly compiled in the funding field of Scopus. Therefore, these publications were not retrieved when we searched Scopus. Finally, there was a good deal of articles indexed in Scopus acknowledging FP7-SSH support (75%) that had not been self-reported by the beneficiaries in CORDIS.

Figure 4. (left) Overlap between scholarly outputs of FP7-Cooperation-SSH projects in CORDIS, OpenAIRE and Scopus (search 1). (right) Overlap between scholarly outputs of FP7-Cooperation-SSH projects in CORDIS and Scopus (search 2).

4. Conclusions

The results of the analysis on the performance of SSH in FP7 is consistent with previous research on its presence in FP6 (Ardanuy, Arguimbau and Borrego, 2022). Economics, sociology and political sciences were the disciplines with the largest number of funded projects. As in our former study, the high level of collaboration at the project level did not extend to the scholarly outputs resulting from these projects, which showed low levels of institutional co-authorship.

The large presence of projects in economics, sociology or political sciences shaped the type of scholarly outputs resulting from the projects, most of them articles published in journals indexed by Scopus. Nevertheless, our study illustrates the difficulties in properly retrieving these scholarly outputs. The comparison of the list of outputs self-reported by beneficiaries in CORDIS with that of scholarly outputs retrieved from Scopus when searching by funding fields showed little overlap. The differences between both sources were partially explained by the fact that some of the outputs in CORDIS were not covered by Scopus. More importantly, a good deal of the articles indexed in Scopus acknowledging FP7 support had not been self-reported by the beneficiaries in CORDIS. Our results suggest that neither of the three sources compared in our study (CORDIS, OpenAIRE or Scopus) is comprehensive enough as to guarantee a complete retrieval of the scholarly outputs resulting from SSH research projects.

Open science practices

Our study compares two open sources (CORDIS and OpenAIRE) and one commercial database (Scopus) to retrieve the scholarly outputs resulting from SSH projects funded under the EU FP7 programme. Both open sources are of different nature. CORDIS provides a public dataset of the publications linked to FP7 projects self-reported by beneficiaries whereas OpenAIRE harvests research outputs from different data providers. Although our study shows the benefits of open sources to compile data for research purposes, it also illustrates the limitations resulting from the coexistence of diverse incomprehensive data sources.

Author contributions

Jordi Ardanuy: Conceptualization, Data curation, Formal analysis, Funding acquisition, Writing—review & editing.

Llorenç Arguimbau: Conceptualization, Funding acquisition, Writing—review & editing.

Ángel Borrego: Conceptualization, Funding acquisition, Writing—original draft, Writing—review & editing.

Andreu Sulé: Conceptualization, Funding acquisition, Writing—review & editing.

Competing interests

The authors have no competing interests.

Funding information

This work was supported by the grant PGC2018-096586-B-100 «Redes de colaboración científica en ciencias sociales y humanidades en Europa: análisis de la participación en proyectos y de la coautoría funded» by MCIN/AEI/10.13039/501100011033 and “ERDF A way of making Europe”, by the “European Union”.

References

Ardanuy, J., Arguimbau, L. & Borrego, Á. (2022). Social sciences and humanities research funded under the European Union Sixth Framework Programme (2002–2006): a long-term assessment of projects, acknowledgements and publications. Humanities and Social Sciences Communications, 9, 397. https://doi.org/10.1057/s41599-022-01412-0

Chi, P.S. (2014) Which role do non-source items play in the social sciences? A case study in political science in Germany. Scientometrics, 101 (2), 1195–1213. http://doi.org/10.1007/s11192-014-1433-1

Chi, P.S. (2015) Changing publication and citation patterns in political science in Germany. Scientometrics, 105 (3), 1833–1848. http://doi.org/10.1007/s11192-015-1609-3

Engels, T.C., Ossenblok, T.L. & Spruyt, E.H. (2012). Changing publication patterns in the social sciences and humanities, 2000–2009. Scientometrics, 93 (2), 373-390. https://doi.org/10.1007/s11192-012-0680-2

Fresco, L. et al. (2015). Commitment and coherence: Ex-post evaluation of the 7^th EU Framework Programme (2007-2013). https://op.europa.eu/s/vXAA

Guns, R., Eykens, J. & Engels, T.C. (2019). To what extent do successive cohorts adopt different publication patterns? Peer review, language use, and publication types in the social sciences and humanities. Frontiers in Research Metrics and Analytics, 3. https://doi.org/10.3389/frma.2018.00038

Hammarfelt, B. & Haddow, G. (2018) Conflicting measures and values: How humanities scholars in Australia and Sweden use and react to bibliometric indicators. Journal of the Association for Information Science and Technology, 24 (2), 924–935. http://doi.org/10.1002/asi.24043

Kania, K., Lemaire, C. & Swinnen, L. (2019). Integration of Social Sciences and Humanities in Horizon 2020: Participants, Budget and Disciplines. https://op.europa.eu/s/xNU8

Kulczycki, E. et al. (2018) Publication patterns in the social sciences and humanities: evidence from eight European countries. Scientometrics, 116, (1), 463-486. https://doi.org/10.1007/s11192-018-2711-0

Kulczycki, E. et al. (2020). Multilingual publishing in the social sciences and humanities: A seven‐country European study. Journal of the Association for Information Science and Technology, 71 (11), 1371-1385. https://doi.org/10.1002/asi.24336

Nederhof, A.J. (2006). Bibliometric monitoring of research performance in the social sciences and the humanities: A review. Scientometrics, 66 (1), 81-100. https://doi.org/10.1007/s11192-006-0007-2

Ossenblok, T.L., Verleysen, F.T. & Engels, T.C. (2014). Coauthorship of journal articles and book chapters in the social sciences and humanities (2000–2010). Journal of the Association for Information Science and Technology, 65 (5), 882-897. https://doi.org/10.1002/asi.23015

Ossenblok, T.L., Engels, T.C. & Sivertsen, G. (2012). The representation of the social sciences and humanities in the Web of Science—a comparison of publication patterns and incentive structures in Flanders and Norway (2005–9). Research Evaluation, 21 (4), 280-290. https://doi.org/10.1093/reseval/rvs019

Van Leeuwen, T. (2013). Bibliometric research evaluations, Web of Science and the Social Sciences and Humanities: a problematic relationship? Bibliometrie-Praxis und Forschung, 2. https://doi.org/10.5283/bpf.173