Retractions, retracted articles and withdrawals coverage in scholarly databases

José Luis Ortega^* and Lorena Delgado-Quirós^*

^* jortega@iesa,csic,es; ldelgado@iesa.csic.es

0000-0001-9857-1511; 0000-0001-8738-7276

Institute for Advanced Social Studies (IESA), Spanish National Research Council (CSIC), Spain

This paper analyses the coverage of retractions, retracted publications and withdrawals by seven scholarly databases (Dimensions, The Lens, OpenAlex, PubMed, Scilit, Scopus and Web of Science). The results show that there are two types of products: traditional citation indexes (WoS, PubMed, Scopus) based in the selection of journals and where withdrawals are not indexed; and new hybrid databases (Dimensions, OpenAlex, Scilit and The Lens), less selective and based in external sources such as Crossref and Microsoft Academic. These differences are mainly due to the coverage of withdrawals.

1. Introduction

A frequent way to correct erroneous studies is to publish retraction notes to inform and therefore remove publications that contain serious problems that compromise the reliability of their results. Scholarly databases have been used as source to quantify the frequency of these document types (Fang et al., 2012). However, this type of literature implies some indexation challenges. For example, the retracted publications have to be updated to inform about their new status or retractions notices should be linked to the retracted publication. This problem gets worse when publishers do not follow the international guidelines (COPE, 2019) regarding to notice retracted publications, remove publications without a notice, or replace the original document by the retraction notice.

On the other hand, the current proliferation of new scholarly databases (Dimensions, The Lens, Scilit) and academic search engines (Google Scholar, Semantic Scholar) provide new sources to obtain and analyze these publications as well as to check how this literature is being covered and estimated.

The main objective of this study is to analyze the coverage of this type of publications (retractions, retracted articles and withdrawals) by seven scholarly databases (Dimensions, The Lens, OpenAlex, PubMed, Scilit, Scopus and Web of Science) to compare their coverage, overlap and indexing problems. These sources were selected because they index this literature with specific labels or typologies that make possible their retrieval.

The data search and retrieval process were accomplished in June 2022. All the queries were limited to retrieve records from 2000 forward.

2. Results

2.1 Coverage and overlapping

Figure 1. Coverage of retractions, withdrawals and retracted articles by database

Figure 1 and shows the number of retractions, withdrawals and retracted articles retrieved from the selected databases. OpenAlex (57,892) and Dimensions (47,913) are the platforms that identify the largest number of these publication types, while PubMed (23,699), Scopus (31,634) and WoS (31,565) are the bibliographic services that index less publications. These coverage differences can be explained by the way in which each database is built. Scopus and WoS are selective sources that limit their coverage to a specific range of journals, which justify the low coverage. PubMed is a specialized database in medicine and related sciences, which also select only journals from that knowledge domain. Scilit, Dimensions and Lens depict similar coverage because they are recent products based on secondary sources (e.g. Crossref, PubMed, OpenAlex). They have larger coverage than the former one because they do not limit their coverage by journals or document type. OpenAlex is the bibliographic service with the largest coverage due mainly to it is based on the defunct Microsoft Academic, a search engine that indexed academic content on the Web without restrictions. It is also interesting to notice that a large part of these differences is due to withdrawn publications.

Figure 2. MDS graph showing differences between databases according to the coverage of retracted publications.

Figure 2 plots the position of the databases according to the proportion of shared publications. Two different groups were detected: On the one hand, WoS, Scopus and PubMed (blue) are traditional bibliographic databases that directly index publications from the publishers; On the other hand, Dimensions, OpenAlex, Scilit and The Lens (green) are hybrid databases the take their bibliographic records from third sources. In this case, Crossref is the common source for all of them. This result shows that coverage similarities are influenced by the way in which bibliographic databases gather their content.

Conclusions

The first conclusion is that the coverage of retractions, retracted publications and withdrawals is rather different between scholarly databases. These differences are mainly due to the indexation of withdrawals by the new hybrid databases (Dimensions, OpenAlex, Scilit and The Lens). Excluding this information, OpenAlex (17,329) and The Lens (17,309) are the databases that collect most retractions, whereas Scilit (27,618), Scopus (19,280) and Dimensions (18,249) include most retracted articles. According to the overlap between databases, it is possible to conclude that there are two differentiated types of products: traditional citation indexes (WoS, PubMed, Scopus) based in the selection of journals and where withdrawals are not indexed; and new hybrid databases (Dimensions, OpenAlex, Scilit and The Lens), less selective and based in external sources such as Crossref and Microsoft Academic.

Open science practices

This communication is based on open (OpenAlex, PubMed) and proprietary sources (Web of Science, Scopus, Dimensions, Scilit, The Lens). Due to this, raw data from proprietary sources cannot be publicly released.

Author contributions

José Luis Ortega has contributed to Writing. Lorena Delgado-Quirós has contributed to Data curation.

Funding information

This work was supported by the research project (NewSIS) “New scientific information sources: analysis and evaluation for a national scientific information system” (Ref. PID2019-106510GB-I00) funded by the Spanish State Research Agency (AEI) PN2019

References

COPE Council (2019). COPE Retraction guidelines -- English. https://doi.org/10.24318/cope.2019.1.4

Fang, F. C., Steen, R. G., & Casadevall, A. (2012). Misconduct accounts for the majority of retracted scientific publications. Proceedings of the National Academy of Sciences, 109(42), 17028-17033.