A systematic analysis of temporal trends, characteristics, and citations of retracted stem cell publications

Abstract

Background

The increasing prevalence of retracted publications in stem cell research presents significant challenges to scientific integrity. Although retraction notices are issued, retracted studies continue to be cited, facilitating the dissemination of unreliable findings. This study aimed to systematically explore the characteristics of retracted stem cell publications and evaluate the impact of retractions on subsequent citations.

Methods

This study was conducted following the PRISMA guidelines. A comprehensive search of Web of Science, Retraction Watch Database, and PubMed was conducted from their inception through July 25, 2024, to identify retracted stem cell publications. Characteristics including publication details, retraction reasons, and citation counts were extracted. To assess the impact of retraction on subsequent citations, we compared citation patterns between a random sample of retracted papers and matched non-retracted controls from identical journals and issues. Further analysis was conducted to determine whether papers citing retracted articles had an elevated risk of subsequent retraction. Descriptive statistics, chi-squared tests, t-tests, and Mann–Kendall tests were used for data analysis.

Results

The systematic search identified 1421 records, with 517 publications meeting inclusion criteria. Temporal analysis revealed two significant trends: an increasing retraction rate that peaked at 0.84% in 2023 and a declining time-to-retraction (median: 30 months, interquartile range: 13–60; Mann–Kendall, tau = − 0.29; P < 0.001). Hospital-affiliated researchers from China contributed to 244 (47.2%) of retractions. Data and image flaws were identified in 360 (69.6%) of retractions. Among 472 Web of Science-indexed retracted publications, 366 (77.5%) accumulated 4884 post-retraction citations, with 114 (24.2%) receiving more citations post-retraction than pre-retraction. Analysis of a random subset of retracted articles (n = 53) demonstrated that only 14 (4.2%) out of 334 post-retraction citations referenced the retraction notice. Compared with 639 non-retracted control publications, retracted articles showed significantly lower post-retraction citation rates (mean rank: 291.32 vs. 351.08; P = 0.01). Moreover, papers citing retracted articles exhibited an 11-fold higher risk of subsequent retraction (odds ratio (OR): 11.09; 95% confidence interval (CI): 7.06–17.43).

Conclusions

This analysis reveals substantial research integrity challenges within stem cell research. These findings suggest the necessity for enhanced surveillance mechanisms and standardized protocols to identify and curtail the dissemination of flawed research.

Background

The frequency of biomedical paper retractions has surged, increasing fourfold over the past 20 years [1,2,3]. Retraction serves as a fundamental mechanism for maintaining the integrity of the scientific literature, ensuring that readers are alerted when research findings are unreliable [4, 5]. The majority of retractions are attributed to data or image integrity, which fundamentally undermines the validity of the reported findings [6,7,8]. Other retractions involving ethical violations or peer review manipulation, although not directly invalidating research findings, compromise scientific credibility [9,10,11].

Stem cells possess a remarkable ability to differentiate into various cell types, making them invaluable in fields such as regenerative medicine, developmental biology, and disease treatment [12,13,14]. Recently, the number of publications on stem cells has surged, accompanied by a corresponding rise in retractions. These retractions are primarily due to issues with data authenticity, which compromises the dissemination of reliable scientific findings [15, 16]. Alarmingly, retracted papers often continue to be cited, potentially compromising research integrity and, more critically, patient safety through the perpetuation of invalidated findings.

The systematic analysis of retracted publications is crucial for identifying common errors and instances of misconduct. These insights serve as critical warnings to researchers and guide journals and institutions in developing robust mechanisms and preventive policies [4]. Moreover, investigating the downstream impact of retracted research can reveal the extent of compromised data infiltration and help develop effective mitigation strategies. However, the characteristics of retracted papers in stem cell research and their influence on subsequent knowledge dissemination remain largely unexplored. This study aimed to characterize retracted publications in stem cell research and evaluate the impact of retraction on subsequent citations.

Methods

This systematic review was registered in the Open Science Framework (https://osf.io/YN4GH) in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) reporting guideline [17]. The PRISMA checklist is available in Additional file 1. The study protocol was deemed exempt from ethical review by the Affiliated Hospital of Southwest Medical University as it did not involve personal information.

Search strategies

A comprehensive literature search was conducted in Web of Science, Retraction Watch Database, and PubMed from their inception through July 25, 2024. Search terms included “retraction,” “stem cell,” “withdrawal,” and their synonyms. The complete search strategy is detailed in Additional file 2.

Study selection

Two investigators (FS and DL) independently screened records for inclusion. Studies were included if they met all of the following criteria: (1) research focused on stem cells; (2) retracted status, regardless of retraction reason; (3) original journal articles (excluding conference abstracts). Studies focusing on cancer stem cells were excluded due to their distinct biological characteristics from normal stem cells. Disagreements were resolved through consensus discussion with a third investigator (WX).

Data extraction and grouping

For each included record, the following characteristics were extracted: title, journal, publisher, authors and their institutions, country, article type, reasons for retraction, dates of publication and retraction, stem cell types, and their applications. Retraction reasons were categorized according to Retraction Watch Database User Guide Appendix (https://retractionwatch.com/retraction-watch-database-user-guide/retraction-watch-database-user-guideappendix-b-reasons/) and previous studies [9, 18]. Additionally, Web of Science was used to retrieve the number of citations before and after the retraction year of each retracted paper (citation was marked as not available if the paper was not indexed by Web of Science), and the journal impact factor (JIF) of the year before publication, as well as 2023 JIF of related journals. To evaluate the impact of retraction on subsequent citations, we randomly selected 15% of retracted publications (group A) and identified matched non-retracted controls (group B) from the same journals and issues, matching for publication type (article or review) and research field (stem cells). Retracted articles without suitable matches were excluded from analysis. To check whether articles citing retracted publications are more likely to be retracted, articles citing group A were defined as group C, articles citing group B were defined as group D. Data collection was completed on July 25, 2024.

Statistical analysis

We used descriptive statistics to summarize the included data. Categorical variables were presented as counts (percentages) and compared using a chi-squared test. Continuous data following a normal distribution were presented as mean (standard deviation, SD) and compared using the Student’s t-test. Continuous data that did not follow a normal distribution were expressed as median (interquartile range, IQR) and compared using the Mann–Whitney U test. We explored trends of absolute frequencies over time by examining plotted time series and tested statistical significance using the Mann–Kendall test. All analyses were conducted with R statistical software version 4.2.2. P values < 0.05 were considered statistically significant.

Results

The systematic search identified 1421 records from Web of Science, Retraction Watch Database, and PubMed. After duplicate removal (n = 537), articles were excluded based on the following reasons: non-stem cell research (n = 205), cancer stem cell focus (n = 145), non-retracted status (n = 2), and insufficient documentation (n = 14). Ultimately, 517 eligible records were included in the study (Fig. 1).

Fig. 1

Study flowchart

Temporal trends of retractions

The 517 retracted publications were originally published during 1989–2024 and retracted during 1994–2024 (Fig. 2). Temporal analysis revealed a significant upward trend in the number of retracted stem cell publications over time (Mann–Kendall, tau = 0.768; P < 0.001). Despite the increasing number of publications, the annual retraction rate peaked at 0.84% (117/13,856) in 2023. The median time-to-retraction was 30 months (IQR: 13–60), with a significant decreasing trend over time (Mann–Kendall, tau = − 0.29; P < 0.001) (Fig. 3).

Fig. 2

Annual number of stem cell publications and retractions from 1989 to 2024

Fig. 3

Time-to-retraction over time. The regression line was fitted using the Local Estimated Scatterplot Smoothing (LOESS)

Characteristics of retractions

Geographic and institutional distribution

The analysis identified 280 authors with multiple retractions, without accounting for the possibility that different authors might have identical names. Author affiliations showed single-country authorship in 439 (84.9%) articles and multinational collaboration in 78 (15.1%) articles. Retractions involving authors from China, the USA, and Japan collectively accounted for 360 (69.6%) of all retractions (Table 1). Medical institutions contributed to 311 (60.2%) of retractions, with a higher proportion from Chinese medical institutions compared to other countries (78.5% vs. 22.8%).

Table 1 Characteristics of retracted publications

Classification of stem cell types and applications

Table 1 shows the types of stem cells and their applications across 517 retracted publications. The majority of retracted studies focused on SSCs (80.1%, 414/517), followed by ESCs (12.8%, 66/517) and iPSCs (7.2%, 37/517). Specifically, MSCs accounted for the most retractions (50.3%, 260/517).

These retracted studies covered a wide range of therapeutic applications. The predominant application was bone regeneration, with 115 retracted articles, followed by neural disease therapy and organ repair. Other significant areas included cancer therapy, vascular regeneration, and dental regeneration.

Journal sources and retraction reasons

The 517 retracted publications revealed a distribution across 235 journals from 82 publishers (Additional file 3: Table S1). Retracted publications were attributed to one or more reasons. Data and image integrity issues were the predominant cause, accounting for 69.6% of retractions (360/517).

These retractions were frequently published in journals with a lower JIF (Fig. 4). The median JIF for 2023 and the JIF of year before original publication were 3.7 (IQR: 2.3–5.4) and 3.5 (IQR: 2.5–6.1) respectively.

Fig. 4

Number of retractions by JIF

Citations to retracted publications

Among the 517 retracted publications, 472 were indexed in Web of Science with trackable citation data. Post-retraction citations were documented for 366 (77.5%) publications, accumulating a total of 4884 citations. Notably, 114 publications (24.2%) received more citations after retraction than before. Retracted publications with higher JIF tended to receive more citations (Fig. 5).

Fig. 5

Average number of citations per article by JIF

To evaluate the impact of retraction on subsequent citations, a random sample of 77 retracted articles (15%) was selected for further analysis. After excluding 24 articles without control articles, 53 articles (group A) were analyzed. The 53 articles were cited 2258 times in total, of which 334 citations occurred post-retraction. However, only 14 citations (4.2%) included a reference to the retraction notice. The majority of citations were found in research articles (1248, 55.3%) and review articles (769, 34.1%).

As described in the Methods section, 639 non-retracted articles (group B) were identified as controls for group A. No significant differences were observed in overall or pre-retraction citations. However, retracted articles had significantly fewer post-retraction citations (mean rank: 291.32 vs. 351.08, P = 0.01) and exhibited a smaller citation increase (post-retraction citations minus pre-retraction citations; mean rank: 271.59 vs. 352.71, P = 0.01) (Additional file 3: Table S2). The likelihood of a retracted article being cited more frequently post-retraction was 48% that of non-retracted articles (OR: 0.48; 95% CI: 0.25–0.91) (Additional file 3: Table S3). These findings suggest that retraction may reduce citation risks.

To assess whether articles citing retracted publications are more likely to be retracted themselves, we compared a set of 2258 articles (group C) that cited retracted papers (group A) with 26,006 articles (group D) that cited non-retracted papers (group B). We found that 37 out of 2258 articles citing retracted publications were later retracted, compared with 39 out of 26,006 citing non-retracted publications (OR: 11.09; 95% CI: 7.06–17.43) (Additional file 3: Table S4). This suggests that articles citing retractions have a significantly higher likelihood of being retracted later.

Discussion

In this study, we comprehensively characterized retracted publications in the stem cell field and analyzed their subsequent citations. Temporal analysis demonstrated an increasing frequency of retractions, accompanied by a reduction in time-to-retraction. This trend reflects improvements in scientific oversight mechanisms, including the rise of post-publication peer review platforms, such as PubPeer [19, 20], and enhanced digital screening tools for detecting data and image flaws. These developments have facilitated more efficient identification and investigation of potentially problematic publications.

Geographic analysis revealed notable institutional patterns. Chinese hospitals accounted for 47.2% (244/517) of all retractions, consistent with observations in other research fields [9, 10, 21]. Based on our experience with the Chinese academic medical system, this pattern may reflect specific systemic pressures within hospital settings, where physicians often face requirements to publish research papers for career advancement while managing heavy clinical workloads. This combination of publication pressure, limited research time, insufficient research oversight mechanisms, inadequate training in research integrity, and lack of funding may contribute to flawed research [22]. In response to these concerns, China has initiated its first nationwide review of scientific misconduct last year [23].

Citation analysis revealed that retraction notices moderately reduced subsequent citation. However, post-retraction citations persist, which echoes findings in other fields [7, 24]. This persistence may be attributed to several systematic issues: inadequate visibility of retraction notices on journal platforms and the propagation of citations through reference chains without primary source verification. The observation that articles citing retracted research face an 11-fold higher retraction risk underscores the importance of citation verification. This suggests that manuscripts citing multiple retracted works should undergo extra scrutiny during the peer review process, preventing the spread of unreliable information. To address the dissemination of flawed research, we recommend that journals implement standardized retraction notices, ensuring their accessibility and prominence on journal platforms. Automated systems, powered by machine learning algorithms, could be employed to flag retracted citations during peer review. Moreover, researchers should rigorously verify the status of all cited references using databases such as PubMed, Web of Science, and Retraction Watch before submitting their manuscripts. Institutions and funding bodies should also consider incorporating citation checks into their review processes, reinforcing the importance of due diligence in academic research. The successful implementation of these measures requires collaborative engagement from multiple stakeholders—publishers, researchers, institutions, and funding bodies—to effectively curtail the propagation of retracted research and strengthen scientific integrity.

To the best of our knowledge, this is the first study to assess the characteristics and citations of retracted publications in the stem cell field. Considering the unique nature of this field—where the translation of research findings into therapeutic interventions is rapidly expanding while legal and regulatory frameworks remain underdeveloped [25,26,27,28,29], some aggressive medical institutions may base clinical treatments directly on findings from individual research publications. If unreliable publication is referenced and applied in clinical practice, it could result in severe adverse outcomes, including tumor formation, immune rejection, and other unforeseen complications [30]. Alarmingly, some of retracted publications in this field are related to studies that assessed the risk or safety of these therapies [31, 32].

Strengths and limitations

This study presents several notable strengths. First, it covers 517 retracted stem cell publications from 1989 to 2024, utilizing multiple databases for thorough data collection. It systematically analyzes the characteristics of these retractions, providing researchers with a deeper understanding of the information surrounding these retracted articles. Second, this study adheres to PRISMA guidelines and employs robust statistical analyses to support its findings. Third, it provides valuable insights into post-retraction citations, highlighting ongoing issues in scientific publishing. Despite these strengths, several potential limitations should be considered. First, the study relied on publicly available retraction databases and journals, which may not capture all retracted publications, particularly those retracted quietly or not formally announced. This underreporting could affect the completeness of the data. Second, the study does not account for the varying policies and practices of different journals and publishers regarding retractions, which could influence both the likelihood of retraction and post-retraction citation behaviors. Additionally, our reliance on Web of Science for citation tracking may introduce systematic biases in our citation analysis. Web of Science primarily indexes English-language journals from North America and Europe, potentially underestimating citations from non-indexed journals, particularly those published in other languages or from developing countries. As a result, the actual impact of retracted papers may be greater than our findings suggest.

Conclusions

This study demonstrates an increase in retractions within stem cell research, with data and image flaws being the most common reasons. While retraction notices help reduce citations, most retracted articles continue to be cited post-retraction. Moreover, articles citing retracted publications exhibited a higher risk of subsequent retraction. These findings underscore the need for coordinated efforts from journals, researchers, and institutions to enhance research integrity through improved retraction policies, rigorous citation verification, and strengthened oversight mechanisms.