Citation: Cunha A, de Bruijn S, Farrell A, Lumbard H, Tosun A, Routledge D (2025) Setting the standard for high-quality studies using open health datasets. PLoS Med 22(12): e1004854. https://doi.org/10.1371/journal.pmed.1004854

Published: December 4, 2025

Copyright: © 2025 Cunha et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Funding: The author(s) received no specific funding for this work.

Competing interests: I have read the journal’s policy, and the authors of this manuscript have the following competing interests: All authors are current paid employees of the Public Library of Science.

Much has been written about the value of open science and hypothesis-driven secondary analyses of open health datasets [1], and PLOS Medicine has been and continues to be a strong supporter of both. There is a growing ecosystem of these rich resources—including the UK Biobank [2], National Health and Nutrition Examination Survey [3], the Global Burden of Disease Study [4], and the Demographic and Health Surveys Program [5], to name a few. Open health datasets have the potential to both democratize science and tackle crucial health challenges. They provide valuable opportunities to the global scientific community to test hypotheses when generating and maintaining the data would otherwise not be feasible, such as in resource-constrained settings [6]. When carefully designed, analyses using such datasets have provided novel insights on important global and national health issues, for example, exploring the social determinants of health outcomes, assessing equity in access to care, or mapping disease burden across populations [7–10]. Well-conducted descriptive studies have demonstrated how socioeconomic status influences mortality, how health system performance varies across contexts, or how care cascades can reveal critical gaps in disease prevention and management [7,9,10].

While the value of data-driven health research using publicly available datasets is undeniable, a recent proliferation of poorly conducted analyses—including growing submissions from suspected paper mill operations [11,12]—has raised concerns that these studies threaten the integrity and value of scientific literature. Utilization of large datasets does not guarantee the quality of research; analyses are only as good as the underlying research questions, assumptions, biases, and representativeness of the data. Unfortunately, there is persistent variability in the methodological rigor of large open health data studies, with inconsistent application of robust statistical methods, suboptimal handling of missing data, and lack of transparency of data sources, analytical decisions, code, and study limitations. These issues have been exacerbated by the application of artificial intelligence to generate formulaic manuscripts reporting single associations without a strong scientific rationale, false-positive findings with inadequate adjustment for multiple testing, and/or selective use of subsets of data [11,12]. Although generating a paper quickly and easily using this approach can seem appealing in work cultures where publication numbers are rewarded, it is a damaging practice that clutters the scientific literature with contributions of little value, or misleading findings with potentially detrimental effects for clinical practice and public health.

To preserve the integrity and value of open health data research, clearer standards for publication are essential. In response to these challenges, several publishers, including PLOS, have announced new policies for retrospective studies using health databases [13,14]. PLOS journals will automatically reject such manuscripts unless researchers provide additional work, such as experiments or primary analyses that validate results and clearly establish their contribution to the field. In addition, the Journal of Global Health has developed guidelines aimed at mitigating the key negative impacts associated with such studies [6].

To assist authors in planning and conducting high-quality secondary analyses of open health datasets, and to provide clarity on editorial standards, we have prepared a 10-point guide outlining factors that contribute to a strong study (see Box 1).