MetaMind: A Multi-Agent Transformer-Driven Framework for Automated Network Meta-Analyses

Abstract

Background Network meta-analysis (NMA) enables simultaneous comparison of multiple interventions by integrating direct and indirect evidence from randomized controlled trials and observational studies, but traditional workflows require extensive manual effort for study identification, data extraction, and statistical modelling, leading to slow update cycles and operational bottlenecks.

Objective To develop and validate MetaMind, an end-to-end, transformer-driven framework that automates NMA processes—including study retrieval, structured data extraction, and meta-analysis execution—while minimizing human input.

Methods MetaMind integrates Promptriever, a fine-tuned retrieval model, to semantically retrieve high-impact clinical trials from PubMed; a multi-agent LLM architecture--Mixture of Agents (MoA)-- pipeline to extract PICO-structured (Population, Intervention, Comparison, Outcome) endpoints; and GPT-4o–generated Python and R scripts to perform Bayesian random-effects NMA and other NMA designs within a unified workflow. Validation was conducted by comparing MetaMind’s outputs against manually performed NMAs in ulcerative colitis (UC) and Crohn’s disease (CD).

Results Promptriever outperformed baseline SentenceTransformer with higher similarity scores (0.7403 vs. 0.7049 for UC; 0.7142 vs. 0.7049 for CD) and narrower relevance ranges. Promptriever performance achieved 82.1% recall, 91.1 % precision and an F1 score of 86.4 % when compared to a previously published NMA. MetaMind achieved 100% accuracy in PICO element extraction and produced comparative effect estimates and credible intervals closely matching manual analyses.

Conclusions MetaMind significantly reduces time and labour—delivering a complete NMA in under one week versus several months manually—while maintaining methodological rigor and scalability across therapeutic areas, representing a major advancement in AI-driven evidence synthesis

Competing Interest Statement

A.C., M.K., X.L., M.G., and Y.Z. are employees of Pfizer Inc. D.Z. is an employee of Teva Pharmaceuticals USA. J.L. is an employee of Takeda Pharmaceuticals USA. S.D. is an employee of Sarepta Therapeutics. S.T. has received institutional research funding from the ECRAID-Base consortium funded by the EU Horizon 2020 programme. V.R. has received research funding from BeOne Medicines Ltd; during the past 36 months has received contracts or grants from Blueprint Medicines, Genentech, Janssen, Merck, Mitsubishi Tanabe, Stryker, and Takeda; received honoraria from Natera and Ironwood; and served in leadership roles with Data Unite, ZebraMD, and AcucareAI. A.L. declares no conflicts of interest.

Funding Statement

The authors received no financial support for this research.

Author Declarations

I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.

Yes

I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.

Yes

I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).

Yes

I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.

Yes

Data Availability Statement

No new data were created or analysed in this study. Data sharing is not applicable to this article

Comments (0)

No login
gif