The broadly accepted standard for demonstrating a favorable benefit–risk profile of a given therapy is an RCT. Although RCTs undoubtedly form the basis of drug development and regulatory approval, they are associated with various limitations, such as ethical concerns regarding the use of placebo as control therapy, resource intensity, recruitment challenges, lack of general population representativeness, and limited generalizability to a real-world setting [19]. Consequently, there is a need to supplement RCTs with evidence generated from other types of studies, such as single-arm, observational, or hybrid studies that analyze data collected in various settings. RCTs are conducted under controlled conditions, ensuring that the only difference between groups is the intervention being studied. Externally controlled studies often rely on RWD, which can introduce variability and confounding factors that are difficult to control.

Increased availability of high-quality RWD has led to a growing acceptance of RWE by regulators of medical products [19,20,21]. Best practices to promote standards for the use of the ECA approach are being developed by the scientific community, investigators, regulators, payers, and sponsors [12, 22,23,24,25]. When applying the ECA approach using RWD, general principles of regulatory-grade RWE generation should be followed [24, 25]. Key elements include fit-for-purpose data source identification, accurate definition and completeness of study variables, assignment of index date, adherence to data curation and data processing standards, adequate choice of analytical methods for addressing confounding in non-randomized studies, and adherence to the Guidelines for Good Pharmacoepidemiology Practice.

Although the ECA framework is a rapidly evolving field, its use by healthcare decision-makers is yet to be fully accepted. Adoption of the ECA approach depends on continued dialog among relevant parties [26,27,28,29]. Multiple proof-of-concept studies in individuals with cancer have demonstrated the feasibility of the ECA approach using EHR databases to complement single-arm clinical trials [26, 28]; however, a limited number of regulatory submissions containing real-world external comparators have been successful [26, 28, 29]. Some health authorities have indicated a preference for a hybrid design with a mix of randomized and external controls over complete replacement of an RCT control arm [26, 28]. A hybrid design allows regulators to evaluate the comparability of the randomized control arm and ECAs, thereby minimizing the risk of misinterpreting the results. Performing a health technology assessment of a new therapy often requires external comparisons. Indirect treatment comparisons (ITCs) represent a classical technique and utilize historical data from RCTs that may have been conducted several decades ago. Contemporary RWD and the ECA approach provide a viable alternative to ITCs [14].

The present study contributes to the body of evidence on the applicability of the ECA approach and RWD collected in routine clinical practice to drug development.

The study was initiated in parallel to the PACIFIC-AF phase 2 trial as a pilot project with the objectives of building an ECA using real-world EHR data from patients with AF treated with apixaban, demonstrating concordance of event rates for key trial endpoints in the internal control arm and the ECAs, and informing a related phase 3 study design (OCEANIC-AF, NCT05643573) [30]. A large cohort of 160,153 patients in the US Optum® EHR data set satisfied the selection criteria of PACIFIC-AF and constituted a substantially larger population than the PACIFIC-AF apixaban control arm. Various matching techniques were used to select patients from this cohort who matched the patients in the PACIFIC-AF apixaban arm on 101 variables [10]. ECAs with ratios of up to 1:10 were built without sacrificing matching quality, allowing high accuracy in treatment effect estimates for key clinical endpoints over a follow-up duration of up to 2 years.

The event rates for the primary, secondary, and exploratory outcomes were broadly similar in the PACIFIC-AF control arm and in the ECAs at 85 days. In the PACIFIC-AF apixaban arm and the 1:10 ECA at 85 days, 2.40% (0 and 2.40%) and 3.68% (0.68% and 3.12%) of patients reported ISTH major bleeding or ISTH CRNMB events (ISTH major bleeding and ISTH CRNMB separately), respectively. An incidence risk of the primary endpoint was expected to be 4% at 85 days in the apixaban control group of PACIFIC-AF during the study planning [10]. IS/SE events were not observed in the PACIFIC-AF control arm, and there were 19 such events (0.76% of patients) in the 1:10 ECA at day 85. Concordance between event rates for the key trial endpoints in the ECAs and the internal control arm was assessed visually with no statistical method applied, which represents a limitation of the study. However, visual assessment was deemed adequate due to the low number of events in the internal control arm of PACIFIC-AF.

The ECA approach enabled the projection of event rates beyond PACIFIC-AF treatment duration of 85 days up to 2 years. For the primary composite outcome of ISTH major bleeding or CRNMB at day 360, the cumulative incidence was estimated at 11.55% (95% CI 10.26–12.93%) in the 1:10 ECA. The cumulative incidence of ISTH major bleeding was estimated at 2.12% (95% CI 1.58–2.78%) in the 1:10 ECA at 360 days and was consistent with the annual cumulative incidence of ISTH major bleeding previously observed for apixaban treatment (2.13%) in the phase 3 ARISTOTLE trial [31]. However, this comparison should be interpreted with caution owing to differences in investigated patient populations. In the present study, for ISTH CRNMB and the composite outcome of IS/SE at day 360, the cumulative incidences were 9.90% (95% CI 8.69–11.19%) and 2.60% (95% CI 1.99–3.32%) in the 1:10 ECA, respectively.

OCEANIC-AF was a pivotal phase 3 RCT with the aim of comparing asundexian and apixaban for preventing stroke or SE in patients with AF [30]. The trial was terminated in November 2023 owing to the efficacy of asundexian being inferior to that of the control treatment. In total, 7395 patients were randomly assigned to receive apixaban. The mean ages of patients in the control arms of both PACIFIC-AF and OCEANIC-AF were identical at 74 years (standard deviation [SD]: 8 years); the mean CHA2DS2-VASc scores were 4.1 (SD: 1.5) and 4.3 (SD: 1.3), respectively. However, there were several critical differences in characteristics such as sex, comorbidities, and previous oral anticoagulant use due to variations in the eligibility criteria of the two trials, and such characteristics are known risk factors for safety and efficacy outcomes in patients with AF. Consequently, differences in the event rates of clinical outcomes observed in the ECAs of PACIFIC-AF and the control arm of OCEANIC-AF are anticipated. Nevertheless, a contextual comparison is warranted.

The median follow-up of OCEANIC-AF was 155 days [30]. The most comparable follow-up period reported in the present ECA study was 180 days. In the apixaban control arm of OCEANIC-AF, the incidence rate of the primary safety endpoint of ISTH major bleeding was 1.93 (95% CI 1.45–2.48) per 100 PY [30] and in good concordance with the incidence rate of 2.39 (95% CI 1.58–3.38) per 100 PY in the 1:10 ECA at 180 days. For the primary efficacy endpoint of stroke/SE, the incidence rate was 1.02 (95% CI 0.66–1.44) per 100 PY in the apixaban control arm of OCEANIC-AF [30] compared with 3.38 (95% CI 2.39–4.53) per 100 PY in the 1:10 ECA at 180 days. It was noted that the incidence rate of IS in the control arm of OCEANIC-AF was notably lower than predicted by the CHA2DS2-VASc score and lower than that observed in the ARISTOTLE trial [30]. Further research is needed to elucidate the impact of coexisting conditions and treatments of patients with AF on the risk of clinical outcomes over time. The ECA approach might play a crucial role in this endeavor.

Owing to differences in the mechanisms by which data are collected in RWD and RCTs, close attention was given in our study to the transfer of clinical definitions used in the RCT to the real-world setting. A rigorous process of applying high-quality conceptual and operational definitions was implemented by subject-matter experts, including de novo development of a RWD-applicable algorithm to identify ISTH bleeding events [9]. Owing to mapping between coding systems (MedDRA and ICD/ATC and NDC) and harmonization of study variables, some deviation in reporting baseline characteristics of patients in PACIFIC-AF in this study and in the published article is inevitable [10].

Individual patient-level data collected in the Optum® EHR data set were sufficiently detailed to replicate the main elements of the PACIFIC-AF trial; however, some limitations associated with use of EHR data apply to this study. As with most secondary data sources, the Optum® EHR data set is vulnerable to misclassification of data, sampling bias, and potential limited follow-up time. To minimize the possibility of missing data in this study, only encounters in the healthcare system were used. The Optum® EHR data set lacks information on dispensation of drug prescriptions and on the exact date and cause of death, with the latter limiting the ability to assess mortality-related endpoints. Secure, de-identified linkage of EHRs with other data sources could potentially address some of these issues but was not implemented in this study.

The Optum® EHR data set covers the USA only, whereas patients in PACIFIC-AF were enrolled from 12 European countries, Canada, and Japan. Matching patients enrolled in PACIFIC-AF to those from the ECA-eligible patient pool on 101 variables was anticipated to eliminate differences in healthcare systems and population structure. Use of multiple RWD sources from various geographical regions should be considered for future cases to address population heterogeneity and the generalizability of the results; however, this might increase operational complexity.

It is also noted that a further limitation of this study is the use of EHR data from 2013 to 2019, with the PACIFIC-AF data being from as late as 2021. To assess the impact of the initial period of DOAC use, a sensitivity analysis was conducted using the EHR data starting from 2015; it demonstrated consistent results with the analysis of the EHR data starting from 2013.