14 Real-World Evidence and Evidence Integration
The randomized controlled trial is the most reliable instrument for establishing that a treatment causes its observed effect. But randomization is not always possible, ethical, or sufficient. A new drug approved in women with hormone receptor-positive metastatic breast cancer may need a label extension for men with the same disease, a population of roughly 2,670 annual U.S. cases, too few to power a randomized trial. A vaccine approved in a clinical trial of healthy volunteers needs its effectiveness measured across a diverse population receiving it in community pharmacies. A drug studied in a three-year trial needs its long-term safety assessed over a decade. A rare pediatric disease may have only a few hundred patients worldwide, none of whom can ethically be assigned to placebo.
These situations are not edge cases. They describe a large and growing fraction of the evidence questions that drug development must answer. The response (drawing on data collected outside of controlled trials to generate evidence about drugs) has evolved over fifty years from an informal practice into a regulated discipline with its own methodology, infrastructure, and regulatory framework.
This chapter covers that discipline. Real-world evidence (RWE) is clinical evidence about the usage and potential benefits or risks of a medical product derived from analysis of real-world data (RWD): data collected outside of traditional randomized clinical trials, including electronic health records, insurance claims, disease registries, patient-reported outcomes, and wearable devices. The distinction matters: RWD is the raw data; RWE is the clinical evidence derived from rigorous analysis of it. Both terms appear in FDA guidance and are often used interchangeably in conversation, but the distinction reflects an important analytic step.
14.1 The Regulatory Framework
EMA and International Harmonization
The EMA’s approach is infrastructure-centric rather than document-centric. Rather than issuing guidance for sponsors to follow, EMA established DARWIN EU (Data Analysis and Real World Interrogation Network) as a coordination center for regulatory-led RWD studies. DARWIN EU pulls from healthcare databases across EU member states, including the UK’s CPRD (Clinical Practice Research Datalink, >60 million patient records), Spain’s BIFAP, Italy’s HSD, and the Danish national registries. These studies are commissioned by EMA scientific committees and fed to national competent authorities and HTA bodies; sponsors cannot commission DARWIN EU studies directly.
EMA’s April 2024 guide (EMA/152628/2024) describes how regulators can request RWD studies and what study types are available. In parallel, ICH published its first harmonization signal on RWE in May 2024: a reflection paper on “Pursuing Opportunities for Harmonisation in Using Real-World Data to Generate Real-World Evidence,” marking the beginning of international regulatory alignment on what has been a jurisdiction-by-jurisdiction conversation.
ICH E6(R3) (2023), the updated Good Clinical Practice guideline, explicitly addresses hybrid designs that incorporate EHR and RWD within interventional trials, extending principles of data integrity and traceability to these sources.
Fit-for-Purpose: The Governing Concept
Both FDA and EMA converge on the phrase fit for purpose as the evaluative standard: not “is this data from a rigorous source?” but “is this data adequate for this specific regulatory question?” A registry dataset that is fit for purpose as an external control for a natural history comparison in a rare pediatric disease may not be fit for purpose as the primary comparator for a Phase III efficacy claim in a common chronic condition.
The fit-for-purpose assessment evaluates data relevance (population, setting, outcomes, time period), data reliability (completeness, accuracy, consistency, provenance), and analytic validity (whether the proposed analysis can extract the target estimand from the available data without biases that exceed what is acceptable for the regulatory purpose). Sponsors who engage with FDA early (through pre-submission meetings under the Advancing RWE Program) have substantially better outcomes than those who submit RWE for the first time in a marketing application.
HTA and Payer Frameworks
Regulatory approval and market access are separate decisions, governed by different evidence standards. FDA determines whether a drug is safe and effective for the proposed indication. HTA bodies such as NICE (National Institute for Health and Care Excellence, UK) determine whether it is cost-effective relative to standard of care, for a specific patient population in a specific healthcare system. These require different evidence and create different RWE demands.
| Dimension | Regulatory (FDA/EMA) | HTA/Payers (NICE, payers) |
|---|---|---|
| Central question | Does it work under trial conditions or real-world use? | Is it cost-effective in this health system? |
| Comparator | Placebo or trial-specified active comparator | Actual standard of care in that country |
| Time horizon | Trial duration | Lifetime (requires extrapolation) |
| Population | Label indication | Patients who will receive reimbursement |
| RWE role | Label expansion, post-approval commitments | Fills long-term effectiveness gaps; informs HTA models |
NICE’s RWE framework (ECD9, June 2022) provides structured guidance for RWE studies submitted in technology appraisals. The concept of managed entry agreements (MEAs), conditional reimbursement contingent on accumulation of real-world evidence confirming effectiveness in routine practice, has become common in Europe. Under MEAs, the drug is initially reimbursed at a discounted price while an agreed-upon RWE study collects data; if the study confirms effectiveness, full reimbursement is established; if not, the agreement is revised. This creates a regulatory-HTA interface where the same RWE study must meet the evidentiary standards of both bodies.
14.2 Data Sources
The quality of real-world evidence is only as good as the data it comes from. Understanding the major RWD source types (what they contain, what they lack, and what biases they introduce) is prerequisite to evaluating any RWE study.
Electronic Health Records capture diagnoses, procedures, medications, laboratory values, vital signs, and clinical notes. They are the richest source of clinical detail, but their coverage is bounded by the patient’s engagement with a particular healthcare system: a patient who sees a cardiologist at a different hospital does not appear in the EHR. U.S. EHR networks of regulatory-grade scale include Flatiron Health (>1.4 million oncology patients across ~280 cancer clinics, with structured abstraction of tumor response and genomic data), Epic Cosmos (>200 million patients across Epic-using health systems), and PCORnet (100+ health systems). In the UK, CPRD provides >60 million patient records from UK general practice. Critically, EHR data are collected for clinical care, not research: coding completeness and measurement fidelity are driven by billing incentives and workflow conventions, not study protocols.
Administrative claims and insurance data capture billing transactions: ICD diagnosis codes, procedure codes, drug dispensing (NDC codes), and enrollment periods. Coverage is population-level for the insured population; data are complete within the coverage period; and follow-up can be long. U.S. sources include Medicare (CMS, covering >65 million beneficiaries), Medicaid, and commercial claims (e.g., Optum Clinformatics, Merative MarketScan, and IQVIA). The tradeoff is clinical depth: claims capture what was billed, not what the patient’s hemoglobin was or how advanced their tumor was.
Disease and condition registries are purpose-built prospective or retrospective data systems that collect uniform data on populations defined by a disease, exposure, or condition. They sit between clinical trials and EHR in terms of data quality: enrollment criteria are defined, data elements are standardized, and outcomes are often adjudicated. The Surveillance, Epidemiology, and End Results (SEER) program covers ~35% of the U.S. population across 18 cancer registries. The Scandinavian cardiac registries (SCAAR, SWEDEHEART) have been the foundation for large-scale registry-based randomized trials. Natural history registries (the Pompe Registry, the Cystic Fibrosis Foundation Patient Registry, rare disease-specific databases) serve a distinct role in rare drug development as the comparator population for single-arm trials.
Linked databases combine two or more data sources using patient identifiers. SEER-Medicare links cancer diagnoses with complete Medicare claims, enabling studies of both clinical outcomes and health service utilization in a population where both are relevant. Flatiron Health linked its EHR with Foundation Medicine’s comprehensive genomic profiling (Flatiron+FMI), enabling studies of genomic markers as predictors of real-world treatment outcomes. Linkage quality, whether the linkage is deterministic (exact identifier match) or probabilistic and what the linkage error rate is, directly affects bias.
Global RWD networks enable federated analysis across many institutions. The FDA’s Sentinel System maintains over 1.3 billion person-years of U.S. administrative claims data (covering more than 371 million individuals as of 2024) for active post-market safety surveillance; it is now expanding to some effectiveness analyses. DARWIN EU (described above) serves the analogous function for EMA. The OHDSI/OMOP network is a global open-science consortium of 544 data sources across 54 countries representing >974 million unique patient records, standardized to the OMOP Common Data Model. OMOP standardizes the vocabulary (SNOMED, RxNorm, LOINC) and table structure, enabling identical analysis code to be run across distributed sites without patient data leaving any institution. PCORnet links >100 U.S. health systems using its own PCORnet Common Data Model (derived from the FDA Sentinel CDM) and serves as the infrastructure for embedded pragmatic trials.
Patient-level data sharing platforms provide access to IPD (individual patient data) from completed clinical trials, enabling external control construction from historical trial patients. Vivli hosts >6,000 clinical trials; the YODA Project (Yale Open Data Access) provides third-party review of data sharing requests for J&J and Medtronic datasets; Project Data Sphere focuses on oncology trial control arm data. These are qualitatively different from EHR or claims because the data were collected under trial conditions with standardized protocols.
Outcome Ascertainment: The Critical Quality Issue
A structural weakness of most RWD sources is that outcomes were not designed to be measured. The diagnosis code for “disease progression” in an EHR reflects what a physician documented for billing purposes, not a protocol-specified assessment against RECIST criteria at a pre-specified timepoint. The gap between documented and protocol-adjudicated outcomes is the source of what are called real-world endpoints: rwOS (real-world overall survival), rwPFS (real-world progression-free survival), and rwTR (real-world tumor response). These are not equivalent to their trial-measured counterparts.
The Friends of Cancer Research established a framework for validating real-world endpoints: the correlation between the real-world endpoint and the regulatory-standard trial endpoint must be demonstrated empirically before the real-world endpoint is used in a regulatory submission, not assumed. Flatiron Health has published validation studies comparing rwOS and rwPFS against clinical trial outcomes across multiple tumor types.
Phenotyping algorithms (computable definitions of conditions, exposures, and outcomes applied to RWD) require validation against chart review. The positive predictive value of ICD coding for specific conditions varies widely (excellent for hip fracture, poor for depression subtypes) and varies across care settings. Non-differential outcome misclassification biases effect estimates toward the null; differential misclassification (where the misclassification rate differs between treatment groups) can bias in either direction. Any serious RWE study should report its phenotyping algorithm, validation statistics (PPV, sensitivity, F1 against chart review), and a sensitivity analysis for outcome misclassification.
14.3 Study Design Taxonomy
RWE studies span a wide range of designs. Understanding the taxonomy is essential because the design determines the appropriate analytic strategy, the achievable estimand, and the regulatory evidence tier.
Observational Comparative Effectiveness Research
In observational CER, both treatment groups are drawn from RWD; there is no randomization. Patients who received Drug A in routine clinical practice are compared to patients who received Drug B. The central challenge is confounding: patients and their physicians chose their treatments for reasons that may also predict outcomes. Analytic methods (propensity scores, outcome regression, doubly robust estimators) attempt to control for measured confounders; unmeasured confounders remain the irreducible limitation.
Observational CER is most credible when: the outcome is objective and reliably captured in routine care (overall survival is better than symptom scores); the treatment comparison is between two established alternatives rather than between treated and untreated (which concentrates confounding by indication); the database is large enough to achieve covariate balance after matching or weighting; and sensitivity analyses demonstrate that the result is robust to plausible unmeasured confounding.
Natural History and Registry Studies
Natural history studies characterize disease progression, event rates, and mortality in untreated or standard-of-care populations. They do not compare treatments; they establish the baseline against which a new treatment’s effect will be measured. In rare disease drug development, a well-designed natural history study is often the indispensable prerequisite for a single-arm trial: without a reliable characterization of the untreated disease course, there is no benchmark against which the single arm’s outcomes can be interpreted.
FDA has issued guidance on registry fit-for-purpose assessment (2023), addressing the data elements, follow-up duration, and outcome adjudication needed for a registry to serve as a regulatory-grade external control comparator.
External Control Arms and Synthetic Control Arms
An external control arm (ECA) replaces the concurrent randomized control arm with a control population constructed from RWD. Patients in the single-arm trial receive the investigational treatment; their outcomes are compared to a pre-specified external control cohort drawn from historical trials, registries, EHR, or claims. The FDA’s 2023 guidance on externally controlled trials specifies that ECAs are most appropriate when: concurrent randomization is infeasible or unethical; the disease has a well-characterized natural history; outcomes can be measured reliably in the external data source; and the comparison population is demonstrably similar to the trial population at baseline.
A synthetic control arm (SCA) is a specific implementation in which patient-level data from historical trials (often proprietary datasets) are statistically matched to the current trial’s population using propensity score or covariate-matching algorithms. Medidata’s platform, drawing on >38,000 trials and 12 million patients, is one of the largest commercial SCA systems. In a widely cited application around 2020, Medicenna Therapeutics used a hybrid synthetic control arm in its MDNA55 program for recurrent glioblastoma to supplement a single-arm trial, substantially reducing the number of prospectively enrolled control patients. SCAs drawn from historical RCT patient data are generally stronger than ECAs from EHR or claims because the data were collected under controlled conditions.
Hybrid Randomized Designs and Bayesian Augmented Controls
Hybrid designs randomize a reduced concurrent control arm while borrowing information from external data to increase effective sample size. The borrowing is implemented through Bayesian methods that “discount” the external data based on how consistent it is with the emerging concurrent control data.
Three Bayesian borrowing frameworks dominate:
The power prior (Ibrahim and Chen 2000) modifies the likelihood of the historical data by a parameter \(\alpha \in [0,1]\): \(\alpha = 0\) borrows nothing, \(\alpha = 1\) fully pools historical and current data. A fixed \(\alpha\) chosen before the trial controls the degree of borrowing; a data-adaptive \(\alpha\) estimated from the data consistency is more robust to historical-current heterogeneity but introduces operational complexity.
The Meta-Analytic-Predictive (MAP) prior (Neuenschwander et al. 2010) derives the prior from a meta-analysis of multiple historical studies, accounting for between-study heterogeneity through a hierarchical model. The resulting prior distribution on the control event rate reflects both the central estimate from historical data and the uncertainty about how much the current trial population may differ. MAP priors are implemented in the RBesT R package and have been used in pediatric trials, where adult trial data informs the prior for smaller pediatric randomized comparisons.
Commensurate priors (Hobbs et al. 2011) make the amount of borrowing a function of the observed similarity between historical and current control arm data, updating dynamically as current data accumulates. When historical and current control arms are concordant, borrowing is high; when they diverge, borrowing shrinks. This dynamic adaptation provides protection against the principal risk of Bayesian borrowing: inadvertently introducing bias by over-weighting historical data that is not representative of the current population.
The effective sample size contributed by borrowed external data depends on the discount applied; regulatory agencies expect pre-specified borrowing parameters with operating characteristics evaluated through simulation across scenarios of historical-current concordance and discordance.
Registry-Based Randomized Controlled Trials
A registry-based RCT (RRCT) conducts randomization within an existing clinical registry, using registry-collected outcomes rather than dedicated trial data collection. The TASTE trial (2013-2014) randomized 7,244 patients with ST-elevation myocardial infarction to thrombus aspiration versus conventional PCI within the SCAAR Swedish cardiac registry; outcomes were collected from registry records without source data verification. The VALIDATE-SWEDEHEART trial compared bivalirudin versus heparin in acute coronary syndrome within the same registry infrastructure. Both trials were published in the New England Journal of Medicine and replicated the operational efficiency of registry-based data collection with the inferential validity of randomization.
RRCTs are most powerful when: an existing registry covers the relevant disease and collects the primary outcome reliably; the randomization can be executed at the point of care within the registry interface; and the registry’s outcome ascertainment is sufficiently standardized for the primary comparison. The Scandinavian registries (supported by universal personal identification numbers enabling linkage across data sources) have been the paradigmatic setting, but RRCT methodology is applicable wherever a well-maintained registry exists.
Platform Trials and External Data
Platform trials (multi-arm adaptive protocols that can add and remove treatment arms over time) intersect with RWE in three distinct ways. (The full treatment of platform trial design is in Chapter 12; this section addresses the RWE-specific aspects.)
External control arms in biomarker-selected platforms. Basket trials study one drug across multiple tumor types sharing a molecular alteration; umbrella trials study multiple drugs in one tumor type stratified by biomarker. Individual biomarker-defined cohorts may be too small for internal randomization. In these settings, the platform uses an external control arm (a historical or registry-based comparator) for the subgroup too small to randomize. The FDA master protocols guidance (U.S. Food and Drug Administration 2022) explicitly addresses the design requirements for external controls within basket and umbrella structures, including pre-specification of the external population and comparability requirements. Oncology platforms such as NCI-MATCH and LUNG-MAP have incorporated external control comparators for molecularly selected cohorts below the threshold for internal randomization.
Registry-based outcome collection in platform designs. The RECOVERY trial (Randomised Evaluation of COVID-19 Therapy) enrolled >40,000 patients across NHS hospitals in the UK and collected primary outcomes through NHS electronic records rather than dedicated data collection, an embedded pragmatic design at platform scale. This approach reduces operational burden dramatically: coordinators enroll patients electronically, treatment is assigned, and outcomes are collected automatically from the health system’s administrative records. The REMAP-CAP trial (Randomised, Embedded, Multifactorial Adaptive Platform trial for Community-Acquired Pneumonia) similarly collected outcomes through ICU data systems. These designs achieve the representativeness of registry data combined with the inferential validity of randomization, the RRCT concept applied to a multi-arm adaptive platform.
Bayesian borrowing across platform domains. In multi-domain platforms that test multiple interventions simultaneously (immunological, anticoagulation, and antiviral domains in REMAP-CAP), the shared concurrent control arm accumulates data across all enrolled patients regardless of which domain they participate in. Bayesian hierarchical models can borrow information across concurrent substudies, effectively treating the platform as generating external-control-equivalent data for each sub-comparison. This is the adaptive trial version of the dynamic borrowing concepts described above, applied within a single protocol infrastructure rather than across separate trials.
Embedded and Pragmatic Trials
Pragmatic trials assess treatments under conditions approximating routine clinical practice: broad eligibility criteria, usual care comparators, real-world outcomes, and heterogeneous patient populations. Embedded pragmatic trials (EPTs) go further: they are embedded within healthcare system infrastructure, using EHR for recruitment, randomization, and outcome collection. IMPACT-AFib is a canonical U.S. example: EHR identified atrial fibrillation patients who had never received anticoagulation, cardiology practices were randomized to receive or not receive automated patient outreach, and outcomes (anticoagulation initiation) were collected from claims without research staff involvement.
FDA’s 2024 guidance “Integrating Randomized Controlled Trials for Drug and Biological Products Into Routine Clinical Practice” specifically addresses EPTs, clarifying the regulatory requirements for EHR-based recruitment and outcome collection. PCORnet (the U.S. patient-centered clinical research network covering >100 health systems) and the NIH Health Care Systems Research Collaboratory have built the infrastructure enabling EPTs at scale.
14.4 Pharmacoepidemiology Design Principles
Observational RWE studies are subject to a set of biases that have no analog in randomized trials. These biases arise from the way treatment is assigned in clinical practice and from the structure of RWD sources. Understanding them is prerequisite to critically reading or designing an RWE study.
Confounding by indication is the most pervasive. Physicians assign treatments based on patient characteristics that may also predict outcomes: sicker patients get more aggressive treatment; frail patients get less. If those characteristics are not fully captured in the RWD, the treatment comparison is contaminated by the indication. The active comparator, new user (ACNU) design is the principal design-level solution.
The active comparator design restricts the comparator population to patients who received a different active treatment for the same indication at the same time, rather than patients who received no treatment. If both drugs are used for the same condition in the same clinical context, much of the confounding by indication cancels: the decision between Drug A and Drug B is less driven by disease severity than the decision to prescribe any drug. Active comparator designs substantially reduce unmeasured confounding relative to treated/untreated comparisons.
The new user design restricts the cohort to patients who newly initiated treatment (no use in a washout period, typically 6-12 months). It eliminates prevalent user bias: prevalent users have already survived early adverse events, been selected as tolerating the drug, and represent a systematically different population than incident users. The new user restriction aligns cohort entry with the start of treatment, enabling correct measurement of baseline covariates before exposure and mirroring the structure of a randomized trial where treatment-naïve patients are enrolled.
Immortal time bias occurs when the interval between cohort entry and first treatment exposure is incorrectly classified as follow-up time in the treated group. During this interval, the patient could not have died (they are still alive to receive treatment), creating an artificial survival advantage for the treated group. Suissa’s 2008 paper is the definitive description (Suissa 2008); the target trial emulation framework (see Chapter 11) prevents immortal time bias by construction through explicit alignment of the eligibility date with the treatment start date.
Index date alignment requires that baseline covariates be measured before the index date (treatment initiation) and outcomes be measured after. Measuring a covariate that could have been caused by treatment (post-initiation lab value) and using it for confounder adjustment induces collider bias. This sounds straightforward but is routinely violated in practice when analysts use “all available” data without strict time-ordering.
Depletion of susceptibles (also called prevalent user bias in the context of comparisons): in studies comparing two drugs, if early users of Drug A who experienced adverse events subsequently switched to Drug B, the Drug B user population is enriched with tolerators and depleted of susceptibles, creating an apparent safety advantage for Drug B. The new user restriction applied to both exposure groups eliminates this.
14.5 Statistical Methods for Real-World Evidence
The statistical toolkit for RWE is broader and more methodologically heterogeneous than for randomized trials. Chapter 11 covers the foundational methods; this section situates them in the RWE context and introduces the additional methods specific to observational causal inference.
Causal Diagrams as the Starting Point
Before choosing an analytic method, the analyst must understand the causal structure of the problem: which variables are confounders (shared causes of treatment and outcome that must be controlled), mediators (on the causal pathway from treatment to outcome, where controlling them would block the effect of interest), and colliders (common effects of treatment and outcome, where controlling them opens a spurious association). Directed Acyclic Graphs (DAGs) provide the formal language for this (Pearl 2009). A DAG-based analysis specifies the minimum sufficient adjustment set for the causal estimand, and identifies which variables should not be controlled (mediators in many contexts, colliders in all contexts).
DAGs are now standard in pharmacoepidemiology training and are increasingly expected in regulatory submissions involving RWE. The software tool DAGitty makes DAG construction and analysis accessible without requiring formal training in graphical models.
Propensity Score and Doubly Robust Methods
Propensity score (PS) methods (covered in Chapter 11) are the workhorses of observational RWE. In the RWE context, the PS model should be fitted to predict treatment assignment from baseline covariates measured before the index date, with attention to variable selection (include all variables associated with outcome; exclude instruments; be cautious about variables associated only with treatment). Balance diagnostics, specifically standardized mean differences (SMD) before and after adjustment with SMD < 0.1 considered acceptable, should be reported and not assumed.
Doubly robust estimators (AIPW, g-computation with IPW, TMLE) are more resistant to model misspecification than PS-only or outcome-model-only approaches, and should be preferred for primary analyses when the analysis is high-stakes. Targeted Maximum Likelihood Estimation (TMLE) combines machine learning-based estimation of both the PS and the outcome model with a targeted bias-correction step that guarantees valid inference even when data-adaptive models are used.
Instrumental Variables
When unmeasured confounding is suspected, instrumental variables (IV) provide an alternative identification strategy. An instrument is a variable that: (1) affects treatment; (2) affects the outcome only through its effect on treatment; (3) is not caused by any unmeasured confounders. In pharmacoepidemiology, instruments include physician prescribing preference (some physicians systematically prefer Drug A, independent of patient characteristics), formulary changes that shift prescribing toward one drug based on calendar time, and geographic discontinuities in access.
IV identifies the local average treatment effect (LATE), the effect in the subpopulation whose treatment status is changed by the instrument, not the average treatment effect in the full population. This limitation matters for generalizability. IV is most useful when a strong instrument can be identified and when the LATE is clinically meaningful; weak instruments produce imprecise estimates.
Difference-in-Differences and Regression Discontinuity
Difference-in-differences (DiD) compares the pre-to-post change in outcomes in a treated group to the pre-to-post change in an untreated group. It controls for time-invariant unmeasured confounders by differencing them out. The key identifying assumption is parallel trends: absent the treatment, the treated and untreated groups would have followed the same trajectory. This assumption is testable using pre-treatment periods and should be reported, not assumed. DiD is common in policy evaluation and increasingly used in pharmacoepidemiology for studies of formulary changes, guideline implementations, or drug market entries.
Regression discontinuity (RD) exploits threshold-based treatment assignment: patients scoring just above a clinical cutoff receive treatment; patients just below do not. Near the cutoff, treated and untreated patients are approximately comparable. RD identifies a local treatment effect at the threshold, useful when such thresholds exist in clinical practice (APACHE score thresholds for ICU admission, lipid level thresholds for statin prescribing, age-based eligibility cutoffs).
Unmeasured Confounding Assessment
No observational method fully eliminates unmeasured confounding. The appropriate response is to quantify how strong an unmeasured confounder would need to be to explain away the observed result.
The E-value (VanderWeele & Ding, 2017, covered in Chapter 11) is now standard reporting practice in high-quality RWE studies. A large E-value means the observed result could only be explained away by a very strong unmeasured confounder, increasing confidence in the causal interpretation.
Negative control outcomes and exposures are outcomes or exposures known to have no causal relationship with the treatment under study. If the analysis finds an association with a negative control, the analytic method is producing spurious results (systematic bias), and the estimate for the primary outcome is unreliable. Negative controls are a powerful validity check that is underutilized in RWE submissions.
Quantitative bias analysis (QBA) uses probabilistic sensitivity analysis, specifically Monte Carlo simulation over plausible values of unmeasured confounder prevalence and effect size, to produce a bias-adjusted distribution of the effect estimate rather than a single point estimate assuming no unmeasured confounding.
14.6 Landmark Applications
Oncology: Ibrance for Male Breast Cancer
The 2019 FDA acceptance of Pfizer’s supplemental NDA for palbociclib (Ibrance) to include male breast cancer in the indication is the canonical landmark case for RWE-supported label expansion. Male breast cancer occurs in approximately 2,670 U.S. patients annually, far too few to power a randomized trial against placebo or even an active comparator. Pfizer assembled real-world evidence from three sources: Flatiron Health’s oncology EHR (real-world tumor response, treatment patterns), IQVIA insurance data (treatment patterns, concomitant medications), and Pfizer’s global post-marketing safety database (adverse event profiles in male patients).
FDA accepted this package as primary evidence to support the supplemental approval, with label language noting it was “based on limited data from post-marketing reports and electronic health records.” The label language is instructive: FDA did not characterize the evidence as equivalent to a randomized trial; it was transparent about the evidence basis while accepting its adequacy for the specific regulatory question (whether the drug’s pharmacology and clinical profile in men are consistent with benefit in women, where randomized evidence exists).
COVID-19 Vaccine Effectiveness at Scale
The COVID-19 pandemic produced the largest-scale rapid deployment of RWE in regulatory history. As vaccines were authorized under Emergency Use Authorization based on clinical trial data in 2020-2021, real-world effectiveness needed to be measured across diverse populations, viral variants, and healthcare settings, questions the trials could not answer in time.
The dominant design was the test-negative design (TND): among all individuals tested for COVID-19 with PCR, compare vaccination status between those who tested positive and those who tested negative. The TND controls for healthcare-seeking behavior (both positive and negative test results come from people who sought testing) while estimating vaccine effectiveness against symptomatic disease. Lopez Bernal et al. used this design in England’s national surveillance data to estimate BNT162b2 effectiveness against the Delta variant (Lopez Bernal et al. 2021). Dickerman et al. used target trial emulation in U.S. veterans data to compare BNT162b2 versus mRNA-1273 effectiveness directly, demonstrating the methodology at continental scale (Dickerman et al. 2022).
Waning immunity was documented observationally months before any randomized follow-up data was available, directly informing booster policy decisions. The COVID-19 experience established that RWE can provide actionable regulatory-grade evidence in a compressed timeframe when the methodological infrastructure is in place.
Registry-Based Trials: TASTE and SWEDEHEART
The TASTE trial (Fröbert et al. 2013) and VALIDATE-SWEDEHEART demonstrated that randomized evidence could be generated at massive scale (7,000+ patients) within existing cardiac registry infrastructure in Sweden, with outcomes collected entirely from registry records. Operational costs were a fraction of traditional Phase III, and the enrolled population was more representative of clinical practice than the carefully selected populations in traditional trials.
These trials opened a methodological option that is now being replicated internationally: where a well-characterized registry exists, the marginal cost of randomization (a software feature added to the registry interface) is low. The resulting evidence has all the inferential advantages of randomization combined with the population representativeness of observational data.
14.7 Infrastructure: OMOP, OHDSI, and Federated Analytics
The methodological machinery of RWE requires computational infrastructure to execute at scale. The OHDSI (Observational Health Data Sciences and Informatics) community has developed an open-science ecosystem built on the OMOP Common Data Model that makes large-scale, multi-database, standardized RWE feasible.
The OMOP Common Data Model standardizes the structure and vocabulary of observational databases: the same table structure and the same coding system (SNOMED CT for conditions, RxNorm for drugs, LOINC for measurements) across all network sites. Identical analysis code (written once) can be run across all 544+ OHDSI network sites representing >974 million unique patients in 54 countries. Patient data never leave the institution; only aggregate results are shared.
The HADES toolkit (Health Analytics Data-to-Evidence Suite) provides 37 R packages for executing standardized epidemiological analyses on OMOP-format data: cohort construction, propensity score estimation, outcome modeling, negative control calibration, and diagnostics. PhenoLibrary maintains a repository of validated phenotype algorithms with distributed validation statistics from multiple sites.
For cases where even aggregate results create privacy risk (rare diseases, small populations, genomic data), federated learning provides an alternative: a machine learning model is trained across distributed sites without sharing patient data, with only model gradients (not data) transmitted. This is more complex than distributed analytics but enables model-level analysis in privacy-sensitive settings.
The DARWIN EU network uses OMOP-compatible standards across EU member states, enabling FDA-aligned analytic methods to be applied to European data in a regulatory-grade framework.
14.8 Generalizability and Transportability
Even a perfectly unconfounded RWE study estimates the treatment effect in the study population. Whether that estimate generalizes to the target population (the patients who will actually receive the drug) is a distinct question, called generalizability (from the study sample to the study population) and transportability (from the study population to a different target population).
Dahabreh and colleagues developed a formal framework for combining evidence from randomized trials and observational data to estimate treatment effects in a target population that extends beyond the trial’s eligibility criteria (Dahabreh et al. 2019, 2020). The key idea: if the factors that make the trial population differ from the target population are measured and modeled, inverse probability of sampling weights can be used to re-weight the trial evidence to represent the target population.
This matters for drug development in two ways. First, trial eligibility criteria systematically exclude older patients, those with comorbidities, and those on concomitant medications, but these patients will receive the drug after approval. Transportability methods provide a principled way to estimate whether the trial effect size translates to the real-world population, which has implications for both labeling claims and post-market commitments. Second, when a drug is approved in one country and sought for approval or HTA in another, the evidence from the approval trial may not transport directly to the target country’s patient population; MAIC (Matching-Adjusted Indirect Comparison) is the widely used HTA implementation of this idea.
14.9 When Does RWE Belong in a Development Program?
The conditions that maximize RWE credibility have an underlying logic: confounding is the irreducible threat, and each favorable condition reduces it in a different way.
The most powerful reduction comes when the outcome itself is not subject to differential ascertainment. Overall survival (death is reliably recorded regardless of which arm the patient was in), hospitalization (a billing event that appears in administrative data for all patients), and treatment dispensing (a pharmacy record) are captured with near-complete ascertainment in most RWD sources. Symptom scores, imaging-defined tumor progression, and patient-reported outcomes depend on protocol-mandated assessments that do not exist in routine care. When the outcome is objective, the main confounding challenge shifts fully to treatment selection; when the outcome is also subjectively ascertained, outcome measurement bias compounds the confounding problem.
Rare diseases and rare populations reduce confounding by narrowing treatment options. When there is no effective standard of care, comparing treated patients against natural history concentrates far less confounding by indication than comparing two marketed alternatives where physicians select treatment based on prognosis. Pediatric oncology and rare hereditary diseases, where the natural history is the only available comparison, represent the settings where regulatory agencies have been most receptive.
External validation of internal evidence is stronger than either alone. An RCT demonstrating efficacy supplemented by RWE demonstrating real-world effectiveness is substantially more compelling than RWE as the primary evidence. FDA’s endorsement of emulation benchmarking studies, which test whether RWE analyses can replicate the findings of completed RCTs in the same setting, reflects this logic: where concordance has been established, regulatory confidence in RWE for similar questions increases.
All of this depends on pre-specification. FDA and EMA expect the external control population, the analytic methods, the confounders to be controlled, and the sensitivity analyses to be specified before any unblinding of the internal trial data. A retrospective RWE analysis decided after seeing the trial result provides much weaker evidence than a prospectively specified analysis completed before the primary result is known.
The Advancing RWE Program’s explicit goal is to extend regulatory acceptance of RWE into effectiveness labeling changes, a domain where it has historically been resisted. The Ibrance precedent, the COVID-19 vaccine experience, and the growing body of emulation benchmarking studies (which test whether RWE analyses using observational data can replicate the findings of known randomized trials) are collectively moving the evidentiary floor. Where those benchmarking studies have found concordance (particularly in oncology with hard endpoints), regulatory confidence in RWE has increased.
Real-world evidence does not replace randomized trials. It answers the questions that trials cannot: what happens to a broader population over a longer period under the conditions of clinical practice, using endpoints that routine care can measure. The methodological and regulatory infrastructure for generating credible RWE is now substantial, and its role in drug development (from informing trial design through supporting regulatory submissions to satisfying post-market commitments and HTA requirements) will continue to expand as that infrastructure matures.
A sponsor is preparing a supplemental NDA for a targeted therapy in a rare solid tumor (approximately 800 annual U.S. cases) and plans to use Flatiron Health oncology EHR data as the primary external control. FDA’s fit-for-purpose framework requires evaluation of data relevance and reliability. Walk through the specific questions FDA would ask about this data source for the proposed regulatory question, and identify the two most likely reasons the submission could be rejected on data-quality grounds.
A retrospective study comparing a newer oral anticoagulant to warfarin uses a prevalent-user cohort (patients who have been on treatment for a median of 14 months at baseline) and starts follow-up from the date of first database capture rather than from treatment initiation. Explain why prevalent user bias and immortal time bias are present in this design, how each one would bias the effect estimate, and what design changes would correct both.
An analyst emulating a hypothetical RCT comparing two antihypertensive drug classes using claims data sets the index date as the date of the first dispensing of the study drug. A reviewer notes that some patients have lab values recorded between the first prescription date and the first dispensing date, which the analyst included as baseline covariates. Why does this violate the target trial emulation framework’s requirement for correct time-zero alignment, and what specific bias does it introduce?
A hybrid Bayesian design for a rare hematologic malignancy uses a MAP prior derived from three historical trials. The concurrent control arm enrolls 30 patients. At the interim analysis, the observed control arm event rate is 40% higher than the historical mean. The prior has a relatively narrow effective sample size of 25. Describe what happens to the posterior estimate of the control arm event rate, how commensurate versus fixed power priors would respond differently to this discordance, and what you would recommend the DSMB do with this information.
NICE is evaluating a KRAS G12C inhibitor for second-line non-small cell lung cancer under a managed entry agreement. The sponsor must design a real-world evidence study to confirm effectiveness in routine NHS practice within 24 months of conditional reimbursement. What study design would you propose (specifying data source, population, comparator, outcome, and analytic method), and what are the two greatest threats to the study’s ability to produce a result that NICE will accept as confirmatory?