See All Projects

Real World Evidence

Focus Area: Current Project

Focus Area: Global Regulatory Engagement

Real World Evidence

Derived from data sources such as electronic health recourse, claims data, registries, and mobile devices, real world evidence (RWE) has the potential to bring innovative products to patients more quickly. Unlike randomized controlled trials (RCTs), which may not be representative of a general population due to strict inclusion and exclusion criteria, RWE may more closely identify how an investigational product will perform in a general population. To address the absence of widely accepted best practices for utilizing RWE, the MRCT Center began collaborating with Duke-Margolis Center for Health Policy to define a framework to establish best practices for utilizing RWE for regulatory decision-making.

Further, the MRCT Center, in partnership with OptumLabs, designed project OPERAND (Observational Patient Evidence for Regulatory Approval and uNderstanding Disease) to better inform the use of Real World Evidence (RWE) from retrospective observational studies in regulatory decision-making and medicine.

The OPERAND project convened a technical expert panel (TEP) comprising key stakeholders from industry, academia, and regulators, to consider the principles behind, the methodology to draw upon, and the appropriate utilization of, observational data in regulatory review and approval and comparative effectiveness
 research. This is a project that will first use retrospective observational data to determine if such data can confirm previously published randomized clinical trial (RCT) results and, if successful, extend the use of the data for other potential applications.

While there are examples of individual RCTs that have been replicated using observational database analyses, there has never been an effort to replicate a large number of trials across several different therapeutic areas simultaneously. Such an effort is needed to shed light on several critical questions particularly relevant in retrospective analyses, including quality of data sources, bias (and ways to mitigate it), and variability in observational database studies. One important issue is the potential of data linkage (e.g., claims data linked with electronic medical records) to control for confounders that is missing in analyses based on claims data alone.

Current Status: Active Project

Anticipated Impact:

  • The proposed OPERAND initiative is focused on the role of observational data in safety studies and label extensions, but it also has direct applicability to decisions of payers and the clinical decision-making of physicians.
  • Ongoing comparisons of results from observational and randomized studies evaluations comparable treatment effects in similar patient populations will continue to be helpful in informing where we can productively rely upon evidence from observational studies versus where we should require randomization to address potential bias and confounding errors.


  • To determine whether observational studies using RWE replicate RCTs submitted for regulatory decision-making
  • To develop empirical data to understand data quality – and the limitations of RWD – from various data sources (e.g. Claims, EHR) and the assumptions necessary to use such data for replication
  • To determine whether and how the addition of EHR to Claims data improves sensitivity and utility of data, and thus RWE utility
  • To determine the sensitivities and variability of various statistical approaches given a common dataset and a common goal
  • Following replication, to determine how RWE informs understanding of effectiveness for on-label indications in approved populations

Key Milestones:

  • March 2017: Started collaborating with Duke-Margolis Center for Health Policy to develop a framework for real world evidence for regulatory decision-making
  • August 2017: Launched Project OPERAND and held first conference call with the Technical Expert Panel (TEP)
  • October 2017: First face-to-face TEP meeting to finalize study design and methods for Phase II approach
  • November 2017- June 2018: Additional TEP teleconferences to finalize study design and methods for Phase II approach. Request for Proposal (RFP) initiated and reviewed by TEP
  • February 2019: Research partners selected to replicate two trials
  • June 2019: Grants announced for two academic institutions to replicate two clinical trials using real world data
  • September 2019: Duke-Margolis released Determining Real-World Data’s Fitness for Use and The Role of Reliability

Project Leadership: Barbara Bierer (MRCT Center), Paul Bleicher (Optum Labs), William Crown (Optum Labs)

Project Manager: Hayat Ahmed