Clinical Trials & Research Archives

Real World Evidence

Real World Evidence

Derived from data sources such as electronic health records, claims data, registries, and mobile devices, real world evidence (RWE) has the potential to bring innovative products to patients more quickly. Unlike randomized controlled trials (RCTs), which may not be representative of a general population due to strict inclusion and exclusion criteria, RWE may more closely identify how an investigational product will perform in a general population. To address the absence of widely accepted best practices for utilizing RWE, the MRCT Center collaborated with Duke-Margolis Center for Health Policy to define a framework to establish best practices for utilizing RWE for regulatory decision-making.

The MRCT Center also worked with OptumLabs to investigate sources of variability in RWE analysis as applied to replication (emulation) of RCTs. Conceptualized by the MRCT Center and OptumLabs, the Observational Patient Evidence for Regulatory Approval and uNderstanding Disease (OPERAND) Project aimed to understand the sources of variation in design, approach, methodologies, statistical analyses, and decision-making using real world evidence (RWE, specifically claims and EHR data) to emulate phase 3 clinical trials. First, RWE is used to replicate published clinical trials, using and analyzing data limited explicitly to the eligibility criteria of the trial. Later, the aperture of inclusion is broadened to determine any change in effect size by the broader population.

The OPERAND project convened a technical expert panel (TEP) comprising key stakeholders from industry, academia, and regulators, to consider the principles behind, the methodology to draw upon, and the appropriate utilization of, observational data in regulatory review and approval and comparative effectiveness research.

Brown University and Harvard Pilgrim Health Institute, were selected to replicate two trials:

(1) the ROCKET trial for atrial fibrillation, a multicenter, randomized, double-blind, double-dummy, event-driven trial that was conducted at 1178 participating sites in 45 countries) and
(2) the Lead-2 trial for Type 2 Diabetes control, a multi-center, randomized, double-blind, parallel assignment of 1091 participants conducted at 190 study sites across 4 continents).

The project helped validate the use of observational data to complement evidence from RCTs and used empirical data to understand methodologies and sources of variability.

Objectives

To develop empirical data to understand data quality – and the limitations of RWD – from various data sources (e.g., Claims, EHR) and the assumptions necessary to use such data for replication
To determine whether and how the addition of EHR to Claims data improves sensitivity and utility of data, and thus RWE utility
Following replication, to determine how RWE informs understanding of effectiveness for on-label indications in approved populations
To advance regulatory decision-making through RWE

key milestones

February 2021: Presentation about OPERAND at FDA/Duke Margolis virtual workshop Evaluating RWE from Observational Studies in Regulatory Decision-Making: Lessons Learned from Trial Replication Analyses
January 2021: With William Crown, co-authored Real-World Evidence: Understanding Sources of Variability Through Empirical Analysis, in Value in Health
December 2019: Duke-Margolis released Adding Real-World Evidence to a Totality of Evidence Approach for Evaluating Marketed Product Effectiveness
November 2019: Duke Margolis released Understanding the Need for Non-Interventional Studies Using Secondary Data to Generate Real-World Evidence for Regulatory Decision Making, and Demonstrating Their Credibility
September 2019: Duke-Margolis released Determining Real-World Data’s Fitness for Use and The Role of Reliability
June 2019: Grants announced for two academic institutions to replicate two clinical trials using real world data
August 2017: Launched Project OPERAND and held first conference call with the Technical Expert Panel (TEP)
March 2017: Started collaborating with Duke-Margolis Center for Health Policy to develop a framework for real world evidence for regulatory decision-making

Project leadership & Staff

Barbara Bierer, MD. Faculty Director, MRCT Center
William Crown, PhD. Chief Scientific Officer, Brandeis University (formerly Optum Labs)
Hayat Ahmed, MSc., Program Manager

Project Resources

Excluding People With Disabilities From Clinical Research: Eligibility Criteria Lack Clarity And Justification

Publication

Published on: October 3, 2022

Published in: Health Affairs

Abstract
The exclusion of people with disabilities from clinical research without appropriate justification is discriminatory, is counter to federal regulations and research guidelines, and limits study generalizability. This matter is understudied, and data on the disability status of trial participants are rarely collected or reported. We analyzed ninety-seven recent interventional protocols in four therapeutic areas registered on ClinicalTrials.gov. Eighty-five percent of protocols allowed broad investigator discretion to determine eligibility, whereas only 18 percent explicitly permitted people with disabilities to use forms of support (such as supported decision making or assistive devices) to facilitate study participation. Eligibility criteria affecting people with disabilities included exclusions for psychiatric (68 percent), substance use (62 percent), HIV or hepatitis (53 percent), cognitive or intellectual (42 percent), visual (34 percent), hearing (10 percent), mobility (9 percent), long-term care (6 percent), and speech and communication (3 percent) disability-related domains. Documented justification was provided for only 24 percent of these exclusions. We recommend greater scrutiny of study eligibility criteria, scientific or ethical justification of exclusions, and accessible study design.

View Article

Diversity, Inclusion, and Equity in Clinical Research: Integrating Considerations for Diversity, Equity, and Inclusion (DEI) into a Recruitment Strategy Document

Tools

Developed on: October 2022

Developed by: MRCT Center Diversity Workgroup

View Resource

2022-10-17-Recruitment-Strategy-Document_IRB

Related Resources

Procedural & Logistical Checklist

An IRB Resource for Participants: Costs and Payments

An IRB Resource for Investigators and Research Teams– Practical Points to Consider: Payment to Research Participants

An IRB Resource for Investigators and Research Teams: Including the Community Voice in Clinical Research

Incorporating DEI into Clinical Research Protocol Templates

Diversity & Inclusion Overlay: TransCelerate’s Common Protocol Template

Diversity & Inclusion Overlay: NIH-FDA Phase 2 and 3 IND/IDE Clinical Trial Protocol Template

See related Resources, maintained on our project specific website:
Diversity, Inclusion, and Equity in Clinical Research website:

IRB Resources

FDA Office of Minority Health and Health Equity Webinar Introducing MRCT Center’s “Achieving Diversity, Inclusion, and Equity in Clinical Research” Guidance Document and Supplemental Toolkit

September 22. 2022

Location: Virtual

Discussion topic: The FDA Office of Minority Health and Health Equity at the U.S. Food and Drug Administration (FDA) presents a webinar featuring Drs. Barbara E. Bierer (Faculty Director, MRCT Center; Professor of Medicine, Harvard Medical School) and Luther T. Clark (Deputy Chief Patient Officer, Merck).

Drs. Bierer and Clark will introduce the recently released MRCT Center’s Diversity Framework, titled Achieving Diversity, Inclusion, and Equity In Clinical Research. After providing a brief introduction, background, and contextual orientation to listeners, Drs. Bierer and Clark will highlight the ways in which stakeholders can contribute to increasing diverse representation in clinical research and, ultimately, improving health equity and public health domestically and abroad.

🎥 Watch webinar recording

⬇️ Download slides

MRCT-Center-Diversity-webinar-with-FDA-2020-09-22-min (1)Download

Return of Aggregate Results

Return of Aggregate Results

Returning results in plain language allows for investigators and sponsors to honor the essential contributions and voluntarism of study participants in multi-regional clinical trials, while improving the transparency of those trials. Returning the summary results of a clinical trial to participants involves communication—in language understandable to the participants—of the results of the trial and the outcome of the study as a whole.

Launched in 2013, the MRCT Center and its collaborators developed resources to lower barriers for returning results, created a number of useful tools, and published a guidance for the clinical trial community. The practical guidance document and toolkit were developed for use by all clinical trial sponsors, including academia, industry, non-profit and government organizations.

The workgroup first presented the results of its efforts at the U.S. Secretary’s Advisory Committee on Human Research Protections (SACHRP) and at the MRCT Center Annual Meeting in December 2014 and released the first version of its Guidance Document and Toolkit three months later. Updates to the Guidance Document and Toolkit were made in 2016 and 2017. Recommendations from the Guidance and Toolkit were incorporated into EMA regulation in 2017 and 2018 and the MRCT Center submitted draft guidance on Provision of Plain Language Summary to the FDA in 2017.

Appreciating the complexity of this topic, the MRCT Center has continued to remain abreast of challenges in the return of results space. We have explored the topic of returning summary results in comprehensive and integrative medicine, as well as launched separate projects delving deeper into returning individual results to participants, and addressing health literacy principles in clinical research that extend to the sharing of Plain Language Summaries. In 2023, the MRCT Center worked with Institutional Review Boards around the country to identify opportunities for returning results to participants within academic health centers and continues to advocate for a centralized location for the sharing of patient-centric study results summaries.

OBJECTIVES

Raise awareness of academia’s responsibility to fulfill the ethical obligation of returning results to participants
Promote greater understanding of procedural and operational requirements for returning aggregate results
Increase the frequency of returning results to participants for all types of studies
Reduce barriers to implementing return of results policies, with a focus on operational elements
Amplify patient/participant voice and preferences for receiving aggregate results

Key milestones

August 2022: Launched workgroup “Return of Results in Academia”
December 2018: Returning aggregate results of clinical trials: Empirical data of patient preferences, published in the Journal of Clinical and Translational Science
December 2017: Version 3.1 of MRCT Center Return of Aggregate Results to Participants Guidance Document and Version 3.1 of MRCT Center Return of Aggregate Results to Participants Toolkit released and Return of Aggregate Results to Participants Principles Version Nov 2017 released
October 2017: Submitted Draft Guidance on Provision of Plain Language Summaries to FDA
March 2017: Version 3.0 of MRCT Return of Aggregate Results Guidance Document and Version 3.0 of MRCT Return of Aggregate Results Toolkit released
December 2016: Return of Aggregate Results to Participants Principles Document released
July 2016: Version 2.1 of MRCT Return of Results Guidance Document and Version 2.2 of MRCT Return of Results Toolkit released
March 2016: Version 2.0 of MRCT Return of Results Guidance Document and Version 2.1 of MRCT Return of Results Toolkit released
October 2015: Version 2.0 of MRCT Return of Results Toolkit released
March 2015: Version 1.0 of MRCT Return of Results Guidance Document and Toolkit launched and harmonized with existing and ongoing efforts
December 2014: MRCT Center Annual meeting – working draft presented of Return of Results Guidance Document and Toolkit and panel discussants from EMA, FDA, OHRP and patient advocates discussed the MRCT Center approach to returning aggregate results to trial participants
January 2014: MRCT Center multi-stakeholder working group launched, comprised of 53 leaders in academia, industry, patient advocacy and health literacy
December 2013: MRCT Center Annual meeting – engaged a diverse multi-stakeholder group in identifying key issues and barriers to returning results and formulating tangible deliverables

Project leadership & Staff

The original project (2013-2017) was co-chaired by:

Barbara E. Bierer, MD. Faculty Director, MRCT Center
Deborah Collyar, BSc. President, Patient Advocates in Research
Laurie Myers, MBA. Global Health Literacy Director, Merck
Carmen Aldinger, PhD, MPH, PMP, Program Manager

The workgroup for the project update (2022 to current) is co-chaired by:

Barbara E. Bierer, MD. Faculty Director, MRCT Center
Sylvia Baedorf Kassis, MPH. Program Director, MRCT Center
Keren Dunn, Executive Director, Research Compliance & Quality Improvement, Cedars-Sinai
Ann Johnson, PhD, MPH, CIP. IRB Director, University of Utah
Liz Martinez, RN, BSN, CCRC. Johns Hopkins University
Kristin Bartlett, Program Coordinator, MRCT Center

Project Resources