Modeling Strategies for Observational Comparative Effectiveness Research: What Works Best When? [Methods Study], 2013-2018 (ICPSR 39479)
Version Date: Sep 2, 2025 View help for published
Principal Investigator(s): View help for Principal Investigator(s)
Douglas Landsittel, University of Pittsburgh
https://doi.org/10.3886/ICPSR39479.v1
Version V1
Summary View help for Summary
Comparative effectiveness research compares two or more treatments to see which one works better for which patients. In some studies, researchers assign patients by chance to several treatments or to have or not have a treatment. But approaches that assign patients by chance are not always suitable. For example, assigning patients to a new treatment may not be good medical care.
For this reason, researchers sometimes do studies using data collected when patients and their doctors choose the treatments. Data from such studies are observational data. When using observational data for research, it can be hard to know if the effect of a treatment is because of the treatment or other factors, such as patients' age or health history. In these cases, researchers use statistical methods to understand the effect of the treatment. Depending on the study's focus and design, some methods work better than others.
In this study, the research team developed guidance for researchers to help them choose methods for their study.
To access the methods and software, please visit the DECODE CER Tool website.
Citation View help for Citation
Export Citation:
Funding View help for Funding
Subject Terms View help for Subject Terms
Geographic Coverage View help for Geographic Coverage
Distributor(s) View help for Distributor(s)
Study Purpose View help for Study Purpose
To develop guidance for researchers for selecting and applying appropriate statistical methods to analyze observational data in comparative effectiveness research (CER). The study aims were the following:
- systematically review existing literature and seek stakeholder input to identify studies from the literature on methods for causal inference in the setting of observational data used for CER;
- analyze simulated data to further characterize statistical properties of common propensity score-based methods;
- develop and disseminate a Decision Tool for Causal Inference and Observational Data Analysis Methods in Comparative Effectiveness Research (DECODE CER) to guide clinical researchers and statisticians in using PS-based methods and instrumental variables (IVs).
Study Design View help for Study Design
Clinical researchers analyzing observational data face challenges in trying to determine which statistical analysis options are appropriate for supporting causal inferences, given their specific questions and data. Options include propensity score matching; weighting and subclassification; doubly robust methods; instrumental variables; and methods for handling data with treatment regimens that could change over time, or time-varying treatments. Studies indicate that the method used may affect the findings. For results that are consistent with the goals of a study, it is important to align an appropriate method to the study question and data.
To guide researchers, the research team conducted a systematic review of the literature, identifying important considerations in selecting and applying statistical methods for causal inference with observational data. In addition, the team conducted simulation studies to investigate the statistical properties of propensity-score-based approaches. Results from the systematic review and simulations guided the development of a decision guide to aid researchers in choosing appropriate methods for CER.
In the systematic review, the research team identified studies of causal inference methods specific to the use of observational data in CER. The review focused on studies with a single binary treatment and a single binary outcome. The team searched the PubMed, EMASE, PsycINFO, and Current Index to Statistics databases for simulation studies or theoretical findings that assessed control of bias and precision in analyzing observational data. Simulation studies compared five different propensity-score-based methods.
Researchers with expertise in statistics, social epidemiology, health policy, outcomes effectiveness research, pharmacy, and physical therapy provided input on literature search strategies, analytic issues, and the decision guide.
Data Source View help for Data Source
PubMed, EMASE, PsycINFO, and Current Index to Statistics
Notes
The public-use data files in this collection are available for access by the general public. Access does not require affiliation with an ICPSR member institution.
ICPSR usually offers files in multiple formats for researchers to be able to access data and documentation in formats that work well within their needs. If you have questions about the accessibility of materials distributed by ICPSR or require further assistance, please visit ICPSR’s Accessibility Center.
This study is maintained and distributed by the Patient-Centered Outcomes Data Repository (PCODR). PCODR is the official data repository of the Patient-Centered Outcomes Research Initiative (PCORI).