Methods for Comparative Effectiveness and Safety Analyses in a High-Dimensional Covariate Space with Few Events [Methods Study], 2013-2017 (ICPSR 39486)

Version Date: Sep 4, 2025

Cite this study |

Principal Investigator(s):
Jessica M. Franklin, Brigham and Women's Hospital

https://doi.org/10.3886/ICPSR39486.v1

Version V1

Slide tabs to view more

Summary

Comparative effectiveness research compares two or more treatments to see which one works best for which patients. Information from health insurance claims could be useful for this type of research. These claims include data on how well patients respond to treatments. But many things--not just treatments--affect whether patients' health improves.

How well patients respond to treatments could depend on patients' ages or medicines they take. It could also depend on how many health problems a patient has and how severe the problems are. Also, a doctor may suggest one treatment instead of another because of a patient's personal situation and health. Researchers need ways to figure out whether changes in patients' health result from treatment or something else.

Comparing treatments is hard in small studies with only a few patients. When there are few patients in a study, researchers can study only a few events. An event is an outcome related to the health problem or treatment researchers are studying. When there are few events and many things that could affect treatment results, it is hard to figure out what causes changes in patients' health. To address this problem, researchers use different statistical methods to account for all the things that could affect treatment results. But researchers don't know which methods might work best in studies with few events. In this study, the research team compared several methods to see which ones worked best.

Citation

Franklin, Jessica M. Methods for Comparative Effectiveness and Safety Analyses in a High-Dimensional Covariate Space with Few Events [Methods Study], 2013-2017. Inter-university Consortium for Political and Social Research [distributor], 2025-09-04. https://doi.org/10.3886/ICPSR39486.v1

Export Citation:

  • RIS (generic format for RefWorks, EndNote, etc.)
  • EndNote

Funding

Patient-Centered Outcomes Research Institute (PCORI) (ME-1303-5796)

Subject Terms

Geographic Coverage

Distributor(s)

Inter-university Consortium for Political and Social Research
Hide

Time Period(s)

2013 -- 2017
Hide

Study Purpose

To evaluate and improve analytic approaches for variable selection and treatment-effect estimation in nonrandomized studies with few outcome events and many confounders

Study Design

Nonrandomized studies are essential in early identification of promising new treatments, rare diseases, and comparing treatment effects in population subgroups often excluded from randomized trials (e.g., children and older adults). Nonrandomized studies may have few outcome events and numerous confounding variables (i.e., variables associated with both treatment and outcomes). Such nonrandomized studies present significant challenges to drawing causal inferences.

Researchers often use propensity-score (PS) models to control for many measured confounders in estimating the causal effects of treatment. Applying PS models involves two steps. Researchers identify a set of variables or confounders to calculate the PS for each patient. Then, researchers use the PS to estimate treatment effects. Researchers typically apply PS methods in analyzing data with many confounders, but PS methods can be unstable when there are few outcome events. Few studies have explored which PS approaches offer the greatest control for confounding in such scenarios.

Researchers conducted two simulation studies evaluating PS models that have been proposed in the literature. Researchers based the simulations on three previously published cohort datasets using the plasmode framework. The plasmode framework creates realistic simulated datasets that mimic traits found in real nonrandomized cohort studies based on large healthcare datasets.

One simulation compared the high-dimensional propensity score (hdPS) algorithm with regularized regression approaches, such as ridge regression and lasso regression. The hdPS algorithm prioritizes a subset of potential confounders to include in the PS model. However, regularized regression approaches adjust for all potential confounders when modeling the outcomes.

The other simulation compared a variety of PS-based estimators of the treatment effect across different conditions. These conditions included whether treatment effects were heterogeneous, which means a treatment's effect differed for different patients, or homogeneous, which means a treatment's effect was the same across patients.

Researchers used bias and mean squared error of the estimated effects to assess performance.

Patient representatives provided input during the study about issues related to nonrandomized research that were important to them.

Data Source

Simulations based on 3 previously published pharmacoepidemiologic cohorts

Hide

Original Release Date

2025-09-04

Hide

Notes

  • The public-use data files in this collection are available for access by the general public. Access does not require affiliation with an ICPSR member institution.

  • ICPSR usually offers files in multiple formats for researchers to be able to access data and documentation in formats that work well within their needs. If you have questions about the accessibility of materials distributed by ICPSR or require further assistance, please visit ICPSR’s Accessibility Center.

pcodr logo

This study is maintained and distributed by the Patient-Centered Outcomes Data Repository (PCODR). PCODR is the official data repository of the Patient-Centered Outcomes Research Initiative (PCORI).