Abstract
Despite the fact that randomization is the gold standard for estimating causal relationships, many questions in prevention science are often left to be answered through nonexperimental studies because randomization is either infeasible or unethical. While methods such as propensity score matching can adjust for observed confounding, unobserved confounding is the Achilles heel of most nonexperimental studies. This paper describes and illustrates seven sensitivity analysis techniques that assess the sensitivity of study results to an unobserved confounder. These methods were categorized into two groups to reflect differences in their conceptualization of sensitivity analysis, as well as their targets of interest. As a motivating example, we examine the sensitivity of the association between maternal suicide and offspring’s risk for suicide attempt hospitalization. While inferences differed slightly depending on the type of sensitivity analysis conducted, overall, the association between maternal suicide and offspring’s hospitalization for suicide attempt was found to be relatively robust to an unobserved confounder. The ease of implementation and the insight these analyses provide underscores sensitivity analysis techniques as an important tool for nonexperimental studies. The implementation of sensitivity analysis can help increase confidence in results from nonexperimental studies and better inform prevention researchers and policy makers regarding potential intervention targets.
Similar content being viewed by others
Notes
The original study estimated hazard ratio of 1.80 with a 95 % confidence interval of 1.19, 2.74.
Note that this is not a loss of generality; if the unobserved confounder is negatively associated with exposure status, we could simply redefine the unobserved confounder to meet this scenario.
References
Arah, O. A., Chiba, Y., & Greeland, S. (2008). Bias formulas for external adjustment and sensitivity analysis of unmeasured confounders. Annals of Epidemiology, 18, 637–646. doi:10.1016/j.annepidem.2008.04.003.
Brent, D. A., & Mann, J. J. (2005). Family genetic studies, suicide, and suicidal behavior. American Journal of Medical Genetics, 133, 13–24. doi:10.1002/ajmg.c.30042.
Cornfield, J., Haenszel, W., Hammon, E., Lilienfeld, A., Shimkin, M., & Wynder, E. (1959). Smoking and lung cancer: Recent evidence and a discussion of some questions. Journal of the National Cancer Institute, 22, 173–203. Available at http://ije.oxfordjournals.org/content/38/5/1175.full.
DiPrete, T. A., & Gangl, M. (2004). Assessing bias in the estimation of causal effects: Rosenbaum bounds on matching estimators and instrumental variables estimation with imperfect instruments. Sociological Methodology, 34, 271–310. doi:10.1111/j.0081-1750.2004.00154.x.
Flay, B., Biglan, A., Boruch, R., Castro, F., Gottfredson, D., Kellam, S., et al. (2005). Standards of evidence: Criteria for efficacy, effectiveness and dissemination. Prevention Science, 6, 151–175. doi:10.1007/s11121-005-5553-y.
Gangl, M. (2004). Rbounds: Stata module to perform Rosenbaum sensitivity analysis for average treatment effects on the treated. Available at http://EconPapers.repec.org/RePEc:boc:bocode:s438301.
Gastwirth, J., Krieger, A., & Rosenbaum, P. (1998). Dual and simultaneous sensitivity analysis for matched pairs. Biometrika, 85, 907–920. doi:10.1093/biomet/85.4.907.
Greenland, S. (1996). Basic methods for sensitivity analysis of biases. International Journal of Epidemiology, 25, 1107–1116. doi:10.1093/ije/25.6.1107.
Harding, D. (2003). Counterfactual models of neighborhood effects: The effect of neighborhood poverty on dropping out and teenage pregnancy. The American Journal of Sociology, 109, 676–719. doi:10.1086/379217.
Harding, D. J. (2009). Collateral consequences of violence in disadvantaged neighborhoods. Social Forces, 88, 757–784. doi:10.1353/sof.0.0281.
Haviland, A., Nagin, D., & Rosenbaum, P. (2007). Combining propensity score matching and group-based trajectory analysis in an observational study. Psychological Methods, 12, 247–267. doi:10.1037/1082-989X.12.3.247.
Jo, B., & Stuart, E. A. (2009). On the use of propensity scores in principal causal effect estimation. Statistics in Medicine, 28, 2857–2875. PMCID PMC 2757143.
Keele, L. (2010). Rbounds: An R package for sensitivity analysis with matched data. R package. Available at http://www.polisci.ohio-state.edu/faculty/lkeele/rbounds.html.
Kellam, S. G., Brown, C. H., Poduska, J. M., Ialongo, N. S., Wang, W., Toyinbo, P., et al. (2008). Effects of a universal classroom behavior management program in first and second grades on young adult behavioral, psychiatric, and social outcomes. Drug and Alcohol Dependence, 95, S5–S28. doi:10.1016/j.drugalcdep.2008.01.004.
Kitahata, M. M., Gange, S. J., Abraham, A. G., Merriman, B., Saag, M. S., Justice, A. C., et al. (2009). Effect of early versus deferred antiretroviral therapy for HIV on survival. The New England Journal of Medicine, 360, 1815–1826. Available at http://www.nejm.org/doi/full/10.1056/NEJMoa0807252#t=articleTop.
Kuramoto, S. J., Stuart, E. A., Runeson, B., Lichtenstein, P., Langstrom, N., & Wilcox, H. C. (2010). Maternal or paternal suicide and offspring’s psychiatric and suicide-attempt hospitalization risk. Pediatrics, 126, e1026–e1032. doi:10.1542/peds.2010-0974.
Lieb, R., Bronisch, T., Hofler, M., Schreier, A., & Wittchen, H.-U. (2005). Maternal suicidality and risk of suicidality in offspring: Findings from a community study. The American Journal of Psychiatry, 162, 1665–1671. doi:10.1176/appi.ajp.162.9.1665.
Lin, D. Y., Psaty, B. M., & Kronmal, R. A. (1998). Assessing the sensitivity of regression results to unmeasured confounders in observational studies. Biometrics, 54, 948–963. Available at http://www.jstor.org/pss/2533848.
Liu, W. (2012). The adult offending and school dropout nexus: A life course analysis. El Paso: LFB Scholarly Publishing LLC (in press).
Love, T. (2008). Spreadsheet-based sensitivity analysis calculations for matched samples. Center for Health Care Research & Policy, Case Western Reserve University, Cleveland. Available at http://www.chrp.org/propensity/.
Luiz, R., & Cabral, M. (2010). Sensitivity analysis for an unmeasured confounder: A review of two independent methods. Revista Brasileira de Epidemiologia, 13, 188–198. Available at http://www.scielosp.org/pdf/rbepid/v13n2/02.pdf.
Manski, C. F., Sandefur, G. D., McLanahan, S., & Power, D. (1992). Alternative estimates of the effect of family structure during adolescence on high school graduation. Journal of the American Statistical Association, 87, 25–37. Available at http://www.jstor.org/pss/2290448.
McCaffrey, D. F., Ridgeway, G., & Morral, A. R. (2004). Propensity score estimation with boosted regression for evaluating causal effects in observational studies. Psychological Methods, 9, 403–425. doi:10.1037/1082-989X.9.4.403.
McCandless, L. C., Gustafson, P., & Levy, A. (2007). Bayesian sensitivity analysis for unmeasured confounding in observational studies. Statistics in Medicine, 26, 2331–2347. doi:10.1002/sim.
McNemar, Q. (1947). Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika, 12, 153–157. doi:10.1007/BF02295996.
Niederkrotenthaler, T., Floderus, B., Alexanderson, K., Rasmussen, F., & Mittendorfer-Rutz, E. (2012). Exposure to parental mortality and markers of morbidity, and the risks of attempted and completed suicide in offspring: An analysis of sensitive life periods. Journal of Epidemiology and Community Health, 66, 233–239. doi:10.1136/jech.2010.109595.
Ridgeway, G. (2006). Assessing the effect of race bias in post-traffic stop outcomes using propensity scores. Journal of Quantitative Criminology, 22, 1–29. doi:10.1007/s10940-005-9000-9.
Rosenbaum, P. R. (2002). Observational studies (2nd ed.). New York: Springer.
Rosenbaum, P. R. (2010). Design of observational studies. New York: Springer.
Rothman, K. J., Greenland, S., & Lash, T. L. (2008). Modern epidemiology. Philadelphia: Lippincott Williams & Wilkins.
Schneeweiss, S. (2006). Sensitivity analysis and external adjustment for unmeasured confounders in epidemiologic database studies of therapeutics. Pharmacoepidemiology and Drug Safety, 15, 291–303. doi:10.1002/pds.1200.
Steenland, K., & Greenland, S. (2004). Monte Carlo sensitivity analysis and Bayesian analysis of smoking as an unmeasured confounder in a study of silica and lung cancer. American Journal of Epidemiology, 160, 384–392. doi:10.1093/aje/kwh211.
Stuart, E. A. (2010). Matching methods for causal inference: A review and a look forward. Statistical Science, 25, 1–21. doi:10.1214/09-STS313.
Stuart, E. A., & Green, K. M. (2008). Using full matching to estimate causal effects in nonexperimental studies: Examining the relationship between adolescent marijuana use and adult outcomes. Developmental Psychology, 44, 395–406.
VanderWeele, T. J., & Arah, O. A. (2011). Bias formulas for sensitivity analysis of unmeasured confounding for general outcomes, treatments, and confounders. Epidemiology, 22, 42–52. doi:10.1097/EDE.0b013e3181f74493.
Wilcox, H. C., Kuramoto, S. J., Lichtenstein, P., Långström, N., Brent, D. A., & Runeson, B. (2010). Psychiatric morbidity, violent crime and suicide among children and adolescents exposed to parental death. Journal of the American Academy of Child and Adolescent Psychiatry, 49, 514–523. doi:10.1016/j.jaac.2010.01.020.
Acknowledgments
The authors wish to acknowledge the NARSAD Young Investigator Award to Dr. Holly C. Wilcox, and Dr. Holly C. Wilcox for allowing us to use the motivating example. We also thank the National Institute of Drug Abuse for the training support for S. Janet Kuramoto (1F31DA0263182), the National Institute of Mental Health (NIMH) Prevention Research T32 Training Grant for the training support for Weiwei Liu (T32 MH18834), and NIMH for the support of Elizabeth Stuart’s time (K25 MH083846). This work was performed while Weiwei Liu was a postdoctoral fellow and S. Janet Kuramoto was a student at Johns Hopkins Bloomberg School of Public Health.
Author information
Authors and Affiliations
Corresponding author
Electronic Supplementary Material
Below is the link to the electronic supplementary material.
ESM 1
(DOCX 72 kb)
Rights and permissions
About this article
Cite this article
Liu, W., Kuramoto, S.J. & Stuart, E.A. An Introduction to Sensitivity Analysis for Unobserved Confounding in Nonexperimental Prevention Research. Prev Sci 14, 570–580 (2013). https://doi.org/10.1007/s11121-012-0339-5
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11121-012-0339-5