Simulated power analyses for observational studies: An application to the Affordable Care Act Medicaid expansion
Journal of Public Economics · 2022 · Bernard S. Black、Alex J. Hollingsworth、Letícia Nunes、Kosali Ilayperuma Simon
中文摘要
统计功效是评估统计估计可能有效性的重要因素。一项低功效的分析即使存在处理效应，也不太可能产生令人信服的证据。更令人担忧的是，来自低功效分析的统计显著估计很可能会误述真实效应量，包括得出符号错误的估计值或大出数倍的估计值。然而，已发表的经济学研究中很少报告统计功效。部分原因在于许多现代研究设计足够复杂，以至于无法用简单公式轻易确定功效。在观察性情境中，功效也可能难以估计。我们以获得医疗保险与死亡率之间的联系作为应用实例，进行模拟功效分析，以概述功效的重要性以及在复杂研究情境中估计功效的方法。我们发现，使用县级或州级死亡率数据对医疗补助扩展进行的标准双重差分和三重差分分析，需要至少使总体死亡率降低2%，才能具备良好功效。尽管进行模拟功效分析没有唯一正确的方法，本文概述了应用研究者如何开展适合其研究情境的模拟。
Abstract
Power is an important factor in assessing the likely validity of a statistical estimate. An analysis with low power is unlikely to produce convincing evidence of a treatment effect even when one exists. Of greater concern, a statistically significant estimate from a low-powered analysis is likely to misstate the true effect size, including finding estimates of the wrong sign or that are several times too large. Yet statistical power is rarely reported in published economics work. This is in part because many modern research designs are complex enough that power cannot be easily ascertained using simple formulae. Power can also be difficult to estimate in observational settings. Using an applied example–the link between gaining health insurance and mortality–we conduct a simulated power analysis to outline the importance of power and ways to estimate power in complex research settings. We find that standard difference-in-differences and triple differences analyses of Medicaid expansions using county or state mortality data would need to induce reductions in population mortality of at least 2% to be well powered. While there is no single, correct method for conducting a simulated power analysis, our manuscript outlines how applied researchers can conduct simulations appropriate to their settings.