Comparing Experimental and Nonexperimental Methods: What Lessons Have We Learned Four Decades after LaLonde (1986)?
Journal of Economic Perspectives · 2025 · Guido W. Imbens、Yiqing Xu
中文摘要
1986年，Robert LaLonde发表了一篇比较非实验估计与实验基准的文章（LaLonde 1986）。他得出结论：当时的非实验方法无法系统性地复现实验基准，这使其可信度受到质疑。在LaLonde的批判性评估之后，方法论取得了重大进展，实践也发生了变化，包括：（1）强调无混杂性假设，并将其与函数形式的考量区分开来；（2）关注协变量分布重叠的重要性；（3）引入基于倾向得分的方法，进而发展出双重稳健估计量；（4）发展用于估计和利用处理效应异质性的方法；（5）更加重视验证性分析，以增强研究可信度。为展示这些进展带来的实践启示，我们重新考察了LaLonde的数据。我们表明，在协变量充分重叠的情境中应用现代方法，能够对处理组与对照组之间调整后的差异得到稳健估计。然而，这并不意味着这些估计具有因果可解释性。为评估其可信度，验证性分析（如安慰剂检验）必不可少，而仅靠拟合优度检验则不够。我们的发现凸显了在利用非实验数据分析因果效应时，仔细考察处理分配过程、审慎检查重叠情况以及开展验证性分析的重要性。
Abstract
In 1986, Robert LaLonde published an article comparing nonexperimental estimates to experimental benchmarks (LaLonde 1986). He concluded that the nonexperimental methods at the time could not systematically replicate experimental benchmarks, casting doubt on their credibility. Following LaLonde's critical assessment, there have been significant methodological advances and practical changes, including (1) an emphasis on the unconfoundedness assumption separated from functional form considerations, (2) a focus on the importance of overlap in covariate distributions, (3) the introduction of propensity score-based methods leading to doubly robust estimators, (4) methods for estimating and exploiting treatment effect heterogeneity, and (5) a greater emphasis on validation exercises to bolster research credibility. To demonstrate the practical lessons from these advances, we reexamine the LaLonde data. We show that modern methods, when applied in contexts with sufficient covariate overlap, yield robust estimates for the adjusted differences between the treatment and control groups. However, this does not imply that these estimates are causally interpretable. To assess their credibility, validation exercises (such as placebo tests) are essential, whereas goodness-of-fit tests alone are inadequate. Our findings highlight the importance of closely examining the assignment process, carefully inspecting overlap, and conducting validation exercises when analyzing causal effects with nonexperimental data.