How Different are Different Differences-in-Differences? A Large-Scale Meta-Analysis
中文摘要
本文运用人工智能辅助的复制分析，评估双向固定效应（TWFE）估计中负权重的实际重要性，分析了来自 116 篇经济学论文的 1,430 个交错处理设定。TWFE 估计与现代双重差分估计高度相关，在 12.0%–14.9% 的比较中出现符号反转，且 TWFE 的精度更高。考虑估计噪声的层次模型意味着估计目标的相关性为 91%–97%，且统计显著的 TWFE 估计与现代估计目标符号相反的概率为 0.7%–2.4%。这些发现表明，负权重在实践中可能并非严重问题，并展示了人工智能如何支持对新计量经济学方法进行系统性、大规模的经验评估。
Abstract
I use AI-assisted replication to assess the empirical importance of negative weights in two-way fixed effects (TWFE) estimation, analyzing 1,430 staggered-adoption specifications from 116 economics papers. TWFE estimates are highly correlated with modern difference-in-differences estimates, with sign reversals in 12.0–14.9% of comparisons and greater precision for TWFE. Hierarchical models accounting for estimation noise imply estimand correlations of 91–97% and a 0.7–2.4% probability of a statistically significant TWFE estimate opposing the modern estimand’s sign. These findings suggest negative weights may be a less serious problem in practice and illustrate how AI can support systematic, large-scale empirical assessments of new econometric methods.