Myths of official measurement: Limits to test-based education reforms with weak governance
中文摘要
评估驱动的学校改革是旨在解决发展中国家学生学业成绩偏低问题的政策组合的核心支柱。我们利用直接审计证据，评估印度大邦中央邦一项改革中官方评估结果的真实性；该改革自2011年以来每年对超过600万名学生进行测试。比较同一批学生对相同测试题目的作答结果发现，行政数据所报告的成绩是独立测试结果的两倍。在校内比较中，采用多套试卷和外部评分的年级中，这一差异较小。总体而言，在治理薄弱的情境下，依赖考试问责的干预措施若缺少保障数据完整性的配套投入，似乎不太可能取得成功。• 标准化测试是发展中国家许多教育改革的基础。• 直接审计表明，印度的官方测试大幅高估了学生的学习水平。• 官方测试与独立测试结果之间的差异反映了人为操纵。• 考试驱动型教育改革需要对数据完整性进行配套投入。
Abstract
Assessment-led school reforms are a central pillar of policy packages recommended to address low student achievement in developing countries. We use direct audit evidence to assess the truthfulness of official assessments in a reform that has tested over 6 million students annually since 2011 in the large Indian state of Madhya Pradesh. Comparing responses to the same test questions by the same students shows a doubling of reported achievement in administrative data versus independent tests. This difference is lower, within schools, in grades with multiple test booklets and external grading. Overall, in contexts with weak governance, interventions relying on test-based accountability appear unlikely to succeed without complementary investments to assure data integrity. • Standardized tests underlie many education reforms in developing countries. • Direct audits show that official tests substantially overstate learning in India. • Discrepancy between official and independent tests results reflects manipulation. • Test-based education reforms require complementary investments in data integrity.