Improving Administrative Data at Scale: Experimental Evidence on Digital Testing in Indian Schools
中文摘要
大规模学生评估是中低收入国家拟议教育改革的基石, 旨在提升处于极低水平的学生成绩。然而, 这一愿景依赖于其假定的可靠性。我利用来自印度一个大邦 (安得拉邦) 的直接审计证据表明, 在目前的实施方式下, 官方学习评估大幅高估学业熟练程度, 并低估了学生成绩低下这一 "学习危机"。在一项涵盖 2,400 余所学校的实验中, 我评估基于平板电脑的数字化测试能否减少扭曲。尽管由教师监考的纸笔测试严重夸大成绩, 但基于平板电脑的测试与独立测试数据高度吻合, 且被标记为作弊的可能性低得多。此外, 我利用基于直接审计的重测, 直接验证了现有 (间接) 统计程序在大规模检测作弊方面的有效性, 并证实利用此类方法以低成本、大规模地监测数据完整性是可行的。总体而言, 这些结果表明, 设计良好的技术辅助干预或可大规模改善数据完整性; 若缺乏数据完整性, 这些学习评估不太可能成为政策行动的催化剂。
Abstract
Abstract Large-scale student assessments are a cornerstone of proposed educational reforms to improve student achievement from very low levels in low- and middle-income countries. Yet, this promise relies on their presumed reliability. I use direct audit evidence from a large Indian state (Andhra Pradesh) to show that, as currently administered, official learning assessments substantially overstate proficiency and understate the ‘learning crisis’ of low student achievement. In an experiment covering over 2,400 schools, I evaluate whether digital tablet-based testing could reduce distortion. Although paper-based assessments proctored by teachers severely exaggerate achievement, tablet-based assessments closely match independent test data and are much less likely to be flagged for cheating. Furthermore, I use the direct audit-based retest to directly validate existing (indirect) statistical procedures for detecting cheating at scale and establish that it would be feasible to monitor data integrity cheaply and at scale with such methods. Overall, these results suggest that well-designed technology-aided interventions may improve data integrity at scale, without which these learning assessments are unlikely to serve as a catalyst for policy action.