Identity Crisis: Multiple Measures and the Identification of Schools under ESSA

Authors: Heather Hough, Emily Penner, Joe Witte
Presented at 2017 AERA

​Abstract: The Every Student Succeeds Act (ESSA) requires a more comprehensive approach to measurement than was required under No Child Left Behind (NCLB), with the intention of including more measures and moving away from adverse consequences of NCLB’s measurement system, namely the narrowing of the curriculum towards tested subjects and content, strategic gaming of accountability structures, and cheating (Figlio & Getzler, 2002; Jacob & Levitt, 2003; Lauen & Gaddis, 2015; Neal & Schanzenbach, 2010). Specifically, ESSA requires states to include multiple measures of student academic achievement, including academic performance as measured by proficiency on English Language Arts (ELA) and math tests, academic growth, graduation rate, development of English Learner (EL) proficiency, and an additional indicator of “School Quality or Student Success” (SQSS), which can include measures of student engagement, educator engagement, student access to and completion of advanced coursework, post-secondary readiness, or school climate and safety. The CORE Districts have already operationalized a number of these SQSS indicators, including chronic absenteeism, suspension/expulsion, and student reports of social-emotional skills and school culture/climate. We use the CORE data, which includes survey responses for more than 250,000 students in grades 5-12, student-level administrative data with demographics, attendance, discipline and test scores; and the results of CORE’s growth model aggregated into measures at the school level. Using this school-level dataset representing over one million students and 923 Title I schools, we illustrate how these measures are related to one another and to the academic measures, and to explore how they could be used in the context of the larger measurement system to identify schools for comprehensive and targeted support and improvement. We find that: 1) In the identification of schools for comprehensive support and improvement, a summative score alone prevents schools with very low-performance on key indicators from being identified for comprehensive support and improvement, and can get in the way of states prioritizing the indicators that stakeholders value most; 2) as the legislation and regulations currently stand, the SQSS indicator will have no weight in state systems, effectively removing them from the accountability system, since it has been shown that measures reported without consequences will not receive the same attention as measures that do (Jacob, 2005); and 3) that ESSA’s guidelines for identifying schools for targeted support for subgroup performance would identify the majority of schools, suggesting that a more conservative approach is needed. The passage of ESSA marks a sea change in education policy, and policymakers are now tasked with decisions that will affect our country’s children for decades to come. In this paper, we utilize CORE’s data to explore the tradeoffs in the identification of schools under different methods as a policy analysis case study. Similar to prior accountability research (Chester, 2005; Linn, Baker, & Betebenner, 2002; McEachin & Polikoff, 2012), our analysis, which illustrates the empirical effect of various policy approaches, can support intelligent policymaking at both the state and federal level as regulations and state-wide implementation plans are developed through the 2016-17 school year and beyond.