|
AEFP Conference
Kansas City, Missouri March 21-23, 2019 Title: First Impressions: Can Initial District Screenings of Applicants Predict Student and Teacher Outcomes on the Job? Session: Preservice Teacher Placements and Coordinating Teacher Quality Authors: Xuehan Zhou, Emily Penner, Sabrina Solanki Abstract Understanding how to identify effective teachers has remained in the forefront of education research for the past decade. We know from the body of literature that cognitive ability predicts teacher quality; also, personality and values have been shown to be positively related to teacher evalution scores and student achievement, although to a smaller degree. One issue with these teacher attributes is that they are not easily-observed during the screening process, making it hard for school districts to know who to hire. Recent evidence suggests, however, that school districts can create more effective screening procedures and thus identify effective teachers at the point of hire. In one example, Goldhaber, Grout, and Huntington-Klein (2017) examine the relationship between two screening instruments and teacher outcomes. Ratings on the screening instruments significantly predicted value added in math and teacher attrition, but not absences. Bruno & Strunk (2018) estimate the impact of a new complex screening system adopted by a large urban public school district on teacher and student outcomes. Bruno & Strunk (2018) prelimary results show that new teachers who receive higher screening scores have substantially lower probablities of receiving unsatisfactory evaluations, take fewer discretionary absences, and their students experience larger test score gains in math and English/language arts (ELA). These elaborate screening procedures however might be cost prohibitive for certain districts. We aim to contribute to extant literature concerning the teacher hiring process by testing whether more basic screening procedures—specifically, applicants written response to essay prompts—predict being hired. And, conditional on being hired, we test whether screening procedures predict three distinct teacher outcomes: 1) teacher absence behavior, 2) teacher retention, and 3) value-added measures of effectiveness. The data used in the proposed study come from an urban public school district in California that employs over 3,500 educators and administrators to serve a diverse student body of over 50,000 students. We use applicant data from this district’s Human Resources Department. As applicants for positions in the district apply through a proprietary online interface, we use data from this interface in concert with administrative records for staff and students. Our data includes all applicants for certificated positions from March of 2009 to October of 2015. During this window, the district received 164,367 complete applications (10,188 distinct individuals). The district ultimately hired 2,883 individuals from this pool. During the application process, applicants were asked to respond to three separate essay prompts that each tested a specific teaching competency. The prompt for essay 1, “Assessment,” asked teachers to describe their role in closing the achievement gap in the school district, and to include experiences or skills that made them well-positioned to close the achievement gap in their response. The prompt for essay 2, “Ability,” asked teachers to describe the goals they established for students and how they monitor student progress. Lastly, the prompt for essay 3, “Strategies,” asked teachers to describe strategies that they use to address disruptive students. Each essay was scored by staff in the Human Resources Department using a rubric, awarding zero to three points on the basis of no evidence, mixed/limited evidence, satisfactory evidence, or strong evidence of each competency addressed in the prompt. We use the scores for each of the three application essays as main predictor variables in our regression analyses. We also include models where an aggregated score is used. Our main analysis tests whether these scores predict teacher outcomes. Each regression model includes a rich set of applicant control variables, in addition to controls for writing quality. We focus our analysis on the following outcome measures: (1) hired status; (2) teacher absenteeism, which involves average number of absences due to illness and a separate measure of chronic absenteeism; (3) teacher retention, which addresses whether a teacher leaves the school district and duration of employment; and (4) teacher effectiveness, as measured by student gain scores and teacher value-added scores in math and English/language Arts (ELA). Preliminary results suggest that higher essay scores predict hired status. Essay scores are also negatively related to teacher absenteeism and retention. Applicants with higher essay scores are 2.5 percentage points less likely to leave a school district than applicants with lower essay scores. In regard to teacher effectiveness, we do not find a consistent pattern between essay scores and teacher value added scores. For certain outcomes, essay scores—specifically, the “Assessment” score—are negatively related to elementary value added scores in ELA. The full set of results, in addition to specific policy implications, will be discussed during the AEFP presentation. Comments are closed.
|
|
