The Role of Measurement in Defining and Continuously Improving Quality Professional Development in Developmental Mathematics

Authors: Carlos Sandoval, Haley McNamara, Ann R. Edwards
Presented at 2017 AERA

​Abstract: For many instructors, Pathways teaching is a radical and challenging change. In contrast to traditional community college mathematics instruction, Pathways pedagogy is grounded in research on teaching for powerful mathematical understanding including practices supporting productive struggle with substantive mathematics, rich mathematical discourse for sense-making, making conceptual connections explicit, and collaborative learning (Bransford, Brown, & Cocking, 1999; Hiebert & Grouws, 2007). In addition, the Pathways integrate interventions from social psychology designed to promote students’ engagement and persistence into mathematics curriculum and pedagogy (Dweck, Walton & Cohen, 2014; Silver & White, 2013). To prepare and support new Pathways faculty the Faculty Support Program (FSP) was developed in 2013, based upon research on effective professional development (Hunzicker, 2010; Guskey, 2002; Garet, et al, 2001; LeMahieu, Roy & Foss, 1995). The FSP design is also guided by principles of improvement science (Bryk et al, 2015), enabling the development of responsive professional development that is continuously improving - i.e., is continuously sensitive to varying and changing conditions, with mechanisms for identifying, designing and testing ideas for improvement, as well as for capturing and sharing learning across the network. This paper reports on these processes of continuous improvement, highlighting the role of measurement in (1) understanding the performance of the FSP, (2) providing direction for improvement efforts, and (3) determining whether changes made to the program were improvements. To accomplish these goals, the FSP team employs a measurement system modeled after metrics of quality health care (Martin et al, 2007). This system consists of system-level measures organized around five quality dimensions (effective, efficient, responsive, community-oriented, and faculty-centered and faculty-owned). Developed after 32 interviews with Pathways faculty and administrators and revised during the first year of program implementation, the measurement system provides stakeholders with a system-level view of FSP performance. Data include, for example, new instructors’ confidence in their skills related to teaching the Pathways and comfort teaching Pathways courses at several time points during their first year, as well as measures of their students’ sense of belonging and mathematical mindsets. The FSP Improvement Team (consisting of a researcher, improvement science facilitators, and Pathways faculty) and the Faculty Mentors (experienced faculty who coach new faculty) meet three times per year to review these data to determine performance of the FSP along the quality dimensions. During the October review, participants examine these data to determine improvement priorities and launch efforts that employ methods such as PDSA cycles to design, test and refine program improvements. For example, in 2015-16 the team undertook several learning and testing cycles focused on improving new instructors’ use of student data because most instructors surveyed did not feel confident in using data to inform or improve their Pathways teaching. At subsequent reviews, the team reviews updated data to determine whether changes made to the program resulted in improvements. This work not only contributes to understandings of the design of effective professional development but, importantly, how measurement -- designed practically and focused on systematic improvement -- can be employed to learn about and improve teacher learning.