EdWeek reported this week some news that will give education evaluation system and instrument designers a sigh of relief. Researchers studying teacher effectiveness data in New York City have concluded that subjective and objective measures of teacher effectiveness show equal levels of validity in the evaluation process. In their abstract the researchers noted, “We find that these subjective evaluations have substantial power, comparable with and complementary to objective measures of teachereffectiveness taken from a teacher’s first year in the classroom.” I’ve quoted sections of the paper‘s conclusion, emphasis added by me, and included my comments.
We find evidence that teachers who receive better subjective evaluations of teaching ability prior to hire or in their first year of teaching also produce greater gains in achievement, on average, with their future students. Consistent with prior research, our results support the idea that teachers who produce greater achievement gains in the first year of their careers also produce greater gains, on average, in future years with different students. More importantly, subjective evaluations present significant and meaningful information about a teacher’s future success in raising student achievement even conditional on objective data on first year performance. This is an especially noteworthy finding, considering that variation in subjective evaluations likely also captures facets of teaching skill that may affect outcomes not captured by standardized tests.
This is extremely important because the predictive ability of an instrument is what ultimately determines if it is successful. What’s more, did the additional rigor of the instrument produce additional relevant data? In this case, the research is showing that it does. Even more, the ability to provide pre-hire, and potentially pre-service, feedback is critical. The next measure of success for these systems is do they contribute to creating a better pool of teaching talent? Having pre-service and pre-hire impact is a great step forward.
Knowledge regarding the power of subjective evaluations and objective performance data has important implications for designing teacher evaluation systems, merit pay, and other polices whose goal is improving teacher quality and student achievement. All schooldistricts evaluate teachers, but evaluation policies are not typically based in high quality empirical research and in many cases produce little differentiation among teachers.
This is exactly the tone of the discussion in Tennessee. In other words, it’s tough, but we can’t have all 5-star teachers when our schools are failing. Something’s got to give and the “we’ve always done it this way” crowd is going to lose like it does in every industry that faces disruption.
Given the current era of increased accountability for schools and the research demonstrating the importance of teacher quality, it is likely that states and school districts will begin to implement policies that put greater stress on teacher effectiveness.
I think this study is going to have big implications in these discussions. I also think it will clear the air for teacher evaluation system and instrument designers because now they can point to peer-reviewed research and say, “Look, it all matters.” That’s going to take the focus off the worries about how much each variable counts, allowing for more rigorous review inside a system that no one side can say is bias or not. It’s now a lot easier to point at data and say, “The bar has been raised.”
