“As President Obama has emphasized, the single most important factor determining whether students succeed in school is not the color of their skin or their ZIP code or even their parents’ income — it is the quality of their teacher,” declared a “How to Fix Our Schools” manifesto signed by 16 school superintendents, published in 2010 in the Washington Post.
Then and now, education policy debates increasingly focus on teacher quality. How can governments encourage existing teachers to work harder, train them to work more effectively, or change the composition of the teacher workforce so that talented educators enter and stay in the profession, while weak teachers leave? Two popular policies to achieve these goals are merit pay and teacher evaluation systems that clear the way for firing consistently poor performers.
On April 26, capping NYU Wagner’s Spring 2012 Doctoral Research Colloquium Speaker Series, UC-Berkeley Associate Professor Jesse Rothstein examined the efficacy of those two policy levers in a presentation of his working paper, “Teacher Quality Policy When Supply Matters.” Using the most reliable empirical data available, the paper builds a simulation model to estimate the effects of pay-for-performance and performance-based retention on teacher effectiveness.
Rothstein, former chief economist for the U.S. Department of Labor, is a cogent critic of the use of “value-added measures” in teacher evaluation. “VAMs” apply sophisticated statistical methods to student standardized test scores to isolate teacher impact on student learning. In an earlier paper, he showed that the same methodology used to measure teacher performance in North Carolina could also prove that 5th grade teachers had large “effects” on 4th grade student achievement. Failing this type of falsification test is a damning indictment of any statistical model that tries to prove causal effects. Yet VAMs remain the dominant approach to measuring teacher performance.
In his current research, Rothstein sets aside his skepticism of VAMs to “cook the model” in favor of these policies. “I’m making the best-case argument for merit pay and performance retention, assuming that we have an unbiased, uncorrupted performance measure,” he said at the outset of his presentation.
What he found was that both bonuses and firing policies have moderate positive effects. They are expensive policies but stack up reasonably in their cost-effectiveness to other well-studied educational interventions. Neither policy had much impact on who enters the teacher workforce. Instead, both approaches (more so with performance retention) showed that poor teachers would drop out over time.
This is pretty thin support for a pair of policies the Obama administration has effectively endorsed through their signature education initiative, the Race to the Top grant competition. At least 42 states are home to school districts that have some form of merit pay, and the number continues to grow. Performance-based teacher retention is also on the rise around the nation. Rothstein’s model suggests that it is possible that these policies may do some good, on an order of magnitude similar to the effect of reductions in class size. However, this finding assumes away the very real problems with VAMs, essential components to most performance pay or retention schemes as they exist today. As Rothstein argued in January in the NY Times, “…truly effective teaching does have long-run payoffs, but value-added analysis does a poor job of measuring it.” Until policymakers have access to reasonable performance measure, it’s tough to make a decent case for policies that hinge entirely on such metrics.