A system meant to reward teachers for improving student performance and taking on tough teaching assignments in struggling schools was at the center of the dispute that led to last week’s teachers' strike against Denver Public Schools.
The Denver Classroom Teachers Association, which was part of the initial design of ProComp, argued that it had become too complicated, that it was difficult to make the connection between actions and rewards, and that teachers have a hard time predicting their annual income. As housing has grown more expensive in Denver, teachers have also come to prefer overall increases in salary instead of one-time bonuses.
But research on ProComp and similar pay-for-performance systems shows principals also have an important role in whether such systems are successful — especially if bonuses or raises are tied to teacher evaluation and feedback systems.
According to the National Council on Teacher Quality’s 2018 analysis of 124 large districts in the country, 51 have a system in which an annual salary increase, which becomes a permanent part of the teacher’s pay, or an annual bonus, which has to be re-earned each year, is tied in some way to the teacher’s performance.
"One of the key assumptions that these systems make is that the evaluation data will be useful to teachers and their supervisors — including principals — for identifying strengths and weaknesses and informing decisions about what kind of professional development teachers need in order to improve their practice," Laura Hamilton, a distinguished chair in learning and assessment for the RAND Corp., said in an email. "Principals play an important role in these systems; they are usually — though not always — the ones who conduct the classroom observations and share the feedback with teachers, and they also influence schoolwide climate and relationships by fostering, for instance, a culture of collaboration and support rather than competitiveness."
Principals needed more time to observe
When RAND and the American Institutes for Research (AIR) evaluated the Bill and Melinda Gates Foundation’s Intensive Partnerships for Effective Teaching, which focused on three school districts and four charter management organizations, the researchers learned that many principals didn’t have sufficient training or enough time to conduct all the observations required by the districts.
“Some teachers we interviewed reported that principals sometimes shortened their planned observation time or were not available to address other concerns and needs that arose,” the researchers wrote in their final report, released last year.
Another element in the success of such systems is whether teachers — and school leaders — trust the measures being used to judge a teacher’s effectiveness. If a principal doesn’t think an evaluation system is reliable, teachers aren’t likely to, either. RAND found that over time, principals were increasingly likely to express agreement with the statement, “I don’t need teacher evaluations or teacher effectiveness ratings to know who the good and bad teachers are.”
While the Intensive Partnership study did not include Denver, it also found — as with ProComp — that teachers are sometimes confused about how they are evaluated or rewarded. For example, the RAND/AIR interviews of teachers showed some believed they could be fired based on one low evaluation score. Principals, therefore, can ensure their teachers have the facts about the evaluation and merit system.
“Principals can play an important role in ensuring that teachers and other staff in their buildings understand how the evaluation and pay systems work,” Hamilton said, adding that even if they can’t address some of the structural issues that are problematic for teachers “they can serve as liaisons between building staff and central office leadership to help ensure good two-way communication.”
For that to happen, however, principals need a good understanding of how such systems work. When University of Colorado Denver researchers evaluated ProComp in 2011, they found many principals didn’t understand all of the system’s incentives.
“To the extent that teachers and principals do not understand the details of the compensation system, it is highly unlikely they will be motivated by it, or that they will change their behavior to pursue these incentives,” the researchers wrote. They added, however, that some principals were succeeding at connecting ProComp to school improvement goals and plans for raising student achievement.
A later study by the University of Colorado, Boulder (UCB), showed principals were also an important component in whether teachers set quality student growth objectives (SGOs), which, if accomplished, could result in bonuses or raises for teachers. One of the complaints about SGOs was that there was wide variation across schools in teachers earning extra money when their students reached the goals.
Only small effects on student outcomes
In general, research has found that systems that link a teacher’s pay to his or her effectiveness in the classroom are not leading to dramatic improvements in student achievement. RAND’s evaluation of the Intensive Partnership sites showed low-income, minority students especially did not benefit.
The Teacher Incentive Fund, a U.S. Department of Education initiative to create performance-based compensation systems, resulted in student learning gains of about three to four weeks, according to a Mathematica Policy Research evaluation. UCB’s research on ProComp showed small positive effects in math, and small negative effects in reading and writing. “However, none of these effects are large enough to be considered of much practical significance,” the researchers wrote.
While it’s unclear what ongoing impact the Denver strike will have on merit-based pay for teachers, some principals have expressed strong support for such systems. In a 2016-17 evaluation of the Dallas Independent School District’s Teacher Excellence Initiative — which includes annual evaluations, observations, student achievement data and student experience surveys — almost three-fourths of administrators showed overall satisfaction with the program on a perception survey.
Administrators responded in the survey that the program helps to identify effective teachers, would help teachers earn more money in the long run, and helps to keep teachers in “campus-based positions.”