The Vanderbilt POINT study certainly sent shockwaves through the education reform ranks earlier this week and led to a piling on of “I told you so…” posts and other shots across the Twitterverse. But before we throw up our hands and decide that there is no other viable compensation system for educators besides the traditional step and level pay system, let’s put the POINT study in perspective.

POINT is important because it was an experimental design, the gold standard for establishing causality. The study tested a narrow scheme of significant amounts of cash (up to $15,000) for getting high value-added results for middle school math teachers in Nashville. The study ran over a three year time span. What POINT tells us is that straight cash-for-scores plans don’t aren’t significantly different than doing nothing when it comes to educator compensation.

It’s also important to note that the study results cut both ways – that is that the step and level pay system alone is no better than the performance pay scheme used in this experiment.

The really important finding is that a performance incentive plan doesn’t do much of anything as a stand-alone reform. So, districts considering a compensation change that ONLY involves cash-for-tests aren’t likely to see an impact on student achievement. Hard charging education reformers pushing these sort of models should take heed – you may be wasting precious school money.

Despite calls that this definitively closes the door on “merit pay” plans from the status quo defenders, there are a number of things this study doesn’t tell us. For example:

• POINT doesn’t address any interactive or compounding effects performance pay has when combined with other human capital initiatives. Many advocates for changes to compensation systems (myself included) argue for comprehensive human capital changes that include compensation reform. That is, changes to compensation come with better professional learning opportunities, better student data systems, better evaluation systems, better collaborative support systems. Something like the TAP System is a good example. Consider this upcoming event, the National Workshop on Strategic Compensation and you’ll see what I’m talking about – much of it focuses on things that don’t relate directly to compensation.
• POINT doesn’t address the long term effects of a compensation system that systematically protects the best employees from outside employment options while simultaneously exposing weaker employees to these outside options. To be more direct, we can “golden handcuff” the most effective teachers and increase their retention rates, while creating an environment where less effective teachers have an incentive to look around.
• POINT doesn’t address the ability of a strategic compensation system to put more money earlier in the hands of less experienced, but effective, educators. We know that attrition rates for early educators are high. Once an educator gets high enough up in the salary schedule they have an incentive to keep coming back for more. POINT doesn’t tell us what might happen if we offered more money earlier in careers to effective, but less experienced teachers. We would likely see an attraction of more capable people and higher retention rates of effective but less experienced educators if they didn’t have to wait until later in their careers to make a decent wage.

Don’t get me wrong – the POINT study is important and it was definitely a shocker to see just how insignificant the differences were from the traditional pay system. Like others, I expected the results to be, at worst, mixed.

But let’s keep it in perspective. Despite the powerful inferences we can draw from the experimental design and the credibility of the researchers assembled to run the study, it’s still fundamentally one study looking at one narrow implementation of performance pay.

The right path is for those considering a compensation reform to do so in the context of a larger human capital reform and to be willing to go the long haul before seeing results. It would be a real shame if this one failure was misused to stop any kind of creative experimentation with educator compensation systems.

Jason Glass
New York, NY