John Merrow: Value Added or Value Lost?

Taking note of how much learning matters has been a passion of John Merrow's for some time now, and that is exactly why many found his post titled "Proof that teachers matter" and his favourable take on the Los Angeles Times story on judging teachers based on value-added test score data so out of character.

After listening to some of his brilliant podcasts with the likes of George Madaus, Alfie Kohn and Deborah Meier, I couldn't believe my eyes when I saw Merrow applauding the Times for bringing this data to the attention of the nation. From what I can tell, Merrow's frustration stems from a lack of accountability for teacher quality. Merrow explains: "the adults in charge have known of the damage that some teachers are doing—and have done nothing, or nothing effective anyway, about it. That’s the high tolerance for mediocrity that I find alarming, and that’s what must be addressed, and soon."

In the same breath he uses to applaud the Times, Merrow rightfully identifies the fatal flaw of the entire Times' story when he says, "I worry that it could be a step backward if it merely heightens the significance of scores on bubble tests..."

It's as if he knows that his frustration is getting the better of him, and for a brief moment, I'm hopeful he'll come to his senses.

But then he drops the ball entirely when he says, "...but that's a risk worth taking." Implying that he fully understands the folly of using test scores to judge the quality of a student, educator or education system but would rather throw caution to the wind.

John Merrow needs to heed the wisdom of the late Gerald Bracey: "There is a growing technology of testing that permits us now to do in nanoseconds things that we shouldn't be doing at all."

If I may be so bold, here are but a few points I think align well with the essence of Bracey's quote.

• Publishing test scores in the paper in an attempt to rank and sort students, teachers, or even whole schools is not only unhelpful, but it is unethical because it gives the false impression that these scores tell us anything about real learning - when really the scores tend to be simply feeding back information about affluence. 
• Depending on the research you look at, any where from 50% to 90% of the variances in test scores can be attributed to factors outside of the teacher's control.
• Campbell's Law stipulates that "the more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it was intended to monitor." In other words, if you place enough pressure on teachers to increase test scores, they will do so, but you may not want to know how they do it. For more on this, speak to Ron Paige about how the Texas Miracle was about improving test scores without improving the schools. 
• Publishing test scores to identify the 'effective' teachers will only polarize the 'good' teachers further from the 'bad'. In his post, Merrow acknowledges the kind of lobbying active parents do to ensure their children get the 'good' teachers. Which teachers do you think the inactive parents and the weaker students will end up with? Could this lobbying actually drive the value added assessments more than the quality of the teaching, making the data self-perpetuating?
• The more time teachers take to prepare students to do well on a test, the less valid and valuable the test scores will be in telling us anything more than the quality of the test preparation.
• Studies have shown a statistical association between high scores on standardized tests and relatively shallow thinking. While this is not definitive research, it encourages educators and parents to think about why so many kids who are real thinkers continue to perform poorly on tests, and why other students who don't think nearly as much tend to do well enough on the tests? No one may understand better than educators how much standardized tests both under-estimate and over-estimate what children understand; I can't think of a better definition of inaccuracy.
• Linda McNeil from Rice University raises a disturbing point when she says, "measurable outcomes may be the least significant results of learning." If there is even a shred of truth to this, then we need to hold technocrats in check because their mania for reducing everything to numbers may encourage us all to value what we can measure when we should be measuring what we value.
• Most of these tests are multiple choice, and while multiple choice tests can be clever they can never be authentic.
• How many excellent high school math teachers couldn't pass your state's English exit exam? How many politicians and policy makers couldn't pass any of your state's high stakes exams? The truth is, the idea that adults couldn't pass these tests is less of an indictment of them and more of an indictment of the tests. Alfie Kohn implores us to not confuse harder with better - after all, many of these tests "are not just ridiculously difficult but simply ridiculous."

If even a fraction of these points have merit, then to say value added assessments and the use of standardized test scores to judge the quality of schools are not ready for prime time would be a gross understatement.

John Merrow and others may be frustrated over education reform (or lack thereof), but that is no excuse for succumbing to Maslow's Maxim: If the only tool you choose to use is a hammer, then every problem looks like a nail.

The good news for John Merrow is that if Diane Ravitch can change her tune in time to be critical of Merrow's favourable stance on publishing test scores to judge teachers, then John Merrow can change his tune tomorrow and state that he was wrong to applaud the education malpractice conducted by the LA Times.