What was school like 80 years ago?

I listened to John Merrow's podcast with Alfie Kohn with my wife's 93 year old grandfather.

The moment it was over, he said, "you know, we never said a word in school. We sat there in our desks and the teacher talked."

"Did you ever raise your hand," I asked.

He laughed, "oh God no. Almost never. I wouldn't dare. You know Shannon (another one of his grand-daughters who teaches grade three) teaches and she says her kids are always talking and discussing and working together."

Bennie then said, "You know, I left school with a lot of questions."

I could see this last comment was said with a sad, regretful tone.

"I was kind of bashful, and I felt like I just shouldn't interrupt the teacher. But you know, years later, I was talking to a Wildlife Officer and he was giving a seminar on some such topic and he told us not to be afraid to ask questions - after all, there's probably a 160 other guys waiting to ask the same question."

I could tell that Bennie took great relief in knowing that his questions were falling on a safe crowd.


"You know, to me grade 9 was a chore, but grade 10, 11 were more enjoyable."

So I asked him, "What was the difference?"

"In 1935, I was in grade 9 and, I had a coal oil lamp. It gave off a kind of red light - not bright at all. But then for grade 10 and 11 we went to a gas lamp - it gave off far better light.

"After grade 10, my old man said he couldn't afford to send me to school, but mom said go ahead and register and he'll have to pay the five dollars a month."

"Was 5 dollars too much money?"

"Well, when you didn't have much, anything is too much, but no - we could afford it. Dad just wanted me to stay home and help him on the farm."

Bennie then asked me, "what grades did you like?"

"Well, I really liked learning about the World Wars. I like the tanks and the aeroplanes. I then got interested in the politics as I learned about Hitler, Churchill, Stalin and Roosevelt. But you got to remember that I was interested in learning about this at home on the Internet."

"Oh, well I learned everything from the radio."

I explained, "when you listened to the radio, you had to listen to whatever they decided to tell you. Imagine if you could have told the radio to share with you what you wanted to hear about - that's the Internet."

He shook his head with disbelief, "oh, wow. That would have been something else."

He's right.

I really enjoyed my chat with Bennie.

John Merrow: Value Added or Value Lost?

Taking note of how much learning matters has been a passion of John Merrow's for some time now, and that is exactly why many found his post titled "Proof that teachers matter" and his favourable take on the Los Angeles Times story on judging teachers based on value-added test score data so out of character.

After listening to some of his brilliant podcasts with the likes of George Madaus, Alfie Kohn and Deborah Meier, I couldn't believe my eyes when I saw Merrow applauding the Times for bringing this data to the attention of the nation. From what I can tell, Merrow's frustration stems from a lack of accountability for teacher quality. Merrow explains: "the adults in charge have known of the damage that some teachers are doing—and have done nothing, or nothing effective anyway, about it. That’s the high tolerance for mediocrity that I find alarming, and that’s what must be addressed, and soon."

In the same breath he uses to applaud the Times, Merrow rightfully identifies the fatal flaw of the entire Times' story when he says, "I worry that it could be a step backward if it merely heightens the significance of scores on bubble tests..."

It's as if he knows that his frustration is getting the better of him, and for a brief moment, I'm hopeful he'll come to his senses.

But then he drops the ball entirely when he says, "...but that's a risk worth taking." Implying that he fully understands the folly of using test scores to judge the quality of a student, educator or education system but would rather throw caution to the wind.

John Merrow needs to heed the wisdom of the late Gerald Bracey: "There is a growing technology of testing that permits us now to do in nanoseconds things that we shouldn't be doing at all."

If I may be so bold, here are but a few points I think align well with the essence of Bracey's quote.

• Publishing test scores in the paper in an attempt to rank and sort students, teachers, or even whole schools is not only unhelpful, but it is unethical because it gives the false impression that these scores tell us anything about real learning - when really the scores tend to be simply feeding back information about affluence. 
• Depending on the research you look at, any where from 50% to 90% of the variances in test scores can be attributed to factors outside of the teacher's control.
• Campbell's Law stipulates that "the more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it was intended to monitor." In other words, if you place enough pressure on teachers to increase test scores, they will do so, but you may not want to know how they do it. For more on this, speak to Ron Paige about how the Texas Miracle was about improving test scores without improving the schools. 
• Publishing test scores to identify the 'effective' teachers will only polarize the 'good' teachers further from the 'bad'. In his post, Merrow acknowledges the kind of lobbying active parents do to ensure their children get the 'good' teachers. Which teachers do you think the inactive parents and the weaker students will end up with? Could this lobbying actually drive the value added assessments more than the quality of the teaching, making the data self-perpetuating?
• The more time teachers take to prepare students to do well on a test, the less valid and valuable the test scores will be in telling us anything more than the quality of the test preparation.
• Studies have shown a statistical association between high scores on standardized tests and relatively shallow thinking. While this is not definitive research, it encourages educators and parents to think about why so many kids who are real thinkers continue to perform poorly on tests, and why other students who don't think nearly as much tend to do well enough on the tests? No one may understand better than educators how much standardized tests both under-estimate and over-estimate what children understand; I can't think of a better definition of inaccuracy.
• Linda McNeil from Rice University raises a disturbing point when she says, "measurable outcomes may be the least significant results of learning." If there is even a shred of truth to this, then we need to hold technocrats in check because their mania for reducing everything to numbers may encourage us all to value what we can measure when we should be measuring what we value.
• Most of these tests are multiple choice, and while multiple choice tests can be clever they can never be authentic.
• How many excellent high school math teachers couldn't pass your state's English exit exam? How many politicians and policy makers couldn't pass any of your state's high stakes exams? The truth is, the idea that adults couldn't pass these tests is less of an indictment of them and more of an indictment of the tests. Alfie Kohn implores us to not confuse harder with better - after all, many of these tests "are not just ridiculously difficult but simply ridiculous."

If even a fraction of these points have merit, then to say value added assessments and the use of standardized test scores to judge the quality of schools are not ready for prime time would be a gross understatement.

John Merrow and others may be frustrated over education reform (or lack thereof), but that is no excuse for succumbing to Maslow's Maxim: If the only tool you choose to use is a hammer, then every problem looks like a nail.

The good news for John Merrow is that if Diane Ravitch can change her tune in time to be critical of Merrow's favourable stance on publishing test scores to judge teachers, then John Merrow can change his tune tomorrow and state that he was wrong to applaud the education malpractice conducted by the LA Times.

Taking Note of John Merrow and Grant Wiggins

I was reading John Merrow's blog Taking Note and his take on the Los Angeles Times' story on judging teachers based on value-added test score data. If you have a moment, I suggest you go read it here. And make sure you read the comments because it is there that you see some heavy hitters chiming in. Here's what I've learned:

• John Merrow shocked the hell out of me when he said, "I applaud the Times for bringing this to the forefront. I worry that it could be a step backward if it merely heightens the significance of scores on bubble tests, but that’s a risk worth taking." John is wrong - nothing is worth heightening the significance of these poisonous tests. Nothing. 

• David Cohen is a beacon for progressive education. He was right to say, "I’m a little surprised. The problems with VAM methodology are well known. The problems with the tests themselves are well known. It was incredibly irresponsible of the Times to report using the labels “effective” and “ineffective.” The data simply cannot support those evaluations."

• Mike Klonsky properly called John Merrow out for sloppy reasoning when he said, "Well, there it is, John. In your own words. You don’t like bubble tests or simplistic value added measures as a way of judging schools or teachers, but you applaud the Times for using them and for outing teachers of kids who score low on those bubble tests."

• Grant Wiggins is apparently a pro-standardized tester and teacher-hater: "I am totally behind John and the LA Times on this. His claim is undeniable: we all know who the better teachers are. I just fail to understand why teachers should get a free pass on accountability when the data is public data. How is it different from a player hitting .167 for his team – in the paper, every day?" If you think Wiggins's baseball analogy is on the mark, you need to read this! Grant you should take a look at it too.

• David Cohen's second comment should be published on the front page of every major newspaper in the country: " This type of analysis is so cold and abstract that it frustrates those of us close to the work. One of my students committed suicide one year. Do you think that matters? How about you, John? Is there a space in the data spreadsheet for that? Another year, my son was born and I missed a few weeks of school. Is that allowable? Is there a space in the spreadsheet for that? Do you think that three weeks of substitute teaching might affect the class? One of my colleagues just went through a fight with cancer and is doing great. Is there a mark on the spreadsheet for “Cancer survivor” and a way to keep that year of data out of “her” test results, when much of the year was with a long-term substitute whose name will NOT appear on the students’ records? The folks who push this stuff down our throats at the school and classroom level NEVER seem to have answers to these questions." 

• Dan Willingham gets it: "John, echoing what David B. Cohen says, if you don’t endorse simple bubble tests or simplistic value added measures, and if you don’t endorse using only VA measures to evaluate teachers, I’m surprised that you liked the Times article."

• Deborah Meier is right. We should all be outraged by the outrageous: "Shocking, awful, embarrassing–especially since I have long admired you both–Grant and, John. I often thought Grant’s thinking cool/cold/logic without the common human touch, but I also respected the insights that flowed from his logic. I just can’t believe you and he wrote that junk, John. What do you think it does to kids, families, human beings…even if the test evaluations were a good measure. Nobody in the field of testing would argue for it–as you surely know."

• Diane Ravitch is right: "The naming of names based on dubious measures is truly disgraceful. I am disappointed and shocked to see you endorsing this approach."

• Jon Becker gets it. Multiple sources of information are needed to assess good teaching and real learning. 

Using standardized test scores as a means of judging good teaching and real learning is a morally bankrupt and intellectually indefensible position to maintain. John Merrow and Grant Wiggins should know better.