Tuesday, March 1, 2011

McNamara's Fallacy and Standardized Testing

I've come to realize that standardized tests serve mostly to make dreadful forms of teaching appear successful.

-Alfie Kohn

When learning is enslaved by the quantifiable, we fall victim to the McNamara Fallacy which refers to the quantifying of success while ignoring other variables - particularly variables that are inconveniently difficult to measure.

Charles Handy unloads a powerful indictment on those who subscribe to valuing only what can be measured:

The first step is to measure whatever can be easily measured. This is ok as far as it goes. The second step is to disregard that which can't be easily measured or to give it an arbitrary quantitative value. This is artificial and misleading. The third step is to presume that what can't be measured easily really isn't important. This is blindness. The fourth step is to say that what can't be easily measured really doesn't exist. This is suicide.
This kind of robotic, reduce everything to numbers, mad science has the education system so distracted, we've almost entirely lost the plot.

I'll go a step further - standardized testing is dehumanizing learning; which, if you think about it, is utter lunacy.

Sometime ago, Ric Murry left a comment on my post Bubble Sheet Season, and I think he summed up this mess nicely:

I propose the reason for all the data is that there are people who love numbers more than people. Therefore, they focus on data and not students. They seek to make teaching a scientific process instead of a artful practice; likely because they lack people skills. They are afraid to truly interact with others. They have found a home in education, where kids are no longer the focus. How can that be? 
Sad thing is, the majority of people are afraid of their math skills, so they believe that someone who can discuss numbers must be smarter, and therefore correct. Bad assumption.


  1. It is painful for me to realize that my students have mastered this quarter's math standards, grown by leaps and bounds in reading and learned to think critically in social studies and science, but the bottom line will be the test. They built solar ovens? Great, but lets see the test. They designed roller coasters? Nice, but where's the data. That's cute that a kid said he fell in love with poetry and now writes it on his free time, but let's see how he analyzes the poem on the Galileo test.

    I will spend a wasted half hour each day this weekend teaching them how to transfer what they already know into the bizarre wording and format of the BIG TEST, just so they score well and we are left alone to do real learning the rest of the time.

    I don't like the wasted half hour, but it's the price I pay if I want to continue being left alone the rest of the quarter.

  2. Thanks for the provocative post. I think it important to remember that there is a difference between data and evidence. All data is evidence, but not all evidence is data. If we want evidence driven decision making in education, we must be open to models of qualitative input, observations, student and parent feedback,etc.

  3. The comments you added by Ric Murry were very powerful and my experience bears this out. How do we help these folks to shift their view to be more student centered? I have always been an optimist and think most things are possible.
    Best regards,
    RJ Johnson

  4. I often lament that it is forgotten that learning is a human interaction. In a 1979 grad class in Statistics, we discussed whether the digitization of human behavior was good or bad. I argued the latter, because it dehumanizes. It categorizes and limits our options. Standard testing's pre-eminence reminds me of that class. It would be ridiculous were it not so sad.

  5. I think Saskatchewan educators would agree that our limited common assessments, particularly the provincial ones, have narrowed the curriculum significantly. Field trips, inquiry, and projects are questionable these days.


Follow by Email