Here is what Education Hell looks like

Chicago is getting ready for a standardized test called the ISAT. Here's a 1 minute video of an Inservice session to help teachers prep for "The Vocabulary Matrix".



I have four quick points:

1. Roller coaster of emotions. I experienced a roller coaster of emotions as I watched this apocalyptic video. First, I was in shock. I couldn't believe this was happening. Second, I was angry. I couldn't imagine sitting in that classroom chanting without speaking up or walking out. Third, I was profoundly sad. If this is the nature of education reform and the future of our schools then I want nothing to do with it. Lastly, I am energized and hopeful. The only thing that cancerous education policies and practices need to survive is for good teachers to say and do nothing. Silence is not only assent to these soul-sucking test and punish practices -- silence is also betrayal to our democracy, public education and ultimately our children. This is why I'm attending the Network for Public Education Conference this weekend in Austin, Texas.

2. This is not Professional Development. This is at best a very poor inservice. Professional development is for professionals to determine their own growth and learning. This is precisely why teachers need a powerful Union that has a strong Professional Association focus to make sure that teachers have control over their own professional learning.

3. Teaching or testing? Teaching to the test and excessive test preparation invalidates inferences that can be drawn from the scores – yet they are the inevitable response to pressure to produce good test scores. Classroom time is devoured by not only the tests themselves but also practice tests, pre- and post-tests, field tests for the tests, benchmark tests, teacher tests, district tests, and state or provincial tests. Because testing is not teaching, this ultimately leads to a loss of opportunities for students to have a broad range of educational experiences, and the first things to go usually end up being the arts and physical activity – which do not lend themselves to be easily tested.

4. This is not how to teach children to read. Things are at their worst when we associate learning to read with taking a test. If you want to see what learning to read looks like, check out this post.

One last bonus horror: Is this being done to the teachers to encourage them to return to their classrooms and do exactly this to their students?

Tests Don't Assess What Really Matters

Leonie Haimson, a New York City public school parent, is the executive director of Class Size Matters, a citywide advocacy group. This post first appeared in The New York Times here.

by Leonie Haimson

Campbell's Law predicts that any time huge stakes are attached to quantitative data, the data itself will become inherently unreliable and distorted through cheating and gaming the system. In the New York City public schools, the overemphasis on standardized testing has led to test score inflation and numerous cheating scandals. Precious resources are diverted to for-profit testing companies, and learning time is lost as students spend weeks preparing for the tests, and teachers are pulled out of the classroom for days at a time to score them. Meanwhile school budgets are scraped to the bone and class sizes are rising. In New York City, class sizes in the early grades are the largest in 13 years.

The new federal mandate that teachers be judged at least in part on how well student test scores have risen is exacerbating this trend. Schools with the greatest numbers of poor, immigrant and special needs children will be increasingly subject to slash and burn tactics: mass closures and/or firings of educators, making it even more unlikely that qualified, experienced teachers will be drawn to working with the most at-risk students in the future.

What would be a better way of evaluating teachers and schools? One cannot take human judgment out of the equation. Student learning should be assessed by multiple measures, including portfolios of class work. Teachers and schools should be evaluated based on the results of those evaluations as well as through peer review and parent and student surveys.The National Academy of Sciences has not once but twice spoken out against imposing this sort of high stakes accountability scheme on our schools, and pointed out the dangers of basing irreversible decisions on erratic and inherently unreliable test scores filtered through imperfect and abstruse formulae.

Schools should also be rated on whether they provide the conditions for a quality education: small classes, a well-rounded curriculum and experienced teachers who are treated as professionals rather than cogs in a machine. Never again should we forget that learning takes place through human interaction, inspiration, creative thought and individual effort. Piling on more standardized testing undermines the conditions that will make this possible.

Jose Reyes, Ryan Braun, Ted Williams and Campbell's Law

What if yesterday's National League Batting Title is metaphorical of how our obsession with numerical data and statistics is corrupting our love for baseball and learning?

Yesterday, September 28, was the last day of Major League Baseball's regular season. The pennant races proved to be full of barn-burning action, but to be honest, that's not what I found most interesting.

Today, I want to talk about Ted Williams, Jose Reyes and Ryan Braun.

First, some history.

Seventy years ago yesterday, Ted Williams went into the last day of the 1941 regular season with an epic batting average of .39955 which would have been rounded up to .400. When faced with the opportunity to not play on the last day of the season, thus ensuring his record-book average, Williams chose to play in a double-header against the Philadelphia Athletics.

Williams ended up going 6 for 8 that day and raised his average to an impressive .406. Later, Williams explained that had he not played, he would not have deserved the title. At the time, this was Williams first of six batting titles, and today, he is the only player since 1925 to finish the regular season with a .400 batting average.

Going into yesterday's final day of the regular season, the 2011 National League Batting Title was a two horse race between New York's Jose Reyes who had a small lead over Milwaukee's Ryan Braun.

Reyes, however, chose a very different strategy than Ted Williams.

After laying down a first inning bunt single, Jose Reyes promptly pulled himself from the game. This strategy allowed Reyes to preserve his league leading average while simultaneously forcing Ryan Braun into a position where he would need to go 3 for 4, in order to have a chance at the title. Unfortunately for Braun, he went 0 for 4.

But here's the difference, while Reyes chose to quit before the first inning was even over, Braun chose to play the entire game. Reyes collects accolades while Braun goes home empty handed. Reyes gets remembered. Braun forgotten.

Are you as disgusted as I am?

This story epitomizes the cancerous effects of our mania for reducing the things we love, like baseball and learning, to numbers. When an obsession with our batting average actually convinces us to quit playing or a fetish for our grade-point average persuades us to quit learning, I think it's time to pause and reflect on how our use of data is sabotaging our ultimate goals.

If the point is to succeed and/or conquer others rather than to stretch one's thinking or discover new ideas and abilities, then it is completely logical and rational for Jose Reyes and students to want to do whatever is easiest. To do what ever is easiest, which sometimes includes quitting like Jose Reyes, can maximize the chances of success or minimize the odds of failure.

To be clear, this isn't a Jose Reyes or little Johnny problem. This is a systemic problem that is the result of our mania for reducing something as magnificently messy as baseball and learning to statistics and grades. Campbell's Law tells us that the more any one indicator (such as test scores or batting averages) are used for decision making, the more that indicator will suffer from corruption, therefore, bastardizing the very processes it was meant to monitor.

There is no substitute for what a teacher can see with their own eyes and hear with their own ears when observing and interacting with students while they are still learning. Because the children are always watching us, I fear they will learn the same dangerous lesson from Jose Reyes that they learn from grading; that real learning and strategically conquering others are the same thing.

Ted Morton and Education

On August 10 I attended a Progressive Conservative leadership question and answer session at the Alberta Teachers Association's Summer Conference. All six candidates attended: Doug Horner, Gary Mar, Allison Redford, Rick Orman, Doug Griffiths and Ted Morton.

I posted a summary of what the candidates said here.

Ted Morton

Afterward, the candidates made themselves available for discussion with individual teachers, so I approached Ted Morton.

Here is a summary of our discussion.

I introduced myself as a teacher and a farmer from Red Deer. Ted was very kind and asked me a number of questions about my family's farm. After some pleasantries I shifted the conversation to education.

During the Q & A the candidates were asked about whether they thought the Alberta Teachers Association should be restructured (read: separate union functions from professional functions perhaps similar to British Columbia). No candidate thought it would be appropriate for government to meddle in what is the teacher's responsibility to organize themselves.

Morton's response went something like this: student achievement in Alberta is very high so I see no need to fix what isn't broken.

Such a statement might at first seem awfully benign but whenever I hear someone use the words student achievement I always stop them and ask what they mean by student achievement. Sadly, student achievement has come to mean nothing more than higher test scores. And so that was my question to Ted Morton: what do you mean when you say student achievement?

Morton's response went something like this:

I am first referring to our scores on standardized testing. We test out well in comparison to other places in Canada and the world. But I know that there are other important things that are not a part of those scores that are more anecdotal evidence.

I then asked him if he was familiar with the American Education system and some of the troubles they are experiencing. His response was something like this:

Well, I know they rank like 30 something in the world in some subjects which is lower.

My response to Morton:

Ted, if all you know about a country, state, province, city, school district, school, classroom or an individual student's education are their test scores, then you don't know much about their education. If you know that some of that other anecdotal evidence is just as, or maybe even more, important than the test scores, then when you say student achievement you need to stop meaning test scores and start meaning the things that the tests can never measure like empathy, responsibility, ethics, creativity and sense of humor.

I then asked him if he was familiar with how the Americans are attempting to hold teachers "accountable" for student "achievement" by tying teacher pay to test scores. He was not familiar.

I asked him if he was familiar with Campbell's Law. He was not. So I took a minute to inform him why this was important for the Albertan context. It went something like this:

Every time we take something as complex and messy as real learning and reduce it to something as artificially and conveniently simplistic as a test score we distort and distract everyone from what is really going on in Alberta schools.

When test scores drive education two things happen. Firstly, the learning opportunities kids get are narrowed significantly to merely what will be tested and secondly, schools built on real learning and good teaching are turned into nothing more than test preparation factories and children into data.

When we place enormous importance and high stakes on any single measurement, like standardized test scores, Campbell's Law tells us that that measurement will become corrupted and will no longer serve as a reliable and valid indicator for the social processes it was suppose to monitor.

In the end, all politicians should heed what one American politician meant when he said:

Making students accountable for test scores works well on a bumper sticker and it allows many politicians to look good by saying that they will not tolerate failure. But it represents a hollow promise. Far from improving education, high- stakes testing marks a major retreat from fairness, from accuracy, from quality, and from equity.

While I can empathize with how Ted Morton might have felt at a Teachers' Conference, which might be described as the equivalent to dropping a vile of blood in a shark tank, I think Morton would have faired far better had he provided something more than familiar hollow promises.

Easing test pressure

I just read Jay Mathews' post Easing Test Pressure won't Save Kids which addresses the Atlanta cheating scandal.

The premise of the post is that despite all the evidence showing the cancerous consequences of high stakes standardized testing, Jay Mathews wants to stay the course.

I guess he hasn't seen enough blood, sweat and tears from the victims of testing. After reading plenty of other blogs and columns point towards the inappropriate and unreasonable pressure brought on by high stakes testing as a source of the cheating, Mathews isn't convinced:

I have trouble squaring with reality their conclusion that the fault was too much test pressure and that our schools will work better once we dial that down.

If he isn't prepared to dial down the test pressure, then I have to assume that Mathews wants to either keep the pressure the same or perhaps even increase it - either way, such a stance is completely ignorant to a social science law called Campbell's Law.

In their book Collateral Damage, David Berliner and Sharon Nichols summarize a stack of high stakes testing cheating scandals. Their premise is built around Campbell's law which states:

The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it was intended to monitor.

Like Wile E. Coyote's relationship with gravity, pundits like Jay Mathews find Campbell's Law to be quite inconvenient - so they simply ignore it. But if you've watched any of the cartoons, you know how this ends. Whether laws are of the physical world or the social science variety, they don't like to be ignored. Humming and hawing over their existence like Matthews does simply won't take us anywhere productive.

Here's Mathews:

We are trying, as a country, to raise achievement so that when students graduate from high school they will have the reading, writing, math and time-management skills that will allow them to do well in the workplace or college. How do we do that without motivating them and their teachers to do the work necessary to achieve those goals?

Unfortunately, student achievement has become code for nothing more than high scores on bad tests. It's also important to note that when Mathews wrote that last sentence above, he wasn't really asking a question. You see, he may have wrote this:

How do we do that without motivating them and their teachers to do the work necessary to achieve those goals?

But what he meant was this:

How do we do that without motivating them and their teachers to do the work necessary to achieve those goals!

Putting an exclamation mark at the end, rather than a question mark, is the equivalent to putting your fingers in your ears and saying, "La la la la la la". It's not an honest attempt to ask a question to which you don't know the answer -- it's an act of willful blindness.

Mathews and other pundits like him not only get school reform wrong, but they also get motivation and learning wrong, and to grasp just how wrong they are when it comes to supporting high stakes standardized testing, Richard Ryan and Netta Weinstein's Undermining Quality Teaching and Learning is a must read. They conclude:

High stakes testing represents a motivational strategy that, because it is controlling and extrinsic in character, often raises targeted test scores in the short term while producing a plethora of unintended negative long- term consequences. Nichols and Berliner (2007) discuss these issues in terms of Campbell’s law: the idea that attaching serious consequences to any indicator increases the probability that its meaning and utility will be corrupted. While that names the problem, it does not explain how and why such corruption occurs. Teaching to the test, narrowing of curricula, crowding out of enriching student activities, test preparation resulting in poor generalization of gains, and the other corruptions we described, are motivated phenomena – they occur because of the controlling nature of high stakes testing policies. These effects of high stakes testing can all be predicted from Self-Determination Theory, and indeed have been for over two decades.

You don't make change by manipulating people who have less power than you, and yet that is precisely what Bush's No Child Left Behind 1.0 and Obama's No Child Left Behind 2.0 are designed to do.

I find it more than a little disturbing that Mathews' title implies that he has the kids' best interest at heart by refusing to ease the pressure of testing. Is he, and others like him, not aware of the harmful and destructive affects high stakes testing has on children? Alfie Kohn writes:

The significance of the scores becomes even more dubious once we focus on the experience of students. For example, test anxiety has grown into a subfield of educational psychology, and its prevalence means that the tests producing this reaction are not giving us a good picture of what many students really know and can do. The more a test is made to "count"—in terms of being the basis for promoting or retaining students, for funding or closing down schools—the more that anxiety is likely to rise and the less valid the scores become.

Accountability, if nothing else, should be about transparency. That is, at the very least, citizens should know what they need to know about their schools, and yet the more pressure applied to high stakes testing, the more distorted and corrupted the scores become. 

While easing test pressure may not be sufficient it most certainly is necessary if we wish to save our kids and our schools.

David Berliner, Wile E. Coyote and Campbell's Law

However beautiful the strategy, you should occasionally look at the results.

-Winston Churchill

The clip above is an excerpt from David Berliner's keynote here.

Where there's smoke there's fire; and where there's high stakes standardized testing there's cheating.

We can bemoan this inconvenience all we want. We can play the blame game until we are blue in the face, but it won't change a damn thing.

The Emperor, in Hans Christian Andersen's Story of the Emperor's New Clothes, was naked whether he liked it or not; and high stakes standardized testing corrupts absolutely whether policy makers and education pundits like it or not.

Like the Emperor's chamberlains who walked along holding up the train that was not there at all, high stakes standardized testing has its own entourage who blindly hold up it's fraudulent vail.

Andrew Rotherham from Time.com has vail in hand:

Critics of today's push for greater accountability are quick to argue that cheating is the inevitable by-product of any high-stakes system. That's ridiculous. While fraud is a fact of life, there are numerous professions with far-reaching consequences for performance in which cheating is not rampant. Besides, that argument insults teachers by implying that they can't achieve challenging goals without cheating.

What's insulting and inaccurate is that Rotherham makes the assumption that chasing high scores on bad tests is a worthy use of our finite resources; and even if there was a consensus towards doing so, it is asinine to believe that we can simply mandate a social science law such as Campbell's Law out of existence.

Education deformers can no more skirt the real world ramifications of Campbell's Law than Wile E. Coyote could avoid the punishing effects of gravity.

If we are too move forward, we all better understand Campbell's Law at least as much as we understand gravity.

Campbell's Law states:

The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it was intended to monitor.

Here are all the posts I have written on Campbell's Law:

Campbell's Law and Standardized Tests

Regression to the mean

High Stakes Testing's Kryptonite

John Merrow: Value added or value lost

John Merrow: Value Added or Value Lost?

Taking note of how much learning matters has been a passion of John Merrow's for some time now, and that is exactly why many found his post titled "Proof that teachers matter" and his favourable take on the Los Angeles Times story on judging teachers based on value-added test score data so out of character.

After listening to some of his brilliant podcasts with the likes of George Madaus, Alfie Kohn and Deborah Meier, I couldn't believe my eyes when I saw Merrow applauding the Times for bringing this data to the attention of the nation. From what I can tell, Merrow's frustration stems from a lack of accountability for teacher quality. Merrow explains: "the adults in charge have known of the damage that some teachers are doing—and have done nothing, or nothing effective anyway, about it. That’s the high tolerance for mediocrity that I find alarming, and that’s what must be addressed, and soon."

In the same breath he uses to applaud the Times, Merrow rightfully identifies the fatal flaw of the entire Times' story when he says, "I worry that it could be a step backward if it merely heightens the significance of scores on bubble tests..."

It's as if he knows that his frustration is getting the better of him, and for a brief moment, I'm hopeful he'll come to his senses.

But then he drops the ball entirely when he says, "...but that's a risk worth taking." Implying that he fully understands the folly of using test scores to judge the quality of a student, educator or education system but would rather throw caution to the wind.

John Merrow needs to heed the wisdom of the late Gerald Bracey: "There is a growing technology of testing that permits us now to do in nanoseconds things that we shouldn't be doing at all."

If I may be so bold, here are but a few points I think align well with the essence of Bracey's quote.

• Publishing test scores in the paper in an attempt to rank and sort students, teachers, or even whole schools is not only unhelpful, but it is unethical because it gives the false impression that these scores tell us anything about real learning - when really the scores tend to be simply feeding back information about affluence. 
• Depending on the research you look at, any where from 50% to 90% of the variances in test scores can be attributed to factors outside of the teacher's control.
• Campbell's Law stipulates that "the more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it was intended to monitor." In other words, if you place enough pressure on teachers to increase test scores, they will do so, but you may not want to know how they do it. For more on this, speak to Ron Paige about how the Texas Miracle was about improving test scores without improving the schools. 
• Publishing test scores to identify the 'effective' teachers will only polarize the 'good' teachers further from the 'bad'. In his post, Merrow acknowledges the kind of lobbying active parents do to ensure their children get the 'good' teachers. Which teachers do you think the inactive parents and the weaker students will end up with? Could this lobbying actually drive the value added assessments more than the quality of the teaching, making the data self-perpetuating?
• The more time teachers take to prepare students to do well on a test, the less valid and valuable the test scores will be in telling us anything more than the quality of the test preparation.
• Studies have shown a statistical association between high scores on standardized tests and relatively shallow thinking. While this is not definitive research, it encourages educators and parents to think about why so many kids who are real thinkers continue to perform poorly on tests, and why other students who don't think nearly as much tend to do well enough on the tests? No one may understand better than educators how much standardized tests both under-estimate and over-estimate what children understand; I can't think of a better definition of inaccuracy.
• Linda McNeil from Rice University raises a disturbing point when she says, "measurable outcomes may be the least significant results of learning." If there is even a shred of truth to this, then we need to hold technocrats in check because their mania for reducing everything to numbers may encourage us all to value what we can measure when we should be measuring what we value.
• Most of these tests are multiple choice, and while multiple choice tests can be clever they can never be authentic.
• How many excellent high school math teachers couldn't pass your state's English exit exam? How many politicians and policy makers couldn't pass any of your state's high stakes exams? The truth is, the idea that adults couldn't pass these tests is less of an indictment of them and more of an indictment of the tests. Alfie Kohn implores us to not confuse harder with better - after all, many of these tests "are not just ridiculously difficult but simply ridiculous."

If even a fraction of these points have merit, then to say value added assessments and the use of standardized test scores to judge the quality of schools are not ready for prime time would be a gross understatement.

John Merrow and others may be frustrated over education reform (or lack thereof), but that is no excuse for succumbing to Maslow's Maxim: If the only tool you choose to use is a hammer, then every problem looks like a nail.

The good news for John Merrow is that if Diane Ravitch can change her tune in time to be critical of Merrow's favourable stance on publishing test scores to judge teachers, then John Merrow can change his tune tomorrow and state that he was wrong to applaud the education malpractice conducted by the LA Times.

Campbell's Law and Standardized Testing

I am so thankful for The Washington Post's Valerie Strauss. She is one of the few journalists that I know of who actually publish something about education that is worth reading.

Today a post written by Justin Snider features a handful of great reasons why we need to be more than a little skeptical of standardized test scores.

The case against standardized testing is a good one, but for many the idea that high test scores are at best unhelpful and at worst harmful to a good education system is quite counter-intuitive.

Today I wish to draw your attention, as Justin Snider has, to a well-known (but not well-known enough) social science law called Campbell's Law. Here is an excerpt from a book by David Berliner and Sharon Nichols:

Campbell's law stipulates that "the more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it was intended to monitor. Campbell warned us of the inevitable problems associated with undue weight and emphasis on a single indicator for monitoring complex social phenomena. In effect, he warned us about the high-stakes testing program that is part and parcel of No Child Left Behind.

I've come to identify Campbell's Law as high stake testing's Kryptonite. But remember that Campbell's Law is not just true for manipulative, top-down, reward and punish education policies. It is a law that rears its unavoidable head whenever you try to legislate professional behavior. In medicine, Campbell's Law plays a role in explaining why linking doctors pay to performance can leave the sickest patients without proper care, and in education, how lower performing students are being left behind by the very law that vowed not to do so.

For more on the devastating effects Campbell's Law has on education, I invite you to read David Berliner and Sharon Nichol's brilliant book Collateral Damage: How High Stakes Testing Corrupts America's Schools.

Regression to the mean

Youngme Moon explains a pardox behind accountability:

The minute we choose to measure something, we are essentially choosing to aspire to it. A metric, in other words, creates a pointer in a particular direction. And once the pointer is created, it is only a matter of time before competitors herd in the direction of that pointer.

In the 1980s and 1990s, a number of prominent hospitals agreed to make public their mortality rates. The agreement was considered a breakthrough in hospital openness, promising to give patients the kind of insider view into hospital quality that they'd never been privy to before. If a hospital's mission is to heal, then what better way to audit the performance of a hospital than to track the ultimate measure of that healing ability?

What soon became evident, however, was that a hospital's mortality rate is a function of an elaborate host of factors - including the type of patients it admits, the amount of experimental research its doctors conduct, and the degree of care it provides - each of which can heavily conflate the intended meaning of the metric.

To put it more bluntly, it soon became evident that the easiest way for a hospital to improve its mortality rate would be to stop admitting the sickest patients. Yet if all hospitals were to do this, the overall effect on the medical system would be chilling: There would be fewer hospitals accepting the most challenging cases, experimenting with the riskiest treatments, becoming specialists in the most intractable disease areas. Hospitals wouldn't get better, they would simply become more like each other.

In recent years, the college ranking system has come under fire for precisely this reason - for dampening the likelihood that universities will experiment with models of pedagogy that may not reflect well in the metrics. The rankings have made it hazardous to be a noncomformist.

This, then, is the problem with uniform systems of measurement. The more entrenched a system of measurement, the more difficult it is for a deviant, an outlier, or even an experimenter to emerge. Another way to say this is to say that a competative metric, any competative metric, tends to bring out the herd in us. The dynamic can be likened to the observer effect in physics, only applied with too little foesight: The act of measurement changes the behavior of the thing being measured.

Many see test scores as a valid indicator of good schools, good teaching and good learning. And they wish to make the data public. Like the hospital example, this is seen as a breakthrough in school openness, promising students and parents the kind of insider view into school quality that they'd never been privy to before.

If a school's mission is to educate, then what better way to audit the performance of a school than to track the ultimate measure of that education?

The problems here are many.

Firstly, where death may be the ultimate indicator of poor health, test scores are far from being the ultimate indicator of good learning.

Secondly, Youngme Moon refers to how the act of measurement changes the behaviour of the thing being measured. She is directly referring to what we know as Campbell's Law (I like to refer to this as High Stakes Testing's Kryptonite). Campbell's Law says:

Campbell's law stipulates that "the more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it was intended to monitor. Campbell warned us of the inevitable problems associated with undue eight and emphasis on a single indicator for monitoring complex social phenomena. In effect, he warned us about the high-stakes testing program that is part and parcel of No Child Left Behind.

Because the very people who are in the system are corrupted by alluring carrots and threatening sticks, it's no accident that high stake measurements skew reality.

Like the hospitals who might turn away the sickest patients in an attempt to improve its mortality rate - schools might turn away or alienate the weakest students in an attempt to improve their test scores.

Current day accountability measures attempt to ensure at least a basic level of standards are upheld; ironically, it may be those very same accountability measures that constantly drop the bar to the lowest common standard. Or as Youngme Moon puts it, this kind of accountability encourages a herdlike regression toward the mean.

Schools aren't getting better, they are simply becoming more like each other.

We are drowning in standardized mediocrity.

All this lends itself well to understanding what educational psychologist Gerald Bracey said:

There is a growing technology of testing that permits us now to do in nanoseconds things that we shouldn't be doing at all.

High stakes testing encourages schools to see struggling students as an albatross - they are seen as clientel who should be avoided. Like the hospital who turn away the sickest patients - this is a disturbingly chilling vision of public education.

High Stake Testing's Kryptonite

The effects of high-stakes testing should not come as a surprise to us. That some very good teachers feel the pressure to cheat for their students in a kind of Robin Hood act to save their children and their school from undue harm should make sense. With the proper pressure, even very good people can be forced into doing 'bad' things.


A well-known (but not well-known enough) social-science law called Campbell's Law helps to explain why high-stakes testing will NEVER work the way it was intended. David Berliner and Sharon Nichols explain Campbell's Law in their book Collateral Damage: How High Stakes Testing Corrupts America's Schools.

Campbell's law stipulates that "the more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it was intended to monitor. Campbell warned us of the inevitable problems associated with undue eight and emphasis on a single indicator for monitoring complex social phenomena. In effect, he warned us about the high-stakes testing program that is part and parcel of No Child Left Behind.

Campbell's Law should disturb anyone who uses data to make decisions. If the stakeholders responsible for caring through with the day to day doing that the data measures feel like their work is attached to a high stakes indicator, they will work to corrupt the validity and reliability of the measurement.

Berliner and Nichols summarize:

Apparently, you can have (a) higher stakes and less certainty about the validity of assessment or (b) lower stakes and greater certainty about validity. But you are not likely to have both high stakes and high validity. Uncertainty about the meaning of test scores increases as the stakes attached to them become more severe.

The high stakes reward-punishment nature of today's testing regime has contributed to its own demise. Everytime someone places more emphasis on testing, the more likely the results gathered will be comprimised - making the data less valid and any decisions based on that data less reliable.

This is a complicated idea with huge implications for policy makers. We can't afford to ignore this law anymore.

No matter how valid or reliable we think certain data is, if high-stakes reward-punishment consequences are to follow the data, then that data becomes more and more invalid and unreliable.