Regression to the mean

Youngme Moon explains a pardox behind accountability:

The minute we choose to measure something, we are essentially choosing to aspire to it. A metric, in other words, creates a pointer in a particular direction. And once the pointer is created, it is only a matter of time before competitors herd in the direction of that pointer.

In the 1980s and 1990s, a number of prominent hospitals agreed to make public their mortality rates. The agreement was considered a breakthrough in hospital openness, promising to give patients the kind of insider view into hospital quality that they'd never been privy to before. If a hospital's mission is to heal, then what better way to audit the performance of a hospital than to track the ultimate measure of that healing ability?

What soon became evident, however, was that a hospital's mortality rate is a function of an elaborate host of factors - including the type of patients it admits, the amount of experimental research its doctors conduct, and the degree of care it provides - each of which can heavily conflate the intended meaning of the metric.

To put it more bluntly, it soon became evident that the easiest way for a hospital to improve its mortality rate would be to stop admitting the sickest patients. Yet if all hospitals were to do this, the overall effect on the medical system would be chilling: There would be fewer hospitals accepting the most challenging cases, experimenting with the riskiest treatments, becoming specialists in the most intractable disease areas. Hospitals wouldn't get better, they would simply become more like each other.

In recent years, the college ranking system has come under fire for precisely this reason - for dampening the likelihood that universities will experiment with models of pedagogy that may not reflect well in the metrics. The rankings have made it hazardous to be a noncomformist.

This, then, is the problem with uniform systems of measurement. The more entrenched a system of measurement, the more difficult it is for a deviant, an outlier, or even an experimenter to emerge. Another way to say this is to say that a competative metric, any competative metric, tends to bring out the herd in us. The dynamic can be likened to the observer effect in physics, only applied with too little foesight: The act of measurement changes the behavior of the thing being measured.

Many see test scores as a valid indicator of good schools, good teaching and good learning. And they wish to make the data public. Like the hospital example, this is seen as a breakthrough in school openness, promising students and parents the kind of insider view into school quality that they'd never been privy to before.

If a school's mission is to educate, then what better way to audit the performance of a school than to track the ultimate measure of that education?

The problems here are many.

Firstly, where death may be the ultimate indicator of poor health, test scores are far from being the ultimate indicator of good learning.

Secondly, Youngme Moon refers to how the act of measurement changes the behaviour of the thing being measured. She is directly referring to what we know as Campbell's Law (I like to refer to this as High Stakes Testing's Kryptonite). Campbell's Law says:

Campbell's law stipulates that "the more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it was intended to monitor. Campbell warned us of the inevitable problems associated with undue eight and emphasis on a single indicator for monitoring complex social phenomena. In effect, he warned us about the high-stakes testing program that is part and parcel of No Child Left Behind.

Because the very people who are in the system are corrupted by alluring carrots and threatening sticks, it's no accident that high stake measurements skew reality.

Like the hospitals who might turn away the sickest patients in an attempt to improve its mortality rate - schools might turn away or alienate the weakest students in an attempt to improve their test scores.

Current day accountability measures attempt to ensure at least a basic level of standards are upheld; ironically, it may be those very same accountability measures that constantly drop the bar to the lowest common standard. Or as Youngme Moon puts it, this kind of accountability encourages a herdlike regression toward the mean.

Schools aren't getting better, they are simply becoming more like each other.

We are drowning in standardized mediocrity.

All this lends itself well to understanding what educational psychologist Gerald Bracey said:

There is a growing technology of testing that permits us now to do in nanoseconds things that we shouldn't be doing at all.

High stakes testing encourages schools to see struggling students as an albatross - they are seen as clientel who should be avoided. Like the hospital who turn away the sickest patients - this is a disturbingly chilling vision of public education.