Renovation is not the same as innovation

Youngme Moon writes about the limits of data in her book Different:

We allow this kind of formal market research to seduce us nevertheless. Consequently, we are more committed than ever to gathering it, using surveys or focus groups or customer interviews. We are more committed than ever to aggregating it, in the form of PowerPoint presentations and executive summaries. We are more commited than ever to drawing conclusions from it, conclusions that typically lead to some kind of renovation of our existing value proposition. And yet, as I noted in early chapter, renovation is not the same things as innovation, and there are times when I wonder whether we wouldn't be better off putting our market machinery aside for a few minutes, just to see what we're capable of coming up with without it.

I know that I am not the first to point this out, but here is the problem with formal market research. Consumers will always be able to tell us how much better they'd like our products to be. But we cannot expect them to be able to tell us how different thsoe products could be. And more important, we cannot expect them to be able to tell us how it might be possible for us to surprise them.

What this means is that if we want to move beyond the incremental kinds of augmentations that dominte our product marketing activity, we need to look beyond the granular pieces of data that our market research infrastructure is likely to generate. Those peices may be rigorously objective, but they are guaranteed to be woefully incomplete. They will only give us half the story. To get the other half, we need to take responsibility fo the hard work fo imagination ourselves.

I am alway hesitant to make education metaphorical of business, but Youngme Moon's vision of business meshes well with my vision of education.

Data has come to be God in education. In business, data is driven by market research. In education, data is driven by test scores. And those test scores then drive our decisions.

Data is very limiting mostly because it tends to conceal more than it reveals.

Test score data tends to squander innovation and places a kind of blinder on educators - employing a kind of test score tunnel vision. Your goal, if you chose to accept it (as if we had a choice), is to raise the scores!

It's time we put our test score, data-driven machinery aside for a moment and let's see what we are capable of coming up with without it. It's time we strike a balance between our analytical and intuitive thinking.

Our over-dependence on data is sabotaging reform in an educationally neutering kind of way.

Learning Purists

Youngme Moon writes about baseball in her book Different:

One of the bestselling books of 2003 was Micheal Lewis's Moneyball, which made the charged case that the right way to think about baseball was through the lens of extreme empiricism. The book was a slap in the face to baseball purists, because the implication was that the collective wisdom of a whole crew of baseball traditionalists - scouts, general managers, and so on - couldn't hold a candle to a statistician with a laptop.

In the polemic that followed the book's publication what became clear was that there was two ways of capturing reality - by the things that are measurable, and by the things that are not - and because the two don't overlap very much, it is easy for proponents of either to diminish the significance of the other.

And yet this is precisely why it can be such a fascinating experience to sit down and watch a really important baseball game - say, the seventh game of the World Series - with a hard-core baseball fan. The hard-core baseball fan is a walking repository of a vast amount of statistical data. She can tick off batting percentages, on-base percentages, runs batted in, and walk-to-strikeout ratios. She can tell you whether a pitch was an outside cut fastball or an inside curve, and what the odds are that this hitter will be able to get on base against the pitcher, in this given situation.

At the same time, the insight that the hard-core baseball fan can bring to a game can go far beyond that which can be expressed in purely statistical terms. When it comes to a big game like the seventh game of the World Series, the hard-core fan understands the journey that each team has traveled to arrive at this place, at this time. She knows the personalities of each  and every player, and is aware of the individual obstacles each has had to overcome to be in this coveted position. For the hardcore fan, the drama is not just in what is happening in the moment, it is in the collision of the present against all that has taken place in the past. Statistics count for something, certainly, but so do a lot of other things. Like context. Chemistry. Momentum. History. The beauty of the game. Which means that for the hard-core fan, to reduce the game to that which can be measured is to lose a lot.

I realize that I risk repeating myself here, but I draw this analogy as a way of illustrating the observational myopia that an over-reliance on empirics can create. As bussinesspeople, we can't afford to disregard the data that market research presents us with; we need to gather it, sift through it, and try to make sense of it as best we can. However, once we have done so, we can't assume that our work is done.

Much better that we approach our craft in the same way that a baseball purist approaches the game. Wherin we respect that statistics matter, but we also respect the fact that to reduce the game to numbers alone is to divest it of its soul.

If we only pay attention to things that we can measure, we will only pay attention to the things that are easily measurable. And in the process, we will miss a lot.

When we reduce education to things that are simply easy to measure, we miss a lot. To focus too much on the testing that typically occurs at the end of the year is to ignore the journey of the entire "regular season".

Like baseball purists, learning purists understand that to reduce learning to numbers in an obsessive effort to quantify quality in a strictly emprical manner risks losing the spirit of learning.

Positive deviants

Youngme Moon writes in her book Different about positive deviants:

The writer-physician Atul Gawande has written about the phenomenon of "positive deviants" in the medical profession, that small set of players who are mired in the same environmental conditions as everyone else but stubbornly refuse to allow themselves to be constrained by conventional wisdoms, and as a consequence are able to identify fresh and often counter-traditional ways to address seemingly intractable problems.

The idea of a positive deviant intrigues me. I'm not a big fan of labels, but it is probably safe to say that I am a kind of positive deviant. When I talk about abolishing grading, changing the homework default or reducing/eliminating curriculum, I am often encountered with a myriad of responses that resemble pragmatism, realism, skeptism, and flat-out apathy.

I'll be honest, I am sometimes shocked by how resistant and closed minded some teachers can be towards change.

Maybe I'm being unfair. Maybe this isn't a teacher thing but a human being thing. I don't know.

Either way, I find it disturbing how rigidly opposed or hopelessly apathetic educators can be towards change.

And then I think of an even more disturbing thought. If this is how they treat positive deviants who are their colleagues, how do they treat the positive deviants that are in their classroom?

anti-creativity checklist

Education has been hijacked.

If education reform can be saved, it must be done so by those closest to the action... I'm talking about the students, teachers and parents.

If we are ever to get education reform right, we are going to need to be creative.

We need to listen to Youngme Moon's 14 rules to avoiding creativity and refuse to follow even one of her suggestions.

If we are to save ourselves and our children from education hell, we need to both listen and ignore her rules,

We can't afford not to...

Regression to the mean

Youngme Moon explains a pardox behind accountability:

The minute we choose to measure something, we are essentially choosing to aspire to it. A metric, in other words, creates a pointer in a particular direction. And once the pointer is created, it is only a matter of time before competitors herd in the direction of that pointer.

In the 1980s and 1990s, a number of prominent hospitals agreed to make public their mortality rates. The agreement was considered a breakthrough in hospital openness, promising to give patients the kind of insider view into hospital quality that they'd never been privy to before. If a hospital's mission is to heal, then what better way to audit the performance of a hospital than to track the ultimate measure of that healing ability?

What soon became evident, however, was that a hospital's mortality rate is a function of an elaborate host of factors - including the type of patients it admits, the amount of experimental research its doctors conduct, and the degree of care it provides - each of which can heavily conflate the intended meaning of the metric.

To put it more bluntly, it soon became evident that the easiest way for a hospital to improve its mortality rate would be to stop admitting the sickest patients. Yet if all hospitals were to do this, the overall effect on the medical system would be chilling: There would be fewer hospitals accepting the most challenging cases, experimenting with the riskiest treatments, becoming specialists in the most intractable disease areas. Hospitals wouldn't get better, they would simply become more like each other.

In recent years, the college ranking system has come under fire for precisely this reason - for dampening the likelihood that universities will experiment with models of pedagogy that may not reflect well in the metrics. The rankings have made it hazardous to be a noncomformist.

This, then, is the problem with uniform systems of measurement. The more entrenched a system of measurement, the more difficult it is for a deviant, an outlier, or even an experimenter to emerge. Another way to say this is to say that a competative metric, any competative metric, tends to bring out the herd in us. The dynamic can be likened to the observer effect in physics, only applied with too little foesight: The act of measurement changes the behavior of the thing being measured.

Many see test scores as a valid indicator of good schools, good teaching and good learning. And they wish to make the data public. Like the hospital example, this is seen as a breakthrough in school openness, promising students and parents the kind of insider view into school quality that they'd never been privy to before.

If a school's mission is to educate, then what better way to audit the performance of a school than to track the ultimate measure of that education?

The problems here are many.

Firstly, where death may be the ultimate indicator of poor health, test scores are far from being the ultimate indicator of good learning.

Secondly, Youngme Moon refers to how the act of measurement changes the behaviour of the thing being measured. She is directly referring to what we know as Campbell's Law (I like to refer to this as High Stakes Testing's Kryptonite). Campbell's Law says:

Campbell's law stipulates that "the more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it was intended to monitor. Campbell warned us of the inevitable problems associated with undue eight and emphasis on a single indicator for monitoring complex social phenomena. In effect, he warned us about the high-stakes testing program that is part and parcel of No Child Left Behind.

Because the very people who are in the system are corrupted by alluring carrots and threatening sticks, it's no accident that high stake measurements skew reality.

Like the hospitals who might turn away the sickest patients in an attempt to improve its mortality rate - schools might turn away or alienate the weakest students in an attempt to improve their test scores.

Current day accountability measures attempt to ensure at least a basic level of standards are upheld; ironically, it may be those very same accountability measures that constantly drop the bar to the lowest common standard. Or as Youngme Moon puts it, this kind of accountability encourages a herdlike regression toward the mean.

Schools aren't getting better, they are simply becoming more like each other.

We are drowning in standardized mediocrity.

All this lends itself well to understanding what educational psychologist Gerald Bracey said:

There is a growing technology of testing that permits us now to do in nanoseconds things that we shouldn't be doing at all.

High stakes testing encourages schools to see struggling students as an albatross - they are seen as clientel who should be avoided. Like the hospital who turn away the sickest patients - this is a disturbingly chilling vision of public education.

Real Learning is different

Youngme Moon writes in her book Different:

When it comes to track-and-field, we may want our runners moving in the same direction, but when it comes to medical care or higher education, we may not.

Moon's book is one of mind-boggling paradoxes. I fear that the traditional, old-school businessperson's head might spontaneously combust upon flipping the first page. Her message is that counter-intuitive!

Her first chapter deals with the competetive herd instinct. The paradox here is that when individuals or companies compete they develop a "herdlike regression towards the mean" that stifles the differences among them.

The entire premise of Youngme Moon's book throws yet more fuel on the rethink education reform fire. It is very likely that our current push for more and more commonality and standardization in an effort to make "objective" comparisons is cancerous to our ultimate goals.

We may talk the talk of differentiation and life-long learning, but for the most part education reform is walking the other direction - making these terms nothing more than a half-hearted punch line in a joke that would be funny if it weren't so damn sad.

What if our greatest efforts to create an "objective" and common education system simply populates our world full of children lost in a blur of similarity? What if the testing treadmill does nothing more than encourage kids to compete like crazy in an effort to keep up with each other - leaving them all just like everyone else?

What if we succeed in this race to the top and we find out that we were the first to get... no where?

How can educators be different in a way that makes a difference?


Maybe it means saying to no to uniformity when everyone else is saying yes.

Or saying yes to true differentiation when everyone else is saying no.

Maybe it means ignoring all test scores when everyone else is celebrating their high scores.

Or teaching less curriculum when everyone else is teaching more.

If you want to stand out from the teaching crowd, then you need to reclaim the true meaning of a word that has lost all meaning: LEARNING.

If you are a regular reader of my blog, then you know that real learning has no instruction manual. There are no step-by step instructions or prescriptive recipes.

Instead, it's about a conversation that goes something like this...

Students should experience their successes and failures not as reward and punishment but as information... measurable outcomes may be the least significant results of learning... we can't test our way to a better learning... there is a big difference between doing things to kids and working with them...

Real learning is a commitment to the unprecedented... a commitment to letting go of curriculum, grades and test scores.

The schools that stand out are the ones that understand this...