The Pitfall of PISA Envy

Whether it be business, education or life in general, it often makes sense to figure out what you want to do and what you do not want to do. It's also a good idea to figure out how you are going to assess your success.

This is good, but I would like to add one more step.

I think it's also important to decide how you will not assess what you've done. I think Maya Angelou provided us with a wonderful example of this when she said:

Life is not measured by the number of breaths you take but by the moments that take your breath away.

In one sentence, Angelou helps us to see what we should be doing (living), while simultaneously showing us how we should and should not measure the quality of our lives.

For the last decade, Finland has been the model nation for education systems around the world. Finland should be applauded for resisting the urge to invest in the Global Education Reform Movement (GERM) and instead pursuing alternative policies. Perhaps most notably, standardized tests are almost completely absent from Finnish schools.

And yet, world attention has been focused on Finland mostly because of their high scores on PISA's standardized tests. 

See the contradiction?

PISA's 2012 rankings show Finland has been replaced at the top with a handful of Asian countries (and a city). By idolizing the rankings, people might drop Finland like a hot-potato to chase after Asian countries who achieve their high scores with very different priorities and questionable means.

Recognizing people or nations for doing the right thing for the wrong reasons can be misleading and ultimately unsustainable. PISA's rankings on their own are useless. The real lessons from PISA are found from researching how each nation achieved their results and then assessing their methods via ethical criteria that is independent of their results. (Things go very wrong when we allow education policy to be driven by circular logic: define effective nations as those who raise test scores, then use test score gains to determine effective nations.)

We need to recognize Finland for doing the right things with their schools for the right reasons, but that means we need to move beyond reducing learning to standardized test scores and PISA rankings. Until then, we run the risk of chasing high performing nations that score well and rank high with methods that are less than enviable.

Assessing the quality of education by how many questions we answer correctly is kind of like judging a life by the number of breaths we take -- both are clear, simple and wrong.