Bubble sheet season

As the end of the year madly approaches, the gymnasium is full of desks where the students sit and fill in their bubbles. A pin could drop and make infinitely more noise than the amount of learning that is (not) occurring.

The multiple choice tests range from 50 to 75 questions.

The kids are told they MUST stay for an hour.

On average, most are done ripping through the bubbles in about 20 minutes.

They sit there in sheer boredom waiting to be dismissed.

The hour is up and it's a mad dash to the door. All but a small handful of students are thankful just to get the hell-out-of there. When the gym fully clears out, the teacher makes their way to the bubble sheet scorer.

The computer crunches the data and spews out something like this:

What do you see? What do you know now that you didn't know before?

This kind of data will only lead teachers to better predict a kid's chances of passing or failing a test  than actually knowing the kid as a human being - as a learner.

There's a reason why Gerald Bracey once said:

There is a growing technology of testing that permits us now to do in nanoseconds things that we shouldn't be doing at all.

Teachers know how busy their day is. We have very little time to piss away. Every nanosecond we spend making, collecting and analyzing this kind of data is nanoseconds we are not actually engaging with kids in a meaningful relationship.

But let's not kid ourselves - those nanoseconds add up, and for many teachers, their day has been annexed by the data mongers.

Too many teachers have become agents of data and slaves to accountability. As if a good education simply comes from a teacher who counts the data so they can be held accountable.

Teachers are living in fear that someone might challenge their grading. When that parent, student or administrator comes knocking at your classroom door, you had better have a diabolical data managing system that clearly illustrates how you could have possibly given lil' Johnny that C. Teachers have resorted  to believing that you can point to your grade book and say "look at all those Cs, how could I have given him anything but a C."

When assessment becomes more about covering your own ass in fear of being held accountable and less about student learning, we fail our children in more ways than we would like to ever admit.

Feedback for Mr. Bower

The following post is for my students to provide me feedback for the final time.

Please provide me a thoughtful response to the following statement:

Mr. Bower was...

You can't construct meaning in a preconceived bubble

When I share with others the language arts and science performance assessments I do with students instead of a multiple choice final exams, I often get asked these kinds of questions:

1. How do you determine from these responses if they are comprehending well enough to understand the level of reading required for the next grade?

2. If they are experiencing difficulties in comprehension, how is this shown in your assessment project and what do you do to remediate this?

3. How do you translate this into constructive feedback for students and parents?

These are great questions. Here are my answers:

1. How do I assess if a student is comprehending? Let's be careful not to pretend that we actually know what it means to be reading at a grade 8 level versus a grade 9 level. Psychometricians might like to believe they can empirically diagnose a student's reading level down to a decimal point, but I would wager that most teachers understand the pitfalls of claiming such arrogance.

At best, multiple choice tests offer us spurious precision measuring a child's ability to decode text. Because reading is first and foremost about constructing meaning, the comfort we might gain from a multiple chioce test's pseudo structure must be seen for what it is: a subjective rating masquerading as an objective assessment.

Grades, stickers, happy faces, check marks, gold stars and other bribes that are born out of reductionist assessments such as multiple choice tests are not a kind of feedback to be accommodated; rather, they are problems to be solved and practices to be avoided.

Rather than concerning myself with "are they ready for the next grade" which is an infinitely ambiguous question, I concern myself more with questions like:

Do students enjoy reading?
Do they read willingly?
Do they construct meaning from reading?

How do I know if a student is constructing meaning? I observe it. Here's a sample of Liam's project. On the left is a poem he selected, and on the right are his thoughts. Click on the picture below to view it larger.

Look at his response! How could I ever have created a multiple choice question that would allow Liam to share with me his connection between a line of fan-fiction poetry from The Legend of Zelda to a Canadian Broadcasting Corporation's radio talk-show that features a 20 minute discussion on the concept of time?

Some might say this is too much work, and they don't have time to look at a student's work like this. To this I say don't be intimidated. The bulk of my assessing occurs while students are actually doing this.

How do I know he's ready for the next grade? I didn't need to number crunch or analyze data, I simply needed to observe him do this.

As I write this blog post, Liam is sitting next to me. I think he just summarized all this nicely:

You can't construct meaning in a preconceived bubble.

2. How do I know if a student is having difficulties comprehending? In three words, here is my answer: I observe it. There is no substitute for what a teacher can see with their own eyes when observing and interacting with students while they are learning.

Here's a sample from Lewis's project. About 2 minutes into this project, I saw him staring at his monitor like a deer in headlights. I quickly assessed that he was having difficulty. I walked over to him and asked if he needed help. He said yes. Below is a sample from his project. He asked me to provide an excerpt for him to read (its on the left). His thoughts are on the right. Click it to view it larger.

He was intimidated by words he couldn't understand - I know this because he told me. I told him to launch http://www.dictionary.com/ so he could look up these words. Then he showed his thinking by sharing with me that he had to look these words up. I can see that even after using the dictionary his dependence on copying the definition tells me he still isn't really getting it.

But what's the alternative? If I give him a multiple choice exam, he would stare at it, dejected, thinking he's an idiot who can't read. To save his own pride, he would likely remove all effort, madly filling in the bubbles only to declare this as "stupid".

A multiple choice test for Lewis would, at best, show me what he can't do. This kind of project allows Lewis the opportunity to show me what he can do. Designing and implementing remediation with this kind of project is intuitive. In contrast, I have no idea how I would remediate for Lewis if I was given his bubble sheet item analysis.

Quite frankly, Lewis and I would both be at a loss if given such data.

3. The best feedback parents can receive about their child's learning is for them to see their child's learning. Back in the old days, show-and-tell, science fairs and barn dances were exhibitions of learning. Communities came together to observe and listen to students while they performed their learning - and if things went really well, parents and community members might have actually interacted with the students. No one needed to translate the results - everyone could see with their own eyes that learning was or was not taking place. However, if we throw a number or letter grade at parents, or God-forbid try to show them an item analysis sheet like the one above, no wonder they need someone to translate the assessment. Summative assessment does not need to be diablolically complicated: gather information and share that information.You might put a hint of an evaluation, if needed, but otherwise, that's it. We don't need a stanine, quartiles, class averages or even grades and tests. Just gather and share. Rinse and repeat.

An understandable rebuttal to this might be that teachers, parents and community members might not be willing to engage in this - they might not have the time or be willing to expend the effort. Too often, this is as sad as it is true. In an attempt to counter this apathy, technology might provide a kind of solution; on-line communities such as discussion forums, blogs, moodles, nings and other social networking software can alleviate problems brought on by the limitations of time and place.

IN CLOSING, I want to address one more criticism I face with my alternatives to language arts and science multiple choice exams. I have had teachers ask me if I will be grading these projects - of course, the question is a setup - what they are implying is that I am cheating my students because I won't be going through their projects with a fine tooth comb, and that I am doing a disservice to them by not spending as many or more hours grading them as they did actually performing.

To this I say, how long does it take to rip a bubble sheet through a scoring machine?

The difference between a multiple choice exam and a performance assessment is not that the multiple choice exam can be counted and the performance can't. When it comes to quantifying either one, their respective strengths become their own weaknesses. The utility of a multiple choice exam and its artificially convenient interrator reliability comes at an alarming price;  authenticity is sacrificed for (perceived) reliability. Meanwhile, the genuineness of a performance assessment can be rich with both quality and quantity, but it's validity comes at a cost; quantifiability may be sacrificed for authenticity.

It might be argued that neither can be properly quantified, but if Linda McNeil from Rice University is correct in saying that measurable outcomes may be the least significant results of learning, maybe we shouldn't be too bothered by it all.

Could you imagine a school district of teachers coming together on a district inservice day where they could sit down and share a wide range of performance assessments? Just imagine the kind of rich dialogue that would revolve around real learning. Now that would be professional development!

The Demise of Social Norms

I have written about how school has shifted from a social norms focus to a market norms focus in a previous post here, and today I wish to look at whether it is possible for us to return to a learning environment built on social norms.

In his book Predictably Irrational, Dan Ariely writes about the dangers of mixing our social and market exchanges:

So we live in two worlds: one characterized by social exchanges and the other characterized by market exchanges. And we apply different norms to these two kinds of relationships. Moreover, introducing market norms into social exchanges, as we have seen, violates the social norms and hurts the relationships. Once this type of mistake has been committed, recovering a social relationship is difficult. Once you've offered to pay for the delightful Thanksgiving dinner, your mother-in-law will remember the incident for years to come. And if you've ever offered a potential romantic partner the chance to cut to the chase, split the cost of the courting process, and simply go to bed, the odds are that you will have wrecked the romance forever.


My good friends Uri Gneezy ( a professor at the University at San Diego) and Aldo Rustichini (a professor at the University of Minnesota) provided a very clever test of the long-term effects of a switch from social to market norms.


A few years ago, they studied a day care center in Israel to determine whether imposing a fine on parents who arrived late to pick up their children was a useful deterrent. Uri and Aldo concluded that the fine didn't work well, and in fact it had long-term negative effects. Why? Before the fine was introduced, the teachers and parents had a social contract, with social norms about being late. Thus, if parents were late - as they occasionally were - they felt guilty about it - and their guilt compelled them to be more prompt in picking up their kids in the future. (In Israel, guilt seems to be an effective way to get compliance.) But once the fine was imposed, the day care center had inadvertently replaced the social norms with market norms. Now that the parents were paying for their tardiness, they interpreted the situation in terms of market norms. In other words, since they were being fined, they could decide for themselves whether to be late or not, and they frequently chose to be late. Needless to say, this was not what the day care center intended.


But the real story only started here. The most interesting part occurred a few weeks later, when the day care center removed the fine. Now the center was back to the social norm. Would the parents also return to the social norm? Would their guilt return as well?


Not at all.


Once the fine was removed, the behavior of the parents didn't change. They continued to pick up their kids late. In fact, when the fine was removed, there was a slight increase in the number of tardy pickups (after all, both the social norms and the fine had been removed.)


This experiment illustrates an unfortunate fact: when a social norm collides with a market norm, the social norm goes away for a long time. In other words, social relationships are not easy to reestablish. Once the bloom is off the rose -once a social norm is trumped by a market norm - it will rarely return.

When school chose to adopt grading, we inadvertently replaced a social norm with a market norm. Learning became, and remains, a business transaction between teacher and student. No wonder some teachers fear that the removal of grading will precipitate apathy in students (as if there isn't already). No wonder some students, who've been convinced the point of school is to collect high grades, have a hard time motivating themselves to learn in the absence of payment (grades).



Dan Ariely strikes with clarity:

You can't treat your customers like family one moment and then treat them impersonally - or, even worse, as a nuisance or a competitor - a moment later when this becomes more convenient or profitable. This is not how social relationships work. If you want a social relationship, go for it, but remember that you have to maintain it under all circumstances...


Today companies see an advantage in creating a social exchange. After all, in today's market we're the makers of intangibles. Creativity counts more than industrial machines. The partition between work and leisure has likewise blurred...


In a market where employees' loyalty to their employers is often wilting, social norms are one of the best ways to make workers loyal, as well as motivated...


Although some companies have been successful in creating social norms with their workers, the current obsession with short term profits, outsourcing and draconian cost cutting threatens to undermine it all.

Ariely often talks about business in his book, but he smartly connects this all to education:

My feeling so far is that standardized testing and performance-based salaries are likely to push education from social norms to market norms. The United States already spends more money per student than any other Western society. Would it be wise to add more money? The same consideration applies to testing: we are already testing very frequently, and more testing is unlikely to improve the quality of education.

I suspect that one answer lies in the realm of social norms. As we learned in our experiments, cash will take you only so far - social norms are the forces that can make a difference in the long run. Instead of focusing the attention of the teachers, parents, and kids on test scores, salaries and competition, it might be better to instill in all of us a sense of purpose, mission, and pride in education. To do this we certainly can't take the path of market norms.


The Beatles proclaimed some time ago that you "Can't Buy Me Love" and this also applies to the love of learning - you can't buy it; and if you try, you might chase it away.

Teachers are not bad people. I can look anyone in the eye and safely pronounce that I believe teachers have the best of intentions. However, someone once said that the road to hell is paved with good intentions, and because grades can only ever be experienced by a student (of any age) as a reward and punishment, our best intentions are immediately trumped by the students' perceptions.

While it cannot be denied that grading has caused incalculable damage, I believe the damage is not irreversible. This is not to say that it will be easy - simply abolishing grading will not be enough to have students dance their way to learning how to divide fractions or sing their way to understanding comma splices, but it is the necessary first step in repairing the student-teacher relationship.


After abolishing grading five years ago, I radically altered my interactions with students. Every year, I work to replace the traditional, teacher-student market relationship with an authentic relationship that better reflects the ethos of social norms.

LA Final Project: Feedback for Mr. Bower

This blog post is for my students to provide me feedback about our language arts final project. Feel free to browse their responses - I found them both enlightening and entertaining.

Three questions:

1. Which better shows your reading, writing and thinking skills: a multiple choice exam or Show Your Thinking Project?

2. Which do you enjoy doing more: a multiple choice exam or Show Your Thinking Project?

3. Which is easier: a multiple choice exam or Show Your Thinking Project? Please explain.

The Relationship Between Accountability and Creativity

My guest today is David Wees, teacher and technology guru at Stratford Hall in British Columbia. David is also an important part of my Professional Learning Community, and I am pleased that he agreed to guest blog here today. You can check out his blog here and his twitter here.

By David Wees

Imagine this graph represents the possible relationships between accountability and creativity.

Where would you put the activities you do as a school?  Here are some examples of activities some school do, and where I think they lie on the accountability vs creativity scale.

What you may notice about this graph is that, for the most part, activities which hold schools and students highly accountability are not associated generally with creativity and that activities which are highly creative can fall short of being very accountable.  It's not a perfect graph, and I think that some of the examples could be moved, but the idea I think is pretty clear: the more you increase accountability, the less flexible the activity, and hence the less ability for students to be creative while completing the activity.

Accountability in this sense means how the activity and the student's performance of that activity, is shared with the student, the teachers, the school, and the wider community.  Standardized tests are considered a "highly accountability" activity simply because everyone has access to how well pretty much any school did, and educators within those schools generally have access to their individual marks, and of course students get feedback about how well they did.

Creative activities to me are generally areas where the student has a lot of choice on how the activity will be completed, and how they will complete the activity.  These are often the types of activities that I think students will actually be able to do once they finish their education, and according to Sir Ken Robinson, our schools fail to provide opportunities to students to do them.

There are a few activities which fall with higher accountability and decent ability for students to be creative, and we often find that these activities are not ones which are done by most schools.  Anyway, I'm sure the model I have up there is imperfect, so I invite you to follow this link to this collaborative Google drawing I've started, and we can add other activities to this chart.

Grading Goslings

In his book Predictably Irrational, Dan Ariely writes about the perils of first impressions:

A few decades ago, the naturalist Konrad Lorenz discovered that goslings, upon breaking out of their eggs, become attached to the first moving object they encounter (this is generally their mother). Lorenz knew this because in one experiment he became the first thing they saw, and they followed him loyally from then on through adolescence. With that, Lorenz demonstrated not only that goslings make initial decisions based on what's available in their environment, but that they stick with a decision once it has been made. Lorenz called this natural phenomenon imprinting.


Is the human brain, then, wired like that of a gosling? Do our first impressions and decisions become imprinted? And if so, how does this imprinting play out in our lives? When we encounter a new product, for instance, do we accept the first price that comes before our eyes? And more importantly, does that price (which in academic lingo we call an anchor) have a long-term effect on our willingness to pay for the product from then on?

While it is true that grading is a relatively new invention in human learning, it is pretty safe to say that whether we are the teacher or the student, grading has become an anchor for us, and that anchor brings with it long-term effects on our willingness to even imagine an education system without grading.

As goslings, we hatched from our schooling shells, and the first moving object we saw was our grade point average - and through out our schooling adolescence we've been following loyally.

But it's time we grow up.

While this imprinting may be natural, grading is not. As Dan Ariely puts it, we are being predictably irrational in our loyalty to grading. For too long we have married ourselves to this misguided anchor. Deep down, we may know that reducing real learning to a number or letter is fraudulent, but this gosling-like imprinting has us stubbornly locked to our initial experiences.

We may not be able to go back in time to alter our own learning environments that anchored us towards grading, but we can provide a more authentic environment for our children so that the first moving object they see is real learning.

Learning as a chore

Token economies and other rewards systems are prevalent in schools and families. The belief being that a kind of behavioural bait-and-switch can be used to encourage or reinforce students to achieve anything from reading,  to doing their times tables or cleaning their room.

In his book Predictably Irrational, Behavioral Economist Dan Ariely describes an experiment he conducted with his students. He started by reading a few lines from Walt Witman's Leaves of Grass:

After closing the book, I told the students that I would be conducting three readings from Walt Witman's Leaves of Grass that Friday evening; one short, one medium, and one long. Owing to limited space, I told them, I had decided to hold an auction to determine who could attend. I passed out sheets of paper so that they could bid for a space; but before they did so, I had a question to ask them.

I asked half the students to write down whether, hypothetically, they would be willing to pay me $10 for a 10 minute poetry recitation. I asked the other half to write down whether, hypothetically, they would be willing to listen to me recite poetry for ten minutes if I paid them $10.

This, of course, served as the anchor. Now I asked the students to bid for a spot at my poetry reading. Do you think the initial anchor influenced the ensuing bids?

Before I tell you, consider two things. First, my skills at reading poetry are not of the first order. So asking someone to pay me for 10 minutes of it could be considered a stretch. Second, even though I asked half of the students if they would pay me for the privilege of attending the recitation, they didn't have to bid that way. They could have turned the tables completely and demanded that I pay them.

And now to the results (drum roll, please). Those who answered the hypothetical question about paying me were indeed willing to pay me for the privilege. They offered, on average, to pay me about a dollar for the short poetry reading, about two dollars for the medium poetry reading, and a bit more than three dollars for the medium poetry reading, and a bit more than three dollars for the long poetry reading. (Maybe I could make a living outside academe after all.)

But what about those who were anchored to the though of being paid (rather than paying me)? As you might expect, they demanded payment: on average, they wanted $1.30 to listen the short poetry reading, $2.70 to listen to the medium poetry reading, and $4.80 to endure the long poetry reading.

With simple manipulation, Dan Ariely was able to arbitrarily make an ambiguous experience into a pleasurable or painful one. Without ever hearing Ariely's poetry reading skills (or lack there of), their first impressions were formed based on whether they were asked to pay or be paid.
Dan Ariely concludes:

The die was cast, and the anchor was set. Moreover, once the first decision had been made, other decisions followed in what seemed to be a logical and coherent manner. The students did not know whether listening to me recite poetry was a good or bad experience, but whatever their first decision was, they used it as input for their subsequent decisions and provided a coherent pattern of responses across the three poetry readings.

Reward systems that bribe kids to learn implant a dangerous anchor. What if kids come to see learning as a mere means to an end? What if they see learning as something to only engage in if the conditions are profitable?

To further make his point, Ariely quotes Mark Twain:

Work consists of whatever a body is obliged to do, and play consists of whatever a body is not obliged to do.

What if all of our extrinsic manipulators, whether they be exorbitant rewards or ghastly punishments, are herding kids to anchor learning in as an obligation - something they ideally would never need or want to engage in?

We need to think long and hard on bait-and-switch systems that frame learning as a chore.

We may be doing far greater harm than we could ever imagine.