Transferring Teachers

Imagine this:

• You are a progressive and innovative teacher who engages students.

• You have worked hard to become very knowledgeable about how children learn and in order to meet all of their needs, you understand that you can't pretend that all students have the same needs.

• You have shifted away from believing that curriculum is something that is designed, laminated and mailed to the school by distant authorities to believing that curriculum is something that is co-constructed and negotiated between teachers and students in the classroom.

• Rather than doing things to students to get short-term compliance, you work with them to nurture long term engagement. 

• Your classroom is grounded in a culture of learning -- not a culture of testsandgrades.

• You understand that real learning can not be reduced to numbers, so you have substituted grades with informative feedback that allows the students to see what they've done well and what they can do to improve. Even though the most important things may not be quantifiable, they can always be observed and described.

• You understand that sixty years of research tells us that we don't internalize knowledge by simply being told to do so. Real learning is constructed from the inside while interacting with others, and that's why your students spend most of their class time on project-based-learning and performance assessments that are collected in learning portfolios.

• You understand that the real purpose of a Professional Learning Community should be about learning and not about standardization.

Now imagine this:

• You are approached by your grade-level or subject department head or a teaching colleague and they are concerned that you are not giving the same worksheets, textbook assignments, quizzes and multiple choice tests on the same days in the same manner as the rest of the school or school division.

• You are "asked" to meet with your administration and they are concerned that you are not a "team player" and that you are not "collaborating" with your Professional Learning Committee. 

• You explain in detail how and why you are doing things differently. You show how your teaching is grounded in evidence and research, and that you are getting results via student engagement.

• You are told by your principal that your teaching assignment is changing next year. Instead of teaching courses that match your expertise and passion, you are teaching something that you have little to no expertise or passion. 

• A year later, your administration observes that for some reason you appear unhappy so they suggest you transfer. When you hesitate at the "offer", your administration reminds you that you really don't have a say in a transfer.

Are there good and bad teachers? Yes. But let's not pretend we can agree on who is good and who is bad. There is no agreement on what constitutes good teaching.

Some people complain that it's nearly impossible to get rid of bad teachers, and there might be some truth to it -- but I find it sadly ironic that too often we treat good teachers so poorly that they leave. If we talked about making good teachers even half as much as we talk about firing bad ones, we would probably get somewhere.

Keep in mind that the tactics used in the example above by administration can be used on any teacher regardless of their quality. Transferring a teacher like this can have two goals:

1. This is an effective way of getting rid of lazy, incompetent teachers who refuse to engage competently and professionally with students, parents, colleagues and administrators (of course, it doesn't actually get rid of them, it just makes them someone else's problem) -- but it can also be an effective way of getting rid of outspoken teachers who refuse to mindlessly comply with every top-down, drive-by directive dispensed by distant authorities who are as geographically distanced from the classroom as they are pedagogically.

2. By making an example of these teachers, administration can keep other teachers compliant.

Sometimes transfers are a breath of fresh air that allows a teacher to re-energize. But sometimes the threat of a transfer can be an instrument of control between the powerful and powerless -- that's why sometimes teacher transfers have less to do with learning and teaching and more to do with compliance, punishment and power.

Innovation can be intimidating because it often involves ambiguity and change -- but ambiguity and change can often challenge competence which is why some of the most competent people can sometimes be the largest obstacles to change.

It takes courageous leadership to empower teachers to engage students in progressive and alternative ways, but that means we have to stop using the threat of a transfer over teachers to merely get them to do whatever we want. Changing and improving school will only happen if the people most responsible for implementing changes are actively engaged in the process.

Giving teacher's permission to make school different and better is not the same as telling teachers to just do what they are told, and you can't expect teachers to be innovative and progressive as long as the threat of a transfer looms over them like a guillotine.

The problem with getting teachers used to simply doing what they are told is that they might get used to doing only what they are told.

If teachers resign themselves to being nothing more than agents of the state for delivering top-down mandated, prefabricated, content-bloated, scripted curriculums then it makes sense to do whatever it takes to manipulate, bribe, threaten, bully, harass kids into doing whatever it is we want them to do. If this is our perspective, then as long as the kids do what we want, even begrudgingly, we consider compliance our mandate. And if this is how we want to treat children, then I guess it makes sense to treat teachers this way too.

But...

...if teachers see their responsibility as engaging every learner in a personalized journey in discovering and constructing their passion, we come to see authentic engagement as infinitely more important than compliance. And if this is how we want to treat children, then I guess it makes sense to treat teachers this way, too.

Ultimately the best teachers come to see school not as something done to kids, but something done by them and with them. Likewise, the best administrators and even policy-makers see school reform not as something done to teachers, but something done by them and with them.

Mr. Bower, I can't do my project anymore

I sometimes think about a conversation I had with one of my grade 8 students while we were learning about World War II and The Holocaust. It went something like this:

Reagan came up to me in the middle of class and said, "Mr. Bower, I can't do my project anymore."

I was a little taken aback. Reagan had been so eager to start her research on Dr. Mengele and initiated a majority of her project. At first glance such a pronouncement from a student might easily be labelled as defiant, but I was pretty sure there was something else going on here. So I asked, "Really, Reagan? Why's that? What's up?"

"At first I thought I wanted to learn more about what happened during The Holocaust, and then I started researching Dr. Mengele in that book you suggested, The Holocaust Chronicle. But now I'm just saddened by it all. It makes me so sad to read about the awful things that these people did to others. I just don't think I want to do this anymore. I don't want to be sad."

"That's fair. I know that sometimes I have a hard time reading books about The Holocaust. Sometimes it's hard to spend a lot of time focusing on such an uncomfortable topic." 

"Yeah. Totally."

"Before you quit your project, Reagan, I have a question for you. Would it be worse if some people like you and me got a little sad from spending time learning about The Holocaust or would it be worse if we avoided being sad and just forgot about The Holocaust?"

Reagan stood there looking at me.

She didn't say a word.

I knew she was thinking.

In short order she went from looking perplexed to certain. She said, "It would be way worse if we forgot."

"Why is that?"

"Because if we forget, we might avoid being sad, but we would risk allowing it to happen again. And we can't do that." She turned and went back to her project. 

I was proud of Reagan.

***

A couple thoughts:

• Can you see how asking Reagan about what was up was more productive than just assuming she was being lazy or defiant? People like it when you seek to understand them before you seek to be understood -- and children are people, too.

• Can you see how lecturing Reagan about why we need to learn about The Holocaust would have missed the point? Can you see how asking provocative questions that inspire thought are the real work of teachers? 

• Can you see how a text-book, computer software or app can't do this? Reagan needed "just-in-time" feedback and guidance that only a real life teacher that she has a relationship with could provide.

• Can you see how this conversation with Reagan would be very difficult to quantify or symbolize on the report card and yet witnessing her new-found realization is what might matter most? The most important things that happen in school may be difficult, if not impossible, to measure but they can always be observed and described -- this is why assessment is not a spreadsheet, it's a conversation. And when we try to reduce learning to a number, we always conceal far more than we ever reveal.

Rethinking Awards Ceremonies and Honour Rolls

St. Basil Elementary and Junior High School in Calgary has decided to end their year-end awards ceremony. Unfortunately, when stories about schools that are trying to make changes like this hit the media, the public can have a hard time suspending judgement long enough to find out what is really going on. (If you need proof of this, take a moment and read some of the explosive comments here.)

If it sounds like I'm in favour of the move away from awards ceremonies that's because I am. In 2007, I worked with my middle school to move away from awards for a few to recognizing all students.

While it may seem counterintuitive to suggest that we should abolish awards ceremonies and honour rolls, this is precisely what the research has been showing us. Alfie Kohn reminds us that "There is a world of difference between getting kids to focus on their performance or achievement and getting them to focus on their learning."

Some people have a hard time grasping the rationale behind removing awards ceremonies, so let's take a moment and address some of the most common myths about moving away from awards.

Myth 1: Awards ceremonies and grades serve a purpose: they act as incentive and motivation for children to learn.

Many people believe that we need to bribe and reward students with grades and honour rolls to motivate them, in my book De-Testing and De-Grading Schools, I write about why this is folly:

Conventional wisdom tells us we grade students to artificially induce their extrinsic motivation to strive for the reward of a high grade, or to avoid the punishment of a low grade. Either way, it is the carrot or the stick that is the driving force. 

The problem here is that we need to stop asking ‘How motivated are my students?’ and start asking ‘How are my students motivated?’. Motivation is not a single entity that you either have a lot or little of. 

There are two kinds: intrinsic and extrinsic. If you are intrinsically motivated then you are doing something for its own sake; if you are extrinsically motivated, you are driven to do something, or not do something, based on a reward or punishment that may be waiting for you. But that is not even the interesting part—the real catch here is that these two kinds of motivation tend to be inversely related. 

When you grow students' extrinsic motivation by bribing them with high grades or threatening them with low grades, you run the risk of growing their extrinsic motivation while their intrinsic love for what you want them to learn shrivels. Grades can only ever gain short-term compliance from students when what we really desire is their authentic engagement. If we give grades and our students are uninterested or disengaged, might it be because they are searching for a more intrinsically motivating reason to learn? I suppose we could use grades to artificially induce children to learn, but I would hope we could help them find a real reason to do so. Needless to say, playing on children’s extrinsic motivation to learn is, at best, a questionable practice.

The problem is not that too few students achieve As -- rather, the real problem is that too many students have come to see collecting As as the purpose of school. Things go very wrong when testsandgrades, awards ceremonies and honour rolls become the primary goal of education.

There is an enormous gap between what we know and what we do. Too often what we admire and aspire to does not align with our actions. If we really want children to become life-long learners then it makes little sense to distract them with artificial bribes and awards. 

Myth 2: Removing awards de-values academics

Replacing awards for a few with recognition for all is not about devaluing anything. In fact, it is about broadening our current narrow vision of what is important by valuing more than just grades in academics. Many honour rolls place an arbitrary premium on core academic subjects while placing less importance on classes that we call options (art, foods & fashion, wood working, etc.) This is about shifting away from valuing only student achievement (read: grades) and moving towards valuing a wide range of students' achievements.

For years, I organized and ran my school's awards ceremony and for years I found a disturbing trend: The same children who received accolades all year for their grades were being marched across the stage multiple times during the year-end ceremony. When you really think about it, grades are rewards and awards ceremonies are awards for getting rewards.

When we used to hand out traditional honour certificates to students, I would grow disheartened to find so many of the certificates left behind on the floor or even in the trash. Many students and parents did not value the certificates. However, when we handed out a personalized recognition poster that celebrated a wide range of students' achievements, including a personal message from their teacher, the students and parents showed far more appreciation. I never found any of these posters on the floor or in the trash. Sometimes I wonder if all students valued these posters because we chose to value all students in a personal way.

Myth 3: Removing awards celebrates mediocrity

What if an entire school's population made the honour roll? What would be the public's reaction? Would this be seen as good news? It's highly unlikely that the school would be given admiration and credit for this. It is far more likely that people would say that the school was being too easy on the kids. 

On the surface, it appears like awards ceremonies and honour rolls are about excellence -- but that's only if excellence is defined as raising standards until failures are created.

Honour rolls and awards ceremonies make success artificially scarce. The allure of winning an award or making the honour roll comes in large part because others will not be allowed to win or make it. If we aspire to making school a great school for all children, then we need to stop making success artificially scarce and exclusive.

When I ran my school's traditional awards ceremony, less than a third of the students were ever invited -- the other two-thirds were left out. When we moved to recognizing all students for their wide variety of successes and strengths, all students and their families were invited to the event. Some of the most thankful parents are those who have children who would never be invited to be recognized by their school's awards ceremonies -- in some cases that's three-quarters of the school.

Too often when schools get around to recognizing things like citizenship, these awards are artificially scarce which means that many students who exhibit good citizenship are arbitrarily ignored.

Why do we devalue something simply because everyone can achieve it? Do we wear dropout rates as a badge of honour? I would hope not. So why do we scoff at schools that celebrate every student? Recognizing every student is no more an exercise in mediocrity than believing all children should graduate from high school. It might be more accurate to say that mediocre schools give awards to only a few students -- great schools recognize all their students.

Myth 4: The real world is a cruel and unjust place where some people win and some people lose -- kids need to learn that the real world doesn't care about them.

I did not become a teacher so that I could merely prepare children to live in a cruel and unjust world. I became a teacher so that I could help children grow up and make the world a better place. Yes, the real world is full of competition that makes winners and losers out of everyone, but we don't need to immerse children in competition to learn this. When we teach children about racism, we don't immerse them in racism. I refuse to subscribe to the notion that because children will one day grow up and have bad things done to them that means we need to do bad things to them in school to get them ready for it.

The real world is not a fixed and known place -- the real world can be made and unmade by the people who live there. Too often, the real world is an excuse to do nothing about the inequities that rob children of their futures. I think there is some truth to what Sydney Harris said, "Anyone who begins to call himself a realist is prepared to do something he is secretly ashamed of doing." I refuse to live in a world that doesn't care about how children feel. And if the real world is that cruel and unjust then I'm going to fight like mad to make the real world a better place and I'm going to take my students with me.

***

Changing schools so that we promote high achievement among all students is not easy. Sometimes the greatest obstacle to changing and improving school is our memory. When some people say they understand what school should look like, they are really saying that they recognize what school was like for them. Too many people are reassured by signs of formal-traditional school and are disturbed by their absence.

That's why when we hear schools trying to make changes to what they've always done, we need to suspend judgement long enough to find out what is really going on, and in the case of abolishing honour rolls and awards ceremonies, the research and rationale are (maybe surprisingly) sound.

For more on rethinking awards ceremonies, I suggest you check out Chris Wejr's blog.

Here's my interview with Doug Dirks on The Homestretch:



Here's my interview on the Simi Sara radio show:



Here's my interview with Jim Brown on CBC radio The 180:

What do kids really learn from failure?

This was written by Alfie Kohn who writes and speaks on education and parenting. This post was first found here. His website is here and he tweets here.

by Alfie Kohn

Education experts have long known that there is more to success -- in school or in life -- than cognitive ability. That recognition got a big boost with science writer Dan Goleman's bookEmotional Intelligence in 1996, which emphasized the importance of self-awareness, altruism, personal motivation, empathy, and the ability to love and be loved.

But a funny thing has happened to the message since then. When you hear about the limits of IQ these days, it's usually in the context of a conservative narrative that emphasizes not altruism or empathy but something that sounds suspiciously like the Protestant work ethic. More than smarts, we're told, what kids need to succeed is old-fashioned grit and perseverance, self-discipline and will power. The goal is to make sure they'll be able to resist temptation, override their unconstructive impulses, and put off doing what they enjoy in order to grind through whatever they've been told to do. (I examined this issue in an earlier essay called "Why Self-Discipline is Overrated.")

Closely connected to this sensibility is the proposition that children benefit from plenty of bracing experiences with frustration and failure. Ostensibly this will motivate them to try even harder next time and prepare them for the rigors of the unforgiving Real World. However, it's also said that children don't get enough of these experiences because they're overprotected by well-meaning but clueless adults who hover too close and catch them every time they stumble.

This basic story, which has found favor with journalists as well as certain theorists and therapists, seems plausible on its face because some degree of failure is unavoidable and we obviously want our kids to be able to deal with it. On closer inspection, though, I think there are serious problems with both the descriptive and prescriptive claims we're being asked to accept.

Is failure rare? The idea that "kids today" have it too easy is part of a broader conservative worldview that's been around for a long, long time. Children are routinely described as coddled and indulged, overprotected and overpraised. But I've been unable to find any data to support this claim, which may explain why it rests mostly on provocative anecdotes. Even if we could agree on how much protection (or parenting) merits the prefixover-, there's simply no proof that the phenomenon is widespread, much less that it's more common today than it was 10, 20, 50, or 100 years ago.

Moreover, even if it were shown that some parents cushion their children more than you or I think they should, that doesn't mean these kids are unacquainted with frustration or failure. To see life through a child's eyes for even a short time is to realize that, quite apart from a parent's willingness to intervene, children frequently come up short, don't get what they want, and find themselves on the receiving end of critical judgments from their peers or adults.

Is failure useful? A hypothetical child who managed to succeed in every one of his endeavors, or who always got everything he desired, might well find it hard to cope if things suddenly turned sour. But are we entitled to conclude from this fanciful thought experiment that failure is beneficial, or that parents and teachers should deliberately stand back rather than help out?

Research certainly doesn't support the idea that failure or disappointment is constructive in itself. A "BGUTI" (better get used to it) rationale -- the assumption that children are best prepared for unpleasant experiences that may come later by being exposed to a lot of unpleasantness while they're young -- makes no sense from a psychological perspective. We may want kids to rebound from failure, but that doesn't mean it's usually going to happen -- or that the experience of failure makes that desired outcome more likely.

In fact, studies find that when kids fail, they tend to construct an image of themselves as incompetent and even helpless, which leads to more failure. (They also come to prefer easier tasks and lose interest in whatever they're doing.) In one study, students were asked to solve problems that were rigged to ensure failure. Then they were asked to solve problems that were clearly within their capabilities. What happened? Even the latter problems paralyzed them because a spiral of failure had been set into motion. By the same token, if an adult declines to step in and help when kids are frustrated, that doesn't make them more self-sufficient or self-confident: It mostly leaves them feeling less supported, less secure about their own worthiness, and more doubtful about the extent to which the parent or teacher really cares about them.

Have some people experienced failure but then gone on to be wildly successful? Obviously. But things don't work out this way for most people. And even when it does happen, we can't conclude that experience with failure was responsible for the success. (Also, we should be careful to define what we mean by "successful." One can end up rich or famous without being an admirable or psychologically healthy human being.)

What determines the impact of failure? Why do some people throw in the towel as soon as things get tough? Why do other people get back on the horse? (And why are so many of us unable to discuss these issues without resorting to stale metaphors?) To talk about grit and resilience is to focus on the attributes of individuals. But it may make more sense to look at the situations in which people find themselves and the nature of the tasks they're being asked to do.

Challenge -- which carries with it a risk of failure -- is a part of learning. That's not something we'd want to eliminate. But when students who are tripped up by challenges respond by tuning out, acting out, or dropping out, they sometimes do so not because of a deficiency in their makeup (lack of stick-to-itiveness) but because those challenges -- what they were asked to do -- aren't particularly engaging or relevant. Finger-wagging adults who exhort students to "do their best" sometimes don't offer a persuasive reason for why a given task should be done at all, let alone well. And if the rejoinder is that it doesn't matter if the assignment is just busywork because kids need to develop "good work habits" across the board, well, a reasonable person would wonder who stands to benefit when children are taught to work hard at anything that they're assigned to do by someone with more power.

A second explanation for students' not rebounding from failure at what they were asked to do is that they weren't really "asked" to do it -- they were told to do it: deprived of any say about the content or context of the curriculum. People of all ages are more likely to persevere when they have a chance to make decisions about things that affect them. Thus, the absence of choice might be a better explanation than a character defect for giving up.

And here's yet another possibility. Maybe the problem is that the educational environment emphasizes how well students are doing rather than what they're doing: It's all about achievement! performance! results! rigor! and not about the learning itself. Educational psychologists have found that when students are induced to think about grades and test scores -- particularly, though not exclusively, when the point is to do better than everyone else -- they will naturally attempt to avoid unnecessary risks. If the goal is to get an A, then it's rational to pick the easiest possible task. Giving up altogether just takes this response to its logical conclusion. "I'm no good at this, so why bother?" is not an unreasonable response when school is primarily about establishing how good you are.

Jerome Bruner said this: We want students to "experience success and failure not as reward and punishment but as information." That's a marvelous way to think about reframing unsuccessful experiences: My experiment, or my essay, didn't turn out the way I had hoped, and the reason that happened offers valuable clues for how I might take a different approach tomorrow. But this requires us (the adults) to do more than reframe or encourage. We have to address the structural factors that get in the way. For example, a student isn't going to view letter or number grades as informational feedback; they'll be seen as rewards and punishments, in part because that's exactly what they're intended to be.

The problem isn't with kids' attitudes or motivation as much as it is with our practices and policies. Yet potential problems with the latter are typically ignored by people who tell kids to grit their teeth, pull up their socks, and try, try again. Worse, these people may explicitly endorse those problematic practices or even call for more rigorous or competitive grading and testing. Some researchers use them to define success and failure -- with high grades or test scores uncritically accepted as a positive outcome for measuring the effects of grit or perseverance.

Indeed, many people oppose even mild attempts to make the whole grading experience less debilitating, such as eliminating zeroes for individual assignments (given that zeroes, when averaged in with other marks, can drag down a child's overall grade disproportionately). Not long ago, a Canadian teacher became a conservative folk hero for defying his district's no-zero policy. He insisted on his prerogative to punish students by giving them the lowest possible grade.

Those who came to his defense invoked the familiar rhetoric of accountability, high standards, and the need to prepare kids for the real world. But ponder the irony! Many students whom a teacher brands with zeroes already see themselves as failures. They're likely to experience his insistence that they be "held accountable" as yet another dose of humiliation and punishment. (And it's the students' perception, not the teacher's intention, that determines the result.) The idea that another goose egg will snap them out of their cycle of failure and put them on the road to success is, to put it gently, naïve. (On the other hand, some people's get-tough response is actually more moralistic than practical. The point may not have been to produce a better outcome for students at all but to make sure they don't "get away with" something. If you do something bad, something bad must be done to you -- regardless of the effect.)

In short, there's reason to doubt the popular claim that kids have too little experience with failure. Or that more such experience would be good for them. What is clear is that the very environments that play up the importance of doing well make it even less likely that doing poorly will have any beneficial effect.

A Q&A on computer grading

I've written about robo-graders here. 

Here's a 4 question Q & A where I'll ask the questions and the answers come from a must-read post titled Computer Grading Will Destroy our Schools.

1. Why is there a push to have student's multiple choice tests and writing scored by a computer?

The reason for the push is both grim and obvious: money. Now that our schools are going to have more standardized tests, there are going to be more student paragraphs that need grading. Grading written work is laborious and time-consuming and, from a school board’s point of view, expensive. What computers offer is the ability to do this task faster and – once they are up and running – more cheaply. To be fair to the school boards, assuming the computer programs work, doing this task more efficiently could yield some benefits. It would lift a significant burden from the harried, overworked and underappreciated group of teachers and grad students we currently pay to grade standardized tests. But I suspect that, for most people, the thought of a computer “reading” essays is reflexively anxiety-provoking. It brings out the inner Luddite. Are we really supposed to believe that a machine can do just as good a job as a human being at a task like reading?

2. Does computer grading eliminate teacher bias?

The way supervised machine learning basically works is this: The computer treats the student’s essay as though it were just some random assemblage of words. Indeed, the jargon term for the main analytical technique here is actually “Bag of Words” (the resonance of which is simultaneously kind of insulting and weirdly reassuring). The program then measures and counts some things about the words that the programmer thinks are likely to be correlated with good writing. For example, how long is the average word? How many words are in the average sentence? How accurate are the quotations from the source text, if any? Did the writer remember to put a punctuation mark at the end of every sentence? How long is the essay? 

The programmer then “trains” the computer by telling it the grades assigned by human graders to a “training set” of essays. The computer compares – mathematically – the various things it has measured to the grades assigned to the essays in the training set. “Johnny’s essay had an average word length of 5 letters and an average sentence length of 20 words. The human told me that Johnny gets a B+. Now I know that much about essays that get a B+.” From there, given an average word length, sentence length, word frequency and so on, the computer is able to calculate the probability that a given student essay would receive a particular grade. When it encounters a new essay, it can take the things it knows how to measure, and – based on what it learned from the grades assigned to the training essays – simply assign the most probable grade. 

It turns out that this works surprisingly well. Shockingly well. What we might think of as totally surface-level or accidental features – like having more words per sentence – are actually correlated very strongly with earning better grades. Statistical analyses, at least, tell us that machine-learning techniques perform just as well as human beings – that is, their grades for new essays are the same or similar a huge majority of the time. Plus there are some ways in which the computers might actually be better. The people who presently do essay grading for tests like the AP English exam work long hours and have to grade essay after essay. Unlike computers, they get tired and irritable and bored. Also unlike computers, they come pre-equipped with a whole bunch of biases. These little nuisances and irritations, we might hope, will wash away if we instead let a computer sort unfeelingly through the bag of words. 

But, alas, it isn’t so. Since the process by which the computer “learns” is anchored to grades assigned by human beings – the training set teaches the computer what kinds of grades we tend to give to what kinds of essays – the tiresome, unsexy little things that make us imperfect are built right into the system. For instance, if the graders who grade the training set tend to strongly penalize nonstandard uses of English – including nonstandard uses more common among racial minorities – so too will the machine. The computer will operationalize, and then perpetually reinstate, the botherations and biases we feed it. The best strategy, thus, will be to use mechanized, look-alike writing, which will be tautologically defined as good writing because it is associated with receipt of a good grade.

 3. Can students "game" these computer graders?

One obvious problem is that if you know what the machine is measuring, it is easy to trick it. You can feed in an “essay” that it is actually a bag of words (or very nearly so), and if those words are SAT-vocab-builders arranged in long sentences with punctuation marks at the end, the computer will give you a good grade. The standard automated-essay-scoring-industry response to this criticism is that anyone smart enough to figure out how to trick the algorithm probably deserves a good grade anyway. But this reply smacks of disingenuity, since it’s obvious that the grade doesn’t reflect what it’s “supposed to” – namely, the ability to write a reasonably high-quality essay on some more or less arbitrary topic. 

Another slightly less obvious problem is that, since the computer is just measuring and counting, it can’t actually give you meaningful feedback or criticism. It has no idea what big-picture themes you were exploring, what your tone was, or even what you actually said. It just tries to approximate the score you should get – that is, it tries to put you into a little box. While this kind of box-sorting is fine for literal grading, it doesn’t really help with teaching you to be a better writer. 

There are other problems, too. A former professor at MIT named Les Perelman has pointed out that the way the automated-essay-grading companies are analyzing their software’s performance is unfairly biased toward the machine. Perelman’s paper, although eye-glazingly dense in data analysis, notes that while a human grader’s reliability is checked by comparing his or her grades to someone else’s, the machine’s reliability is checked against a resolved grade, which reflects the judgments of multiple human readers. But the standard statistical measure of agreement, called Cohen’s Kappa, is – as Perelman puts it – “meant to compare the scores of two autonomous readers, not a reader score and an artificially resolved score.”

 4. What's the worst that can happen with computer graders?

Our culture will stop engaging with students on those very aspects of the humanities that make them worth studying in the first place. We are going to end up with a system that dispenses rewards in a way that is indifferent to – and divorced from – the most alluring parts of the humanities, those creative capacities that they let us engage. If our instruction in the humanities necessitates ignoring these abilities, then it is my opinion that there no longer is much point to teaching the humanities at all, and we should end the charade. In other words, if this kind of mechanized, standardized-test-friendly drivel is all we can offer our children as “the humanities,” then who cares about the humanities? 

Once the use of automated essay grading becomes common knowledge, the implicit message will be hard to miss. For any self-aware, warm-blooded American teenager, the conclusion will be all but inescapable: Nobody cares what you have to say. It could be brilliant and moving; it could be word-salad or utter balderdash; it really doesn’t matter. Content, feeling, creativity, thematic depth – none of it matters. Today’s students will recognize this; they will react to it; and it will inform who they grow up to be. Indeed, I confess that if I were a teenager, my response would be the same as theirs – the selfsame response that we tend to associate with (and dismiss as just) teen angst. What is the point, after all, in being rewarded by a system that doesn’t care who you are? If no one is going to read the essays, we might as well rip them up.