Ravitch: Should Computers Grade Essays?

This was written by Diane Ravitch who is an advocate for progressive education reform who works tirelessly to call out education deformers and their corporate-style reforms that work to undermine teachers and public education. Diane blogs here and tweets here. This post was originally posted here.

by Diane Ravitch

Todd Farley is the scourge of standardized testing. His book, “Making the Grades,” is a shocking exposé of the industry. Todd spent nearly 15 years scoring tests, and he knows the tricks of the trade.

In this article, he skewers the latest testing craze: machine-scoring of essays.

Having demonstrated the fallibility of humans who score essays, Farley is no more impressed by computer scoring. As he puts it:

“…the study’s major finding states only that “the results demonstrated that overall, automated essay scoring was capable of producing scores similar to human scores for extended-response writing items.” A paragraph on p. 21 reiterates the same thing: “By and large, the scoring engines did a good [job] of replicating the mean scores for all of the data sets.” In other words, all this hoopla about a study Tom Vander Ark calls “groundbreaking” is based on a final conclusion saying only that automated essay scoring engines are able to spew out a number that “by and large” might be “similar” to what a bored, over-worked, under-paid, possibly-underqualified, temporarily-employed human scorer skimming through an essay every two minutes might also spew out. I ask you, has there ever been a lower bar?”

Farley quotes the promoters of automated scoring, who say that the machines are faster, cheaper and more consistent than humans. Also, they make money.

He concludes: “Maybe a technology that purports to be able to assess a piece of writing without having so much as the teensiest inkling as to what has been said is good enough for your country, your city, your school, or your child. I’ll tell you what though: Ain’t good enough for mine.”

One of the responses to Farley’s post came from Tom Vander Ark, who is a tech entrepreneur and a target of Farley’s post.

Vander Ark wrote: “The purpose of the study was to demonstrate that online essay scoring was as accurate as expert human graders and that proved to be the case across a diverse set of performance tasks. The reason that was important is that without online scoring, states would rely solely on inexpensive multiple choice tests. It is silly to suggest that scoring engines need to ‘understand,’ they just need to score at least as well as a trained expert grader and our study did just that.”

A reader of this blog saw this exchange on Huffington Post and sent me this comment:

“Diane–we use an automated essay scorer at my school, and I have seen coherent, well-thought out writing receive scores below proficient, while incoherent, illogical writing (with more and longer words, and a few other tricks that automated scorers like) receive high scores. The students who suffer the most are the highest level students, the verbally gifted writers who write with the goal of actually being understood, “silly” as that may be.”

“In fact, all standardized testing penalizes the brightest students–those who think outside the box. Standardized testing is the box.”

Ravitch: How Teachers Unions Lead the Way to Better Schools

Here's an excerpt from an interview with Diane Ravitch on How Teachers Unions Lead the Way to Better Schools:

What do you see as the role of teachers unions in preserving public education?

For many years, there has been an effort to diminish teachers unions and to blame them for all the problems of public education. I believe the reason, first of all, is that some people just hate unions. But there’s also a political reason that’s very specific. That is that if you silence the union, then there’s nobody at the table when the legislature or the governor wants to cut the budget, so they can hack away at will. That’s happening in states across the country. I was in Texas a few weeks ago, and there the legislature cut over $5 billion dollars from the education budget, but they did manage to squeeze out $500 million dollars for more testing. They have a weak union. They had no one at the table to say, “You can’t do this.” And no one cared what the teachers thought anyway.

Rethinking Thinking

Here is a collection of media that can be used to work with and engage students around the topic of rethinking thinking This is a part of a project focused on working with and engaging students in ways that help them make positive changes and/or informed choices in their lives.

I hope to add more content to this page. Please consider leaving a comment with your suggestions for more video, poetry, short stories, still images, quotes, music videos and lyrics, art and books.

Stanley Milgrim Shock Experiments (3 parts)

Rethinking School

Here is a collection of media that can be used to work with and engage students around the topic of rethinking school. This is a part of a project focused on working with and engaging students in ways that help them make positive changes and/or informed choices in their lives.

I hope to add more content to this page. Please consider leaving a comment with your suggestions for more video, poetry, short stories, still images, quotes, music videos and lyrics, art and books.















Born to Learn from Born to Learn on Vimeo.

Is this your classroom?

Why Standardized Testing Will Never Work

This was written by Andrew Campbell who teaches grades 4 and 5 in Brantford, Ontario, Canada. Andrew tweets here and blogs here. This post was originally posted here.

by Andrew Campbell

This is an image of Belgian artist René Magritte‘s famous painting “The Treachery of Images“, painted in 1928-29 when Magritte was 30 years old. I’m sure it would take several art history courses to discuss what it means so I won’t try, but I will share my opinion.

Magritte is quoted as saying “…it’s just a representation, is it not? So if I had written on my picture “This is a pipe,” I’d have been lying!”. Magritte is discussing the nature of representations. He’s saying that no matter how good a representation is, it can never be the thing that it represents. There is always something lost.

No matter how good a painting of a pipe is, it isn’t a pipe. You can’t stuff and smoke it. A musical recording isn’t the same as hearing a musician live. A sporting event watched on TV is not the same as seeing the game live. You can make the argument that they are, in some ways, better (“a painting of a pipe doesn’t stink up my house!”) but no one would ever say they are the same.

So it is with learning and testing. Learning is a live construction of understanding that teachers have a chance to facilitate and observe. We provide opportunities and support and hope it happens. Sometimes it does and sometimes it doesn’t. We give feedback and try to do what we can to improve on it, build on it, take it further. And we share our observations with the learners and with others because that improves the learning.

At some point (I’m not sure when), and for reasons I’m unclear, some people decided they did not trust teachers. When teachers said learning was happening these others said “Prove it!”. The results of trying to prove learning to those outside the process is standardized testing.

What the users and proponents and advocates of standardized testing fail to grasp is that test scores only represent learning, they cannot “be” learning. A student may score well on a test but that information may be lost the next day because it was not “learned”.

I “learned” all the molecular variations of the Krebs Cycle and could regurgitate them and get a high grade in Organic Chemistry, but that information was gone from my brain within a few days. I can still, however, vividly remember details from CS Lewis’ “The Lion, The Witch & The Wardrobe” as read to me by Mrs. Dickinson in class 6 at St Stephen’s CE School in Burnley, Lancs. I can remember where I was sitting in the room, the quality of the light and the sound of her voice. I can remember the images I created in my head of the mighty Aslan. This “deep learning” took years to build as I revisited it and built connection after connection to it.

Standardized testing will never accurately assess learning because learning doesn’t work that way. Some things I teach my students this year won’t ‘click’ until later, when they are ready for them or when their minds open to them. Learning’s a complex and complicated process and can’t be accurately reduced to numbers. At some point we have to trust the learners. As my grandmother Hannah Green often reminded me, “a watched pot never boils, love”.

The numbers can act as a loose guide to help with instruction. They can shine a light on certain areas and illuminate some parts and make them easier to see. They can hint at areas of weakness and help to guide instruction in the hands of a skilled teacher. But they are useless to someone sitting in an office, away from the messy learning, someone who is trying to figure out whether learning is occurring.

No matter how much they want it to be, this is not a pipe.

Brooklyn Castle