Assessment and measurement are not the same thing

Too many people confuse measurement with assessment as if they were the same thing.

They are not.

Some things are made to be measured. For example, I'm 6'1''. Height is a one-dimensional thing that can be reduced to a measurement in standard units. We need standard units for height or we would have all kinds of mass confusion.

Some things in life, however, are not made to be measured. While my height can be accurately described as 6'1'' without debate, my personality, character, intelligence, athleticism and learning can not be meaningfully reduced to a symbol. When we reduce something as magnificently messy as learning to a number, we always conceal far more than we ever reveal.

The most important things that children learn in school are not easily measured. The most meaningful things in life may, in fact, be immeasurable. The good news, however, is that the most important and meaningful things that we want children to learn and do in school can always be observed and described. This is precisely why it is so important to remember that the root word for assessment is assidere which literally means 'to sit beside.' Assessment is not a spreadsheet -- it's a conversation.

Testsandgrades should be replaced with projects and performances collected in portfolios.

When student learning is made visible to parents through portfolios, blogs, student-led conferences and parent-teacher interviews then they are not nearly so desperate for less meaningful information such as testsandgrades.

This is my 16th year of teaching in public schools. I threw my gradebook away in 2006. For those who are interested in learning more about what school and learning looks like without testsandgrades, you can read my chapter from my book for free here. And you can read all of my blog posts about abolishing grading here.

Without democracy, our economy will destroy public education

What is the purpose of school?

If we want to improve school, this is a question we need to ask more often.

When progressives propose that we move beyond traditional schooling, some people are skeptical and some people are cynical. 

Here's the difference:

The skeptic: "Don't we need testsandgrades to prepare children for the future?"

The cynic: "Don't we need testsandgrades to prepare children for the future!"

Sometimes the difference is subtle but these are two very different reactions. We need skeptics to ask questions and challenge change as much as the status quo. 

We don't need cynics. Cynics hold us all back and stifle progress because they can't differentiate between the (a) status quo, (b) change for the sake of change and (c) improvement. The good, the bad and the ugly are all the same to a cynic.

The other day I was having a conversation with someone who told me that he has 3 businesses, two of which are doing well, while the third was losing him money. He is concerned that progressive education that moves beyond testsandgrades won't produce employees that can produce him profits.

I fumbled through a response that left me thinking about how I could have better responded. Here's what I wish I had said:

Firstly, in school the process is the point and in work the product is the point. Children don't go to school to work -- they go to school to learn.

Ultimately, I didn't become a teacher so that I could create employees that can make other people money. I became a teacher so that I could inspire children to become democratic citizens who participate in our democracy in a way that makes our world better, which includes becoming ethical and profitable employees and employers in our economy.

The fate of our democracy is dependent on public education, and without democracy, our economy will destroy public education.

The problem with tests that are not standardized

This was written by Alfie Kohn who writes and speaks on parenting and education. Kohn tweets here and his website is here. This post was originally found here.

by Alfie Kohn

I’m baffled by the number of educators who are adamantly opposed to standardized testing yet raise no objection to other practices that share important features with such testing.

For starters, consider those lists of specific, prescriptive curriculum standards to which the tests are yoked. Here we find the same top-down control and one-size-fits-all mentality that animate standardized testing. Yet from the early days of the “accountability” movement right down to current efforts to impose the Gates-funded Common Core from coast to coast, an awful lot of people give the standards (and the whole idea of uniform standards) a pass while frowning only at the exams used to enforce them.[1]

Example #2: Elaborate rubrics used to judge students’ performance represent another form of standardized assessment that’s rarely recognized as such. The point is to break down something, such as a piece of writing, into its parts so that teachers, and sometimes the students themselves, can rate each of them, the premise being that it’s both possible and desirable for all readers to arrive at the same number for each criterion. Rubrics are borne of a demand to quantify and an impulse to simplify. One result, argues Maja Wilson, is that “the standardization of the rubric produces standardized writers.”[2] But, again, even many teachers who are outraged by standardized tests don’t blink when standardization is smuggled in through the back door. Some insist, against all evidence to the contrary, that there’s no problem as long as one uses a good rubric.

It’s my third example, though, on which I’d like to linger. When teachers test their students, the details of those tests will differ from one classroom to the next, which means these assessments by definition are not standardized and can’t be used to compare students across schools or states. But they’re still tests, and as a result they’re still limited and limiting.

As with rubrics (and grades), there’s a reflexive tendency to insist that we just need better tests, or that we ought to just modify the way they’re administered (for example, by allowing students to retake them). And, yes, it’s certainly true that some are worse than others. Multiple-choice tests are uniquely flawed as assessments for exactly the same reason that multiple-choice standardized tests are: They’re meant to trick students who understand the concepts into picking the wrong answer, and they don’t allow kids to generate, or even explain, their responses. Multiple-choice exams can be clever but, as test designer Roger Farr of Indiana University ultimately concluded, there is no way “to build a multiple choice question that allows students to show what they can do with what they know.”

We can also concede that some reasons for giving tests are more problematic than others. There’s a difference between using them to figure out who needs help — or, for more thoughtful teachers, what aspects of their own instruction may have been ineffective — and using them to compel students to pay attention and complete their assignments. In the latter case, a test is employed to pressure kids to do what they have little interest in doing. Rather than address possible deficiencies in one’s curriculum or pedagogy (say, the exclusion of students from any role in making decisions about what they’ll learn), one need only sound a warning about an upcoming test — or, in an even more blatant exercise of power, surprise students with a pop quiz — to elicit compliance.

Even allowing for variation in the design of the tests and the motives of the testers, however, the bottom line is that these instruments are typically more about measuring the number of facts that have been crammed into students’ short-term memories than they are about assessing understanding.[3] Tests, including those that involve essays, are part of a traditional model of instruction in which information is transmitted tostudents (by means of lectures and textbooks) so that it can be disgorged later on command. That’s why it’s so disconcerting to find teachers who are proud of their student-centered approach to instruction, who embrace active and interactive forms of learning, yet continue to rely on tests as the primary, or even sole, form of assessment in their classrooms.

While some of their questions may require problem-solving skills, tests, per se, are artificial pencil-and-paper exercises that measure how much students remember and how good they are at the discrete skill of taking tests. That’s how it’s possible for a student to be a talented thinker and yet score poorly. Most teachers can, without hesitation, name several such students in their classes when the exams are designed by Pearson or ETS, but may fail to see that the same thing applies in the case of performance on tests they design themselves.

Not only do tests assess the intellectual proficiencies that matter least, however — they also have the potential to alter students’ goals and the way they approach learning. The more you’re led to focus on what you’re going to have to know for a test, the less likely you are to plunge into a story or engage fully with the design of a project or experiment. And intellectual immersion can be all but smothered if those tests are given, or even talked about, frequently. Learning in order to pass a test is qualitatively different from learning for its own sake.[4]

***

Many years ago, the eminent University of Chicago educator Philip Jackson interviewed 50 teachers who had been identified as exceptional at their craft. Among his findings was a consistent lack of emphasis on testing, if not a deliberate decision to minimize the practice, on the part of these teachers.[5]

The first reason for this, I think, is that exemplary educators understand that tests are not a particularly useful form of assessment. Second, though, these teachers learned at some point that they didn’t need tests. The most impressive classrooms and curricula are designed to help the teacher know as much as possible about how students are making sense of things. When kids are engaged in meaningful, active learning — for example, designing extended, interdisciplinary projects — teachers who watch and listen as those projects are being planned and carried out have access to, and actively interpret, a continuous stream of information about what each student is able to do and where he or she requires help. It would be superfluous to give students a test after the learning is done. We might even say that the more a teacher is inclined to use a test to gauge student progress, the more that tells us something is wrong — perhaps with the extent of the teacher’s informal and informed observation, perhaps with the quality of the tasks, perhaps with the whole model of learning. If, for example, the teacher favors direct instruction, he or she probably won’t have much idea what’s going on in the students’ minds. That will lead naturally to the conclusion that a test is “necessary” to gauge how they’re doing.[6]

Assessment literally means to sit beside, and that’s just what our most thoughtful educators urge us to do. Yetta Goodman coined the compound noun “kidwatching” to describe reading with each child to gauge his or her proficiency. Marilyn Burns insists that one-on-one conversations tell us far more about students’ mathematical understanding than a test ever could — since all wrong answers aren’t alike. Of course this assumes that we’re really interested in kids’ understanding, not merely their level of phonemic awareness or ability to apply an algorithm. The less ambitious one’s educational goals, the more likely that a test will suffice — and that the wordstesting and assessing will be used interchangeably.

One can fill a bookshelf with accounts of other forms of authentic assessment: portfolios, culminating projects, performance assessments, and what the late Ted Sizer called “exhibitions of mastery”: opportunities for students to demonstrate their proficiency not by recalling facts on demand but by doing something: constructing and conducting (and explaining the results of) an experiment, creating a restaurant menu in a foreign language, turning a story into a play. In other words, when some form of evaluation is desired after, rather than during, the learning, tests stillaren’t necessary or even particularly helpful. They needn’t be used for “summative,” let alone for “formative,” assessment.

Many of us rail against standardized tests not only because of the harmful uses to which they’re put but because they’re imposed on us. It’s more unsettling to acknowledge that the tests we come up with ourselves can also be damaging. The good news is that far superior alternatives are available.

NOTES

1. See my essay “Beware of the Standards, Not Just the Tests,”Education Week, September 26, 2001. This phenomenon is even more pronounced in Canada. Its education system is completely decentralized; each province controls its own policies. Despite the considerable variation in the amount of testing from one to the next, however, all of the provinces have very specific grade-by-grade curricula that every teacher is expected to teach. Objections to this level of control, with the concomitant diminution of autonomy for teachers, are rarely heard — even in provinces where there is outspoken resistance to testing.

2. Maja Wilson, Rethinking Rubrics in Writing Assessment(Heinemann, 2006), p. 39.

3. A spate of recent studies that attracted considerable attention in the popular press argues that frequent tests (including self-tests) are more effective than other forms of studying. But the outcome measure in these studies is almost always limited to the number of facts that are correctly recalled on later tests. Rather than offering an argument in favor of conventional assessment, these experiments actually illuminate how words like “learning” and “achievement” — as used by researchers and journalists alike — often mean little more than the successful, and presumably temporary, process of memorizing facts. For a close look at one such study, see this essay.

4. I recently made this point — about how the anticipation of being tested can distract students from engaging with ideas — in a Twitter post that was retweeted more than 400 times. This degree of popularity led me to suspect I had been misunderstood. I followed up with a clarification that all tests have this effect, not just standardized tests. The retweet rate dropped off by 90 percent.


5. Philip W. Jackson, Life in Classrooms (Teachers College Press, 1968/1990).

6. Frank Smith once wrote, “A teacher who cannot tell without a test whether a student is learning should not be in the classroom.” I see what he means, but his formulation strikes me as a bit harsh. Teachers need help to learn how to assess without tests, and they need support and encouragement to eliminate a practice that is still used by most of their colleagues and widely expected by administrators, parents, and the students themselves. Moreover, the barrier to gauging how successfully students are learning often lies not with the teacher but with features of the school structure, such as classes that are too large or periods that are too short. That’s an argument for organizing to change these problematic policies, not for continuing to test.

Testsandgrades and teachers

Why some schools are giving letter grades a fail

This was written by Erin Millar who is a journalist and author with a lifelong interest in education, innovation and creativity. For nearly a decade she has written for leading Canadian and international publications including Reader’s Digest International, Maclean’s, The Globe and Mail, The Walrus, The Times of London and others. This piece first appeared here.

by Erin Millar

Krista Wolfram credits an innovative new assessment program with helping catch her daughter’s difficulties in school early. When Antonio Vendramin, principal of Georges Vanier Elementary school in Surrey, B.C., announced to parents in November that the school would no longer award letter grades, Ms. Wolfram was skeptical. “Some of us were scared of change,” Ms. Wolfram, whose daughter is in Grade 2, recalls. “I grew up with As, Bs and Cs.”

Instead of reporting to parents only two or three times a year, teachers began regularly communicating using an online student portfolio system called Fresh Grade. Ms. Wolfram quickly discovered that her daughter was having difficulty with her writing. “Every day her teacher would snap a photo of her journal or a video of her writing with her phone or iPad,” Ms. Wolfram explains. “I could see exactly where she was struggling, and I could work with my daughter and her teacher to help.”

Because Ms. Wolfram was able to intervene early, her daughter was writing at her grade level by the time she received her report card just before spring break. “If we had to wait until her report card to find out, she would have failed writing,” she says. “I wouldn’t have known she was struggling.”

The Surrey Board of Education pioneered a pilot program eliminating letter grades in several elementary schools in September and now more than 40 classes at 13 elementary and six secondary schools have joined the experiment. Nine more schools are set to join soon, and the results will be reviewed this summer.

The Surrey school district is not alone. Schools around the world are experimenting with new ways to assess student achievement that do not rely solely on high-stakes reports that use numerical marks and letter grades. The movement is in part a response to calls from employers for the school system to emphasize skills such as creativity and communication, not just knowledge of traditional subjects. But even as recent research suggests that descriptive feedback better supports students in developing these soft skills than traditional grades, parents, educators, and higher education institutions are struggling to adapt. If there are no grades, how we will know whether students are prepared for jobs or further study?

The idea that traditional measures of academic achievement don’t support learning isn’t new; in 1998, the British education researchers Paul Black and Dylan Wiliam published a widely-cited article demonstrating how increasing descriptive feedback raises student academic achievement. More recently, the Nobel Prize-winning economist James Heckman has published numerous studies showing how grades and standardized tests don’t predict later life success such as employment and income level.

In response, school boards in Canada are implementing new approaches to assessment. Elementary students in Ontario no longer receive letter grades on their fall report card. Quebec has reduced the number of formal reports to parents, opting for more descriptive feedback.

In June, the Calgary Board of Education announced a plan to eliminate letter grades up to Grade 9, a measure that was hotly debated. The new report cards would have used four phrases − exemplary, evident, emerging or support required − and not included written comments from teachers. Many parents were concerned about the proposal. Cathy Ward, a spokeswoman for the Calgary Board of Education, says that while some schools have begun transitioning to new ways of assessment, the board is reviewing the plan and no firm decisions will be made until they consult further with parents and teachers.

At Fraser Heights Secondary School in Surrey, English teacher Leah Samson no longer uses numerical marks to give students feedback on assignments. She devotes considerable class time to teaching students how to assess their own learning and give effective peer feedback. She also has regular one-on-one meetings with students to discuss their goals and progress. Instead of Ms. Samson telling students how they are doing, they are expected to articulate to her what they learned in class, how it relates to their learning goals and where they’re struggling.

Ms. Samson is still required to give students a letter grade at the end of each semester, but the letters have taken on a whole new meaning to her and her students. Struggling students have a much clearer idea of how they can move from a C to a B or an A. “Once grades are removed, students are learning for themselves rather than learning for their teacher.”

The move away from grades matches a growing belief among employers that traditional assessment is not the best way to help students develop the skills they need to succeed in today’s world. In national and global surveys, employers don’t complain about applicants lacking specific knowledge or technical skills, which are easy to test and express in a letter grade; they want employees who can analyze critically, collaborate, communicate, solve problems and think creatively. A 2012 McKinsey & Co. survey of 8,000 students, educators and employers in nine countries found that there was a gap between what educators thought students needed to succeed and what employers really wanted. “Education providers will say that all skills are important, whereas employers will place much clearer prioritization on soft skills – where the likes of team work and work ethic come out quite strongly,” Mona Mourshed, director of education at McKinsey, said.

Traditional assessment largely focuses on measuring students’ ability to regurgitate information and was designed for certification and accountability purposes rather than to support student learning, argues Maria Langworthy, chief research officer with Ontario education expert Michael Fullan’s research group. “Think about working in a knowledge-based economy where the sort of end products we produce are intellectual, things like software, design, social policy,” she explains “There’s no multiple-choice test that can capture the value and complexity of those sorts of products.”

Success -- it's not always what you see

Four years ago I wrote a post titled What leads to success? I took an old TEDtalk and explained how testsandgrades do not lead to success.

When I came across this graphic on Twitter via Greg Miller, I have a couple questions:

Students. Have you ever worked really hard and learned a lot about something and received a low grade? Have you ever slacked off and learned almost nothing but received a high grade?

Parents. Can you think of someone you went to school with, and you knew they were really smart, but they always received low grades? Can you think of someone who received really high grades but you knew they were a dolt and that they had, at best, a superficial understanding?

Teachers. Can you think of a student who you knew to be a critical and creative thinker but often scored low on standardized tests? Can you think of a student who you knew to be quite a shallow and superficial thinker, but often scored high?

When I ask these questions to students, parents and teachers, I often get a lot of head nodding. People seem to understand the point.

Testsandgrades are broken. 

They don't tell us what we think they tell to us and they distract us from learning.

For an authentic alternative to testsandgrades, check out my chapter Reduced to Numbers: from concealing to revealing learning.

Obvious to you. Amazing to others.

Technology has promise and peril.

I've written before about some of the perils, but today I want to write more about its promise (like I wrote here, here, here, here and here).

I see technology and social media as a way to share. It reminds me of the old Japanese proverb that says no one of us is as smart as all of us. I love how the Internet can be used to remove obstacles such as time and place so that we can learn from others that the limits of time and place might never allow.

No one ever changed the world by sharing their testsandgrades.

This is why I think handing things in only to the teacher is so 1996 -- today, students should be sharing what they learn with the world.

There is a lot wrong with standardized testing, but the solution is not merely better tests or tests on computers. Alberta has already announced a move away from Provincial Achievement Tests in grades 3, 6 and 9 and some schools are making attempts to move beyond grades and awards ceremonies.

In the past, post-secondary and employers may have been content with transcripts full of grades and resumes with awards certificates, but things are changing. The real world is (slowly) changing. The nature of transcripts and resumes is changing.

When we reduce something as magnificently messy as learning to grades and awards, we always conceal more than we reveal. Things go very wrong when testsandgrades and awards become the primary goal of education, not just the means for understanding it.

Good enough schools help kids fill their transcripts and resumes with testsandgrades, but truly great schools help all students make their learning visible by sharing it with their parents and maybe even the world. Project-based learning, performance assessments and portfolios shouldn't just supplement testsandgrades and traditional report cards -- they should replace them.

Whether it is social media or 3D printers, technology can play a wonderful role in making this happen.

If you want to learn more about how technology can be used to best support student learning, I suggest a blog post, an article and two books:

Assessment and Technology

Telling Time with a Broken Clock: The trouble with standardized testing

Invent to Learn by Gary Stager and Sylvia Martinez

The Children's Machine by Seymour Papert

When I grow up... I want to be like mommy!

Take a look at the picture. What do you think this little girl child drew?

I don't know about you, but it looks to me like this picture has a mom dancing on a stage around a pole while people watch and offer cash -- and the caption says "When Grow up... I want to be like mommy!"

This is either very funny or very tragic.

I came across this picture on-line some time ago and the story goes that the child handed this in to her teacher, and when the picture came home, the mother felt the need to clarify via a letter:

Dear Mrs. Jones,

I wish to clarify that I am not now, nor have I ever been, an exotic dancer.

I work at Home Depot and I told my daughter how hectic it was last week before the blizzard hit. I told her we sold out out every single shovel we had, and then I found one more in the back room, and that several people were fighting over who would get it. Her picture doesn't show me dancing around a pole. It's supposed to depict me selling the last snow shovel we had at Home Depot.

So where am I going with this?

Learning is messy. Real learning is magnificently messy. This is true when adults are learning and über true when children are learning. Anyone who has spent ten seconds with children knows they are wonderfully weird. I wasn't there when the mother saw this drawing, but her letter makes it clear that she could see how the daughter's drawing could send, at the very least, mixed messages about how she pays the bills. In fact, I bet the mother had to actually sit down beside her daughter and take the time to engage her in a two-way conversation in order to gain insight into her thinking behind the drawing.

The latin the root for 'assessment' is assidere which means "to sit beside". This is why the best teachers know that assessment is not a spreadsheet -- it's a conversation between the teacher and the student.

Here's my point:

Without taking the time and effort to actively engage the student in a conversation, we might all assume the mother is an exotic dancer. Without talking with the student, we might (wrongly) assume to know what is really going on.

Too many of us think we understand what a grade such as a B- or 67% means, but that's not really true -- we are just used to them. Grades are at best a primitive form of feedback that leads many of us to assume that we know what is going on.

Too many of us have come to depend on the conveniences of grading; after all, grading can make assessment suspiciously easy -- which can lead to some less than desirable consequences.

I tell my students all the time that when we assume, we make an ASS out of U and ME. When we skip personally interacting with children and assess them via reductionist tools such as testsandgrades, we sacrifice validity and reliability for efficiency. When we try and reduce learning to a grade, we conceal far more than we ever reveal.

There is no substitute for what a teacher or parent observes while working with students while they are still learning. And if teachers and parents want to know if students are learning, we have to actively engage children in conversations that take a lot of time and effort.

I stopped grading my students, but I assess them every single day.

For more on abolishing grading, check out these posts:

The case against grades

I want to abolish grading but where do I start?

Grading without Grading

My de-grading philosophy Q & A

And remember, friends don't let friends grade. Join the Grading Moratorium today!

Albert Einstein on school

Is standardized testing changing ME for the worse?

This was written by Bill Ferriter who teaches grade 6 language arts. He blogs here and tweets here. This post was first found here.

By Bill Ferriter

Blogger's Note: This was a tough post to write. It feels like a confession that I should probably just keep to myself -- but I gotta believe that other teachers of tested subjects are thinking the same thoughts as I am. While this isn't super polished, I hope it makes y'all think. More importantly, I hope y'all will still stand with me even after knowing how testing has changed who I am as both a person and a practitioner.

Bill

______________________

Regular Radical Readers know full well how I feel about the impact that standardized testing has had on education.

#notabigfan

#puttingitmildly

Having spent the past FOUR DAYS watching my students take multiple choice exam after multiple choice exam -- 10 HOURS of learning time that none of us will ever get back -- I'm wrestling with the moral consequences of teaching tested subjects again tonight.

You see, early on Monday I decided to start recording the spontaneous thoughts about testing that came to my mind over the course of the week on an index card that I carried in my back pocket. By the end of the day today, I had 10 different thoughts on my card -- and I honestly didn't much like what I saw.

Here are three thoughts that have me particularly troubled:

"I'm actually feeling pretty good about things! I know for a fact that I mentioned almost everything that was on my test."

This was my response to a buddy who asked how I was feeling after my students finished our science test on Wednesday. You see the trouble spot, don't you? Since when did mentioning things become a cause for celebration?

The answer is easy: Mentioning things is a cause for celebration when your end of grade exam covers a massive curriculum and measures progress by asking low-level, fact-driven questions. I definitely prioritized coverage over meaningful learning in the past few months even though I'm doubtful that my kids will remember much of what we learned in our short-sighted sprint to measurable glory.

I should be ashamed of that, shouldn't I? And as a guy who believes that true learning should inspire kids to change the world around them for the better, I am. But I am also relieved that nothing on the test would have caught my kids completely off guard.

#sheesh

"If they are going to evaluate me based on test scores, they'd also better find a way to spread out the special programs kids on our grade level."

Because of a nontraditional calendar, my school is broken into four groups of teachers and students that are called tracks. The track that I work on tends to have more students with learning disabilities than the other tracks simply because we have more special education teachers in the building while we are in session.

But because of limited budgets and positions, many of those students are mainstreamed into science classes -- the subject that I teach -- without any special services. And because of limited budgets and positions, our state didn't design any modified versions of the end of grade science exams for kids with significant learning disabilities. Every student took the exact same test.

That left me worrying about my evaluation scores, y'all. I should be ashamed of that, shouldn't I? And I am. Instead of seeing my students with disabilities as the unique, beautiful, capable people that they are, I saw them as a liability -- as kids that were likely to hurt my professional standing.

#sheesh

"I'm glad he thinks his kids struggled."

After our common exam was done, I crossed paths with another science teacher in our building who was pretty convinced that his kids had struggled on our common exam. He was definitely feeling defeated and I could tell that he was professionally down.

As a guy who is passionate about the power of Professional Learning Communities, I should have been there to lift him up, right? I should have been ready to lend a hand and to help him brainstorm ways that we could both improve our work together. I should have been a sounding board and a source of support -- of him as a practitioner and as a person.

But the first thing that popped into my head after our conversation was, "I'm glad he thinks his kids struggled. Maybe my scores will be better than his."

I should be ashamed of that, right? And I am. Collaboration with colleagues has helped me to become the teacher that I am today. My best instructional practices were polished with -- and by -- intellectually generous peers. But I'm more than a little convinced that my "me first" thinking is nothing short of an inevitable by-product of working in a state that has decided that competition between teachers for contract protections is a good idea.

#sheesh

Long story short: I'm starting to realize that standardized testing isn't just changing EDUCATION for the worse. It's changing ME for the worse. I wrestle with that reality every time troubled thoughts like these roll through my mind -- and I'm honestly not sure how to feel about myself as a practitioner anymore.

Detesting and Degrading or De-Testing and De-Grading?

This was written by Paul Thomas who I had the pleasure of co-editing De-Testing and De-Grading Schools: Authentic Alternatives to Accountability and Standardization. Paul tweets here and blogs here. This post was originally found here.

By Paul Thomas

It began with an idea, a play on words: Children (and increasingly, teachers) detest school because the current test-and-grade paradigm is degrading so why not de-test and de-grade the schools?

With that idea in mind, I contacted Joe Bower, whose stance against grades and tests I followed on Twitter, and we began discussing an idea of an edited volume addressing de-testing and de-grading our schools, a direct confrontation of the current high-stakes accountability movement. We were fortunate to invite Alfie Kohn on board for the introduction and a chapter. From there the book was developed—although we struggled through a few hiccups with publishers, landing at Peter Lang USA.

Since the volume doesn’t preview each chapter, I want to offer below some snippets from each chapter, and invite you to join the authors of this volume in our tribute to the teachers at Garfield High (Seattle, WA) for their courageous stance against MAP testing (to whom the volume is dedicated):

De-Testing and De-Grading Schools: Authentic Alternatives to Accountability and Standardization

Joe Bower and P. L. Thomas, editors

Table of Contents

Introduction: The Roots of Grades-and-Tests, Alfie Kohn:

Most of the contributions to this book focus on problems with either grades or tests. In an article about college admissions published more than a decade ago, however, I suggested that we might as well talk about “grades-and-tests” (G&T) as a single hyphenated entity (Kohn, 2001). There are certainly differences between the two components, but the most striking research finding on the subject is that students’ G&T primarily predicts their future G&T — and little else. It doesn’t tell us much at all about their future creativity, curiosity, happiness, career success, or anything else of consequence.

In fact, the case for the fundamental similarity of grades and tests runs deeper than their limited predictive power. Both are “by their nature reductive,” as P. L. Thomas, co-editor of this volume, observes in his chapter. I would add that both emerge from — and, in turn, contribute to — our predilection for three things: quantifying, controlling, and competing. All of these are defining characteristics of our educational system but also permeate our culture more generally.

Part I: Degrading Learning, Detesting Education: The Failure of High-Stake Accountability in Education

Chapter One: NCLB’s Lost Decade for Educational Progress: What Can We Learn from this Policy Failure?, Lisa Guisbond, Monty Neill, and Bob Schaeffer (FairTest.org):


It is not too late to revisit the lessons of the past ten years and construct a federal law that provides support for equity and progress in all public schools. With that goal in mind, this report first provides an overview of the evidence on NCLB’s track record. Second, it looks at recent efforts at NCLB “reform” and what past evidence says about their likely outcomes. Finally, it points to alternative strategies that could form the basis for a reauthorized federal law that would improve all schools, particularly those serving our most needy students.

Chapter Two: High-stakes Testing Assessment: The Deus Ex Machina of Quality in Education, Fernando F. Padró:


From here on forward the discussion reflects how assessment and quality are used as proxies for each other. The discussion comes more from a higher education viewpoint than a P-12 one, but one reason for this is that higher education is facing many of the same issues and pressures; therefore, the concerns at the macro level are more similar than dissimilar. In other words, it is another way at looking at those external influences impacting education and all aspects of educational activity from early childhood until the brink of formally entering the workforce. While the focus is not always on testing and assessment, the discussion is always about testing and assessment because that is the stock in trade within the quality model that is strongly impacting education.

Chapter Three: Technocratic Groupthink Inflates the Testing Bubble, Anthony Cody:


The sooner this groupthink bubble bursts, the better off we will be. In our classrooms, we must do our best to give our students meaningful opportunities to learn, in spite of the intense pressure to raise test scores. In the public arena, we can help burst the bubble by focusing on the big picture data that shows that in spite of a decade of obsessing over data, there is no evidence that better learning results (Hout & Elliott, 2011). We can help burst the bubble by calling out the self-appointed umpires like NCTQ, the Media Bullpen, and dozens of other test-obsessed advocacy groups that are attempting to overwhelm critical discussion of these issues. And we can support efforts to give voice to other points of view, through organizations that allow parents, teachers and students to raise their voices, without the filtering effect of foundation funding.

Chapter Four: Mean Scores in a Mean World, Lawrence Baines and Rhonda Goolsby:


Today, personnel from state departments of education are about as welcome in public schools as vultures. A wake of vultures seldom attacks healthy animals, but prey upon the wounded or sick. So, when student achievement levels wane, the state sees its role not as helper, but as disciplinarian—to punish a school for allowing its students to post achievement scores below the mean. If a school is contacted by the state, the news inevitably is bad— at best, a public humiliation and at worst, a tumult of teacher and administrator firings in a takeover. Firing people, while enjoyable for select politicians, is a tactic that helps neither student nor teacher.

Chapter Five: Degrading Literacy: How New York State Tests Knowledge, Culture, and Critical Thinking, Julie Gorlewski and David Gorlewski:


In June 1999, New York State anticipated the political and pedagogical movement that has engulfed public schools through the federal legislation entitled No Child Left Behind (USDE, 2003). The state’s education department implemented learning standards meant to drive local district curricula. In addition, the state unveiled a plan to attach the standards to mandatory assessments for students in grades 4, 8, and 11, beginning in the area of English language arts (ELA). Consequences for students and educators were significant and comprehensive. In addition to gauging individual student performance, tests at all levels were designed to measure schools’ progress towards meeting the learning standards and to rank schools according to student achievement. Scores and rankings were to be published and distributed by districts, the state education department, and media outlets; and schools with consistently inadequate scores and unacceptable levels of improvement were threatened with the designation “School Under Regents Review (SURR).” So-called SURR schools would be required to show rapid, significant improvement on standardized assessments or face state takeover (NYSED, 1999). Tests were equally high-stakes for students. In June 1999, passing the commencement level ELA examination (intended for students in grade 11) became a graduation requirement for the high school graduating class of 2000.

Chapter Six: The Aesthetics of Social Engineering: How High Stakes Testing Dehumanizes/Desensitizes Education, Morna McDermott:


Schools in America, at least since the industrial age, have been vehicles of social control. Factory model schools, designed during the industrial era, and guided by the industrial paradigm served that framework through economic, ideological, and political means. Now, just as decades ago, high-stakes testing (HST) is the weapon of choice used by education reformers to manipulate the educational system in ways that benefit their agenda to privatize public; pushing a standardized and highly regulated curriculum (to match with the required tests), increased social engineering (using and tracking student data via the HST for other purposes), and corporate profit (through the development, implementation and evaluation of the HST). One cannot deeply understand the origins or purposes of today’s high stakes tests without examining the social, political, and economic climate in which they exist. High stakes testing is the thread that ties together a larger picture of reform that includes: privatization of public education, replacing public schools with charter schools, enforcing a curriculum which “force feeds” meaningless data to already disempowered and disenfranchised communities, and uses “accountability” to turn data into big profits. Each of these issues, as they interface with testing policies and effects, will be explored in this chapter.

Chapter Seven: Standardized Testing and Boredom at an Urban Middle School, Richard Mora:


While conducting a multi-year, gender study at an urban K-8 school, I witnessed and documented the ground-level impact the push toward greater accountability in public education had on the group of 33 working-class, Latina/o students that I followed. At Romero, as I call the school, standardized test scores served as the ultimate measure of the school’s performance. As a result, entire class periods, hours at time, were dedicated to both district and statewide assessments, with teachers teaching to the test, to the practice tests, and to pre-practice tests. During these tests and the various quizzes and exams their teachers administered, the students had to sit quietly at their desks for long stretches of time, an expectation that proven difficult for most.

Additionally, during the sixth grade, the majority of students I observed had a double math period meant to prepare them for the upcoming state exam. Students found these experiences excruciatingly frustrating and repeatedly summed up their feelings with some variant of the statement, “School is so boring.”

Chapter Eight: Reconciling Student Outcomes and Community Self-Reliance in Modern School Reform Contexts, Brian Beabout and Andre Perry:


Education for African-Americans has historically been linked to the broad movement to improve their lot in life. Ceaselessly, from slavery and Jim Crow, towards full membership in American society, schooling was as much about academic learning as it was for ensuring the sustainability of the community in which the school was situated. Due to both de jure and de facto racial segregation of their communities and public schools, there have historically been high levels of self-determination in schooling for African-Americans (Anderson, 1988). The boundaries of the racial community were often undistinguishable from the geographic communities in which African-Americans lived. Racial uplift became theraison d’être in all sectors of Black society, but education offered a pragmatic focus for community development, political empowerment, and economic enfranchisement. This has meant black teachers, the visible presence of the African-American experience in the curriculum, and significant local decision-making power….

This current pervasiveness of market approaches is reflected in the reform language of state takeover, school turnaround, and reconstitution. As a consequence, since 2001, administrative control of many schools serving students of color has shifted from local educators and elected school boards to the states and the federal government who set the accountability policies and determine student and school accountability rules based on test scores. The following chapter interrogates this facially benign policy of raising student achievement with respect to the potential impact on the legacy self-determination of African-American schooling.

Chapter Nine: The Role of Assessment in Empowering/ Disempowering Students in the Critical Pedagogy Classroom, David Bolton and John Elmore:


Since the focus of teacher education at West Chester University has shifted toward teacher training, Democracy and Education, the one foundations course that students take is often where they learn critical perspectives on education. In this class, students define and examine their own philosophies and beliefs about the purpose of education in democratic society and compare, contrast, reject, and borrow from the philosophies of others. Since one of the stated goals of education at West Chester University is to create public intellectuals, it is critical that that foundations course be as empowering as possible.

Learning to think critically about assessment should be a vital part of this foundations course. If students are critically examining the purpose and content of education, i.e., instruction, then students also must learn to become critical assessors of their students. They must be given the intellectual tools to refocus the debate about assessment, so that assessment is not their master, but is a tool that will empower them as they teach their own students.

Part II: De-Grading and De-Testing in a Time of High-Stakes Education Reform

Chapter Ten: The Case Against Grades, Alfie Kohn

Chapter Eleven: Reduced to Numbers: From Concealing to Revealing Learning, Joe Bower:


Since 2006, I have worked to identify and remove things like grading that traditional school has done for so long. And when I share this with others, I receive mixed responses. Some listen intently, nodding their heads in agreement, as if deep down they have always sensed something wrong with what Seymour Papert (1988) described as School with a capital ‘S’ — which is a place that he explains as having a bureaucracy that has its own interests and is not open to what is in the best interest of the children. Unfortunately, when most people close their eyes and think of their Schooling, many have experienced no other kind of School than the one with a capital “S.” Some listen in shock and awe at how school could even function without such things as grading. The people who have a hard time comprehending how children could learn without extrinsic manipulators concern me the most. They are so invested into traditional schooling that they have never questioned its foundation. Unfortunately, some have a distrustful view of the nature of children. Meaning that they believe that without grading there would be nothing to stop children from running amok.

Chapter Twelve: Assessment Technologies as Wounding Machines: Abjection, the Imagination and Grading, John Hoben:


For me the questions surrounding grading are incessant: Do I subtract marks for improper citation style in a paper where a young teacher talks about the death of her father with remarkable insight, wisdom and grace? What grade do I give a teacher who has the courage to write and share her struggles with breast cancer and her fears about leaving her young daughter? Or to a young man who writes about his mother’s struggles with the late stages of multiple sclerosis? Conventional grading gives no consideration to the marks these students should receive for having taught me about grace under fire, about humility and a quiet kind of perseverance instead of “sorting students like so many potatoes” (Kohn, 1994, p. 38). As a quantifying technology which presents teachers with a set of bureaucratic practices for the management of human subjects, grading is a machinery of abjection: a set of technical and administrative practices which works by “casting out” since schooling needs the threat of the wound to maintain its own internal boundaries and hierarchies. More than a simple means of disciplining students and teachers (Foucault, 1995), grading is a mode of schooling the imagination rather than allowing the imagination to radically transform schools. It does this by excluding those who do not fit prescribed models of excellence and teaching us to revile those who do not conform to dominant ways of thinking and being.

Chapter Thirteen: No Testing Week: Focusing on Creativity in the Classroom, Peter DeWitt:


When I entered college, a friend’s parents, who were both teachers, tried to persuade me from entering the field of education, which I found very sad. They were both excellent teachers but they said the profession was changing, and not for the better. I politely smiled and listened to their concerns but I continued down the same path despite their warnings. After working in an after-school program, I knew that I wanted to be an educator. I never forgot the disappointment I felt when those two retired teachers tried to talk me out of entering the profession that they spent so much time in. After seventeen years in education, first as a teacher and then a principal, I understand why they felt the way they did so long ago. However, I still maintain hope that things will get better and strongly believe it is my job as the school leader to help teachers find that love again.

I have come to a crossroads in my career. According to the movie Under the Tuscan Sun, that sounds very Oprah of me. When I began teaching I remember more seasoned teachers stating that if you stay in education long enough you will see the pendulum swing from one side to the other. It is my hope that the pendulum has swung to one very dysfunctional side long enough and will make its way to a side that is based in common sense and sound educational practices before many of us end our careers.

It seems as though policymakers in education want educators to pay attention to research, data and accountability, but they feel that they do not have to play by the same rules. Apparently research, data and accountability only matter when it tells policymakers what they want to hear. Unfortunately, the direction they have been leading education is not good for kids. It is bordering on educational malpractice. Just like the present economic issues in the U.S., education will continue to benefit only the top percentage of kids who can afford it.

Chapter Fourteen: Creating an Ungraded Classroom, Hadley Ferguson:


It is often easy to identify the beginning of an adventure; but where that journey will take you is usually a mystery. That was certainly the case with my adventure into ungrading and using portfolios for assessment. There have been many unexpected twists and turns in the road, unanticipated challenges as well as significant and rewarding successes. When I asked my administration if I could teach an ungraded class, I knew that I was stepping away from the security of my established practice and into a place where all of my skills and knowledge would have to be applied in fresh ways. A new adventure was truly starting. I asked for and was given permission to teach the only ungraded class in an otherwise school with grades. The school was in a time of transition, and teachers had been challenged to experiment with the best strategies for meeting the changing needs of 21st century students. My class, 7th grade history, became a place where learning took place within a new set of standards and expectations. While there were a wide variety of assignments and assessments, none of them was going to end in a grade.

Chapter Fifteen: “Parents Just Want to Know the Grade”: Or Do They?, Jim Webber and Maja Wilson:


Occasionally, someone has the nerve to suggest that grades are overrated, that a focus on them is detrimental, and that everyone might be happier and learn more if we de-emphasized or got rid of them completely. A widely discussed article on Inside Higher Education (Jaschik, 2010) described Cathy Davidson’s efforts to “get out of the grading business.” In her English classes at Duke University, students held regular meetings to decide if their work was acceptable or needed revision. Davidson gave no grades—only descriptive feedback. At the end of the experiment, Davidson declared, “It was spectacular….It would take a lot to get me back to a conventional form of grading ever again.” …

Still, the research accumulates: a study (Pulfrey, Buchs, & Butera, 2011) demonstrated that when students anticipate grades on papers (with or without comments), they become more likely to avoid difficult work than when they anticipate teacher comments without grades. This finding complements Ruth Butler’s (1987) study showing that grades (with or without comments) lead to lower levels of intrinsic motivation and creativity. But suggest that we act on this research—by de-emphasizing or replacing grades in the classroom—and even sympathetic teachers conjure up parent protests: “I’d be the first to get rid of grades and just do writing conferences and narrative feedback! But parents just want to see the grade!”

Chapter Sixteen: De-grading Writing Instruction in a Time of High-stakes Testing: The Power of Feedback in Workshop, P. L. Thomas:


It is now 2012, and I am at the end of my first decade as a college professor of education. After 18 years teaching high school English, a career that was deep in my heart and bones as a teacher of writing, I moved to the university in part as an act of professional and scholarly autonomy. Teaching in education courses, however, has proven to be far less fulfilling and off-kilter to my central concerns with directly addressing human literacy—fostering writers.

After being allowed to teach one section of the university’s introductory English course, I was fortunate that my university re-imagined its curriculum, replacing the two required freshman English courses with two freshman seminars designed to inspire and fuel student engagement in learning. One of the freshman seminars must be writing intensive, and the seminars are taught by professors across departments—not just the English faculty.

This curriculum change has afforded me a unique opportunity to teach a writing-intensive freshman seminar each fall at the university level, where I have the autonomy to implement writing workshop and, most significantly, to de-grade the feedback process of my students crafting their essays. In that context, this chapter opens with a brief discussion of how the writing curriculum has suffered a failed history in K-12 education—almost completely disconnected from the research and craft of composition as a field. Then, I detail my own evolution as a teacher of writing from my high school years as a teacher and into my recent experiences with de-grading the writing classroom for freshmen. I also examine how K-12 teachers of writing are both inhibited in best practices for composition because of the accountability era as well as how those teacher should and can reclaim the teaching of writing for all children.

Chapter Seventeen: One Week, Many Thoughts, Brian Rhode:


Have you ever had the pleasure of watching a school bloom? I have. I watched the walls around me burst into color, like flower petals extending themselves to the great warmth of the spring sun. Splashes of primary shades crawled throughout the school thoroughfares in which I spend my days as a professional. The entrances to classrooms became bustling hives of activity and the productivity was evidenced in the variety of posters, pictures and projects that emerged. Suddenly my small elementary school in upstate NY resembled a field of flowers in the full throws of its spring awakening!

I am certain many of you are asking what possibly ignited such a school-wide explosion of creativity. Quite simply, it was the result of a week without testing. My principal, Dr. Peter DeWitt, had the idea back in the fall of 2011 to give us, as a staff, a much-needed break from the relentless drive of standardized assessment based instruction. As a veteran of the classroom himself, he recognized a way to re-invigorate his teachers by endorsing a respite from the type of instruction that seems to stand in a starkly antagonistic position to the attitudes and beliefs that typically bring people into teaching.

Conclusion: Striving Towards Authentic Teaching for Social Justice, Lisa William-White:


What does it mean to prepare emergent teachers in an era where we bear witness to anti-immigrant discourses and policies; where we see (or even know) scores of people who live in poverty (Measuring Child Poverty, 2012); or where there is widespread bullying of children and youth in schools and communities (From Teasing to Torment, 2005)? What doespreparation mean in a country where we have championed education reform since the 1950s; where we extol the importance of literacy andcritical thinking; and yet, we further prescribewhat constitutes appropriate knowledge, including what content teachers must teach (Common Core State Standards Initiative n.d.)? And, what does this all mean in an era of education deform (Pinar, 2012) – a time of shrinking state budgets, eroding of educational enrichment opportunities for children and youth, rising tuition costs in universities, and where democratic learning spaces in higher education are further undermined by business models for educational decision making?

How do I know what my students know if I don't test them?

Here is an actual Twitter conversation I had with someone we will call Sally:

Joe: The best teachers need not use testsandgrades to know what their students know. 

Sally: How do I know how well my students understand parts of speech without giving them a test on it? 

Joe: Have them write something meaningful that is in a context and for a purpose. 

Sally: How does that test their knowledge of how a noun differs from a verb? 

Joe: well, why do you want children to learn the difference from a noun and a verb if not to write meaningful stuff? 

Sally: Okay, meaningful stuff. Only meaningful if it can be communicated. Only communicated through understanding of language.

Can you see how when we are frantically focused on the minutia of content delivery that testsandgrades can hijack learning and become the raison d'être for school?

Project Based Learning and Portfolio Assessments are authentic alternatives to testsandgrades.

When I have these conversations, I like to remember what Grant Wiggins meant when he said:

When practice becomes unmoored from purpose, rigidity sets in.

What if America enlisted as much creativity in rethinking standardized testing as Mitt Romney has placed in paying his taxes?

Essentially, here is what Nikhil Goyal asked Mitt Romney:

Given that standardized testing in America has increased to historic levels and has had negative effects such as stripping billions of dollars from classroom budgets, increases in teaching to the test, and decreases in creativity, how would you as President change this trend?

Here's a summary of Romney's 3 minute response:

Because I know no other way, I'm not going to change anything about standardized testing, so you better just get used to it. In fact, I plan on intensifying standardized testing to rank and sort not only students but teachers, too. 

Let's examine this video of Mitt Romney and his take on education and testing more closely.

ROMNEY: First of all, you will find through out your life that there are tests.

There is a big difference between preparing kids for a life of tests and preparing them for the tests of life. Defenders of standardized, fill-in-the-bubble, forced-choice examinations have the audacity to cite a 'real world' need for such examinations, and yet standardized testing is what constitutes an amazingly contrived and unrealistic form of assessment.

ROMNEY: I don't know a way to evaluate the progress of students other than by evaluating it through testing of some kind or another.

At this point, I'm prepared to give Romney full credit for admitting his ignorance and incompetence around education and testing. At least he's being honest. It is very likely that Romney actually knows no other way than standardized testing -- but this is not an argument for testing, it's an indictment of his limited understanding and exposure to more authentic forms of assessment such as project-based learning, performance assessments and portfolios. Unlike policymakers and politicians who see education from 30,000 feet up, progressive educators engage in this kind of assessment everyday. Nothing is more dangerous than an idea when it's the only one you have. If the only tool you have is a hammer, then everything looks like a nail; If the only tool you have is a standardized test, every child looks like a number 2 pencil. If Romney truly wanted to broaden his understanding for how to assess learning, maybe he should talk to a teacher.

ROMNEY: If there are tests that are ineffective or that measure things that are not really relevant, obviously you try and improve the test, but you'll still have an SAT when you graduate from high school.

Too many people assume that standardized tests measure the quality of education. This assumption gets us into a lot of trouble because standardized tests were never intended to serve this purpose. Standardized tests are a tool for ranking individual students not rating whole classrooms, schools or nations. There's also strong evidence to suggest that standardized tests are really measuring out-of-school factors such as affluence and poverty (Here's an American example and a Canadian example). A little known fact is that the SAT actually stands for nothing. Clinging on to these blunt instruments and merely tinkering and refining them is the equivalent to re-arranging the deck chairs on the Titanic. It's also important to note that real learning is not found in teacher grades or student tests. The best way to know if a child is learning is to watch them learn. There is no substitute. Testsandgrades merely offer a crutch to those who wish to judge schools without ever spending time in schools. It's important to note that any intelligent conversation about standardized tests would have to include understanding Campbell's Law and the MacNamara Fallacy, both of which Romney appears to have no clue.

ROMNEY: You'll find that through out your life that there are going to be tests. We always complain about them. I complain about them when I was a student, and we don't like tests but there is really no other way we've found out to determine whether a student is succeeding or not succeeding, or frankly whether the teacher is succeeding or not succeeding. So I don't have a better model.

The costs of standardized testing and victims of standardized testing are prevalent. Shrugging or snickering at them as if they are like the weather is not a responsible or professional position to take. Martin Luther King Jr once said, "The day we see truth and do not speak is the day we begin to die." Apathy is runs rampant enough in our society, we certainly don't need our leaders to be also. Some people see standardized testing like death and taxes where resistance is futile. But I wish Romney would see standardized testing more like his taxes. What if America enlisted as much creativity in rethinking standardized testing as Romney has placed in paying his taxes?

ROMNEY: If teaching to the test means learning how to read and write and learning how to do basic math skills then there's nothing terribly wrong with that. 

Testing is not teaching. Writing a test is not learning. Tests can only measure a sample of what we would ever want our children to learn and become. What is on the test might not be as important as what is not on the test. As a parent and an educator I want my children to grow up and live a happy and meaningful life that is full of creativity, empathy, family, courage, integrity, honesty, curiosity, motivation, responsibility, citizenry, leadership, innovation, ingenuity and love. This is but a small list of very important things that standardized tests ignore which is why encouraging teaching to the test needlessly narrows the educational opportunities that we want and need for our children. Proponents of teaching to the test are either irresponsibly ignorant or dangerously deceptive -- either way, they have no business being our leaders.

ROMNEY: What I was concerned about before we had these kinds of tests is that you might have faculty members go off in a completely different tangent from the basic math, english and science skills our kids need to succeed.

Well, you've heard it from Romney himself. He sees standardized tests as a way of managing and disciplining teachers. Even if you are a teacher-hater and see no problem with this kind of adversarial relationship between politicians and teachers, you can't ignore the fact that standardized tests were never created to serve this function.

ROMNEY: I'm not going to replace testing. I'd love to improve it.

This tells you everything you need to know about where Romney wants to take American education. Heck, even the Chinese are moving away from their love affair with testing. This also goes to show that the standardized test and punish brand of accountability is not a daring departure from the status quo -- in fact, it is the status quo that has been sucking the life out of classrooms for decades. Romney is a brilliant example of what I call the American education contradiction: Use standardized test scores to show that public education in the United States is failing but then implement market-based reforms that are almost entirely contradictory to the reforms and policies found in high achieving countries.

ROMNEY: I'd love to have the students grade the teachers at the end of the year, as opposed to just the other way around so that teachers get feedback.

Testsandgrades are a primitive form of feedback. Assigning people a score or a grade does nothing to inform them of what they are doing well, or what they can improve -- all they can do is rank and sort. Testsandgrades are not about feedback or learning, they are about manipulation. Romney likes testing because he can use testing to manipulate and control the public in a way that undermines public education and teachers' unions.

ROMNEY: I hope students are very involved in the political process and in the process in the quality of your education.

Ultimately, I find it sadly ironic that Romney can look 17 year old Nikhil Goyal in the eye and say that he hopes students get involved more in shaping their education system, but then spend 3 minutes lecturing him on how, as President of the United States, he would merely ignore and intensify the ills of standardized testing -- and that Nikhil, and other kids like him, should just get used to it.

Under what circumstances would this be deemed exemplary leadership?

Testsandgrades are just tools -- it's how they are used that matters most.

When I make the case that testsandgrades have no place in education, I am often met with a common rebuttal that goes something like this:

Testsandgrades are just tools. They are not inherently good or bad. It's how they are used that matters most.

This rebuttal has troubled me for sometime because deep down I have always sensed there to be something off with this reasoning, but until now, I've had a hard time articulating it.

While I have been loosely familiar with Marshall McLuhan who told us, "We shape our tools and thereafter our tools shape us."

But it wasn't until I read Alfie Kohn's Schooling Beyond Measure that I felt like I could put my thoughts to words:

You've heard it said that tests and other measures are, like technology, merely neutral tools, and all that matters is what we do with the information. Baloney. The measure affects that which is measured. Indeed, the fact that we chose to measure in the first place carries causal weight. His speechwriters had President George W. Bush proclaim, "Measurement is the cornerstone of learning." What they should have written was "Measurement is the cornerstone of the kind of learning that lends itself to being measured." 

One example: It's easier to score a student writer's proficiency with sentence structure than her proficiency at evoking excitement in a reader. Thus, the introduction of a scoring device like a rubric will likely lead to more emphasis on teaching mechanics. Either that, or the notion of "evocative" writing will be flattened into something that can be expressed as a numerical rating. Objectivity has a way of objectifying. Pretty soon the question of what our whole education system ought to be doing gives way to the question of which educational goals are easiest to measure.

The question isn't just how we should use our tools, like testsandgrades, rather we should be taking step back and asking should we be using these tools at all.

If we enslave education to the quantifiable, we fall victim to what some have called the McNamara Fallacy.  If we can't abandon our needless obsession with reducing learning and people to numbers, school will continue to be a place that less and less of our children will feel like they belong.

Throw away your gradebook

Every gradebook I have ever seen has rows and columns that have students names on the left and assignments along the top, and inside each cell is a space for the grades that await each student. By definition every student is mandated to do the same assignment as their peers.

But what if this kind of standardized tracking is precisely what is holding us back from providing children with a more personalized learning environment?

What if we no longer required all students to do the same projects at the same pace?

I've maintained that differentiated instruction and differentiated assessment would be a heck of a lot more attainable if teachers threw away their gradebooks.

Since I threw mine away in 2004, I have liberated my students (and myself) from the expectation that everyone has to do the same assignments at the same time.

Having high standards for all my students does not require standardization, and throwing out my gradebook helped me understand this.

The Folly of Multiple Choice

"Anyone can confirm how little the grading that results from examinations corresponds to the final useful work of people in real life."

-Jean Piaget

It's final exam time at my school, and my teacher colleagues are collectively herding to the multiple-choice test scoring machine. For just under $800 CAD, our scoring machine can:

• Scan up to 35 sheets per minute 
• Grade up to 100 questions per pass 
• Score exams with up to 200 questions 
• PC compatibility for advanced data collection and analysis

The front of the instruction manual proudly reads “GRADING YOUR TESTS JUST GOT EASIER!” After I watched scoring sheet after scoring sheet scurried through the scoring machine, I can personally attest to how easy this really is. It's no secret why multiple choice exams are so popular among teachers – their utility is second to none. But, what are the cons to multiple choice tests? Here are a few items to think about before giving your next multiple choice test:


Ambiguity

Misinterpreting a question can result in an "incorrect" response, even if the response is valid. A free response test allows the test taker to make an argument for their viewpoint and potentially receive credit. Depending on the number of possible answers that are provided, a test taker could have a chance of completely guessing the correct answer. It is conceivable for a student to select the wrong answer for the right reasons or to select the right answer for the wrong reasons. The results of such a multiple-choice exam are surrounded with uncertainty and doubt.


No partial credit

Even if a student has some knowledge of a question, they receive no credit for knowing that information if they select the wrong answer. Free response questions may allow a test taker to demonstrate their understanding of the subject and receive partial credit.
Even carefully constructed exams that reflect very detailed curriculums can be used to improperly assess students. If an exam was created to carefully reflect a certain curriculum, you might see only one question that covers a specific outcome. What if that student did in fact understand that outcome but for any number of reasons, they get the question wrong? That means that this test would report that that student understood nothing of that concept – which most likely would be wholly misleading and untrue. How often can a teacher honestly report that a student understands nothing?

Overemphasis on timeliness

A premium is placed on speed at the cost of creativity and thoroughness. This overemphasis on timeliness also contributes greatly to the ambiguity of the exam. Most test-takers are taught to madly fill in the remaining answers before having their exam taken away by the exam supervisors. There is no way to differentiate between these random guess responses and the responses that were carefully and thoughtfully selected. Recognizing guessing as a problem, some test creators enact a penalty such as deducting a mark for incorrect answers – the hope being that test takers will not guess and instead leave the question blank. This solution may stop the guessing, but it still does not address the ambiguity, as all those unanswered questions will simply show that the test taker got them all wrong – when in truth, the test taker may have had some level of understanding, but because they couldn't get finished in time, or they were too scared to guess, they receive no credit.


Subjectivity

How is the length of the exam decided? How many questions are necessary to show enough understanding? In the case of reading comprehension exams, how many reading selections will there be, and what is an appropriate length? How many answers will there be to select from? Which outcomes will be tested? Which will be excluded? Which will be more heavily weighted?

Depending on the date, the question "how many planets are there in our solar system?" has a different answer. What about all those students who were penalized for excluding Pluto as a planet before 2006?

The point here is not to try and figure out the answer to these questions; rather, there is no one answer for these questions. And yet, the choices made by the test taker can have an immeasurable effect on the test's results. One of my favorite quotes on the subjectivity of testsandgrades comes from Paul Dressel who said, "A mark or grade is an inadequate report of an inaccurate judgement by a biased and variable judge of the extent to which a student has attained an indefinite amount of material.


Behaviouristic in nature

These tests only care about whether the student got the right answer. They can't measure whether the student has a true understanding for the content. Even in a subject such as math that can be (mis)labelled as very black and white and right or wrong, it should very much matter how a student comes to answer the question 2+2=4. Did that student simply memorize his cue cards, or does he actually understand the addition process? A multiple choice test does not and cannot concern itself with understanding such valuable information.

Poor Testing can lead to Poor Teaching

Some teachers may use multiple choice exams voluntarily while others may find their use compulsory. Either way, teachers may feel pressure to achieve high scores on these tests, and that kind of pressure can lead to poor teaching, such as the use of lecturing on the behalf of the teacher and memorization on the behalf of the student. Take math for example, many teachers may teach tricks or shortcuts such as: when dividing two fractions, simply flip the second fraction and multiply. A student could mindlessly comply and perform quite well by choosing the correct multiple choice answer. In cases like this, a poor assessment tool has lead to a poor teaching technique (one that relies on mindless compliance and memorization rather than true understanding); however, if we use the test scores as an indicator for learning, that teacher and student appear successful. Inferences made from multiple choice tests can be undermined leaving the successful and superficial students indistinguishable.


Interrater Reliability

Multiple Choice exams are created with one right answer in mind for each question. This straightforward scoring system is used so that any two raters will always agree upon how well a student did. This need for agreement, also known as interrator reliability by statisticians, is gained at an alarming price; Authenticity is sacrificed for (perceived) reliability.

If we were compelled to identify who truly benefits from this kind of artificial measurement, I sincerely doubt anyone could honestly say that this is for the kids. Ultimately, this is an example of the needs of the system trumping the needs of the learner. Alfie Kohn puts it this way:

"You know it's a bad assessment if it's multiple choice. Multiple choice tests can be clever but they can't be authentic. You can't learn what kids know and what they can do with what they know, if they can't generate a response - or at least explain a response. Or as one expert in psychometrics told me many years ago, "Alfie don't you get it, multiple choice tests are designed so lots of students who understand the material will be tricked into picking the wrong response". That's why teachers would never dream of giving a multiple choice test of their own design because the same thing applies there."

Testing Test-taking skills

Multiple choice exams require a certain amount of test taking skills, and some students have better test taking skills than others. Many teachers will actually teach students strategies for writing multiple choice exams. For example, some test takers understand that an answer that has the words "always‟ or "never‟ is usually NOT the correct answer, because rarely is something ever "always‟ or "never‟. This is considered a fairly good strategy, and students who are aware of it may have a better chance of doing well.

However, there are some test takers who have come to believe in poor strategies. For example, some students believe the pattern of responses matters and so they say to themselves, “This can't be another "b‟ answer as we have just had three in a row.” Or they believe in myths such as “when in doubt, pick C”. Granted, we can all probably agree this is a silly strategy, but what if students actually use it? The format of the exam has skewed the measurement of that student's learning.


Averaging Averages

Traditional practice encourages test raters to not only mark each question right or wrong, but to also tally up the number of correct responses and compare that to the total number of questions – of course, we know this to be the average or mean. However, what does this number actually tell us?

Let's pretend there are three questions on the test for every outcome we taught. You could then look at the data and see how many of those three questions a specific student got right or wrong. Let's say for those three questions a student got 1 out 3 correct but for another three questions, that tested a different outcome, the student got 2 of 3 correct. Separately, he understood 33% of the first outcome and 66% of the second outcome. However when you average these averages, he gets 3/6 which comes to a mark of 50%.

What do these numbers mean anymore? Imagine how diluted the average has become when you have 50 to 100 questions that may be measuring the same number of different outcomes. And yet these grades' importance is elevated to grand heights. (Note that the problem of averaging averages is not exclusive to multiple choice exams)

Collaboration = Cheating

Ask any parent for a list of characteristics they wish their children to develop as they grow into adults and there is a very good chance that collaborative skills are somewhere on that list. When you think back to your schooling, how often were you permitted to collaborate with others during examination? If you did try to collaborate, we all know what that was called – cheating! And you got in trouble for it.

Unfortunately, there may some progressive classrooms out there, but it would be a very safe bet to make that most classrooms still have students sitting and writing their exams in isolation. Regardless of your job or profession, how often are you told to figure something out in total and complete isolation – no books, no help, no talking? In the real world, there simply aren't that many times you are expected to solve a problem or perform a task in complete and total isolation – and even if you were, it would be awfully archaic to refuse you the opportunity to reach out for the help you needed to get the task done.

When we say to children, "I want to see what you can do, not what your neighbor can do", this turns out to be code for "I want to see what you can do artificially deprived of the skills and help of the people and resources around you. Rather than seeing how much more you can accomplish in a well functioning team that's more authentic like real life." (Again note that the lack of collaboration during exams is not exclusive to multiple choice exams)


Thinkingcuffs

The very nature of multiple choice tests slaps students with a pair of thinkingcuffs. Who does the majority of the thinking on a multiple choice exam? Who asks all the questions? Who proposes all the answers? Thinking is messy. Learning is messy, but multiple choice tests conveniently remove the mess. All students are required to do is circle or fill in a dot. If we were truly interested in assessing student learning, shouldn't we encourage the students to show us as much of their thinking as possible? Because no one can construct meaning in a preconceived bubble, reducing something as beautiful as learning to a bubble sheet is an exercise in needless oversimplification.

Differentiated Instruction and undifferentiated Assessment

Many teachers today would readily admit that all learners learn differently, and it is the teachers responsibility to address these different learning styles with differentiated instruction; however, many teachers still use multiple choice tests in an attempt to measure their student's learning. There is a real disconnect between our understanding of differentiated instruction and our attempts to measure learning with our undifferentiated, standardized assessment tools.

While it is true that all children should have the opportunity to get an education that does not mean that all children should get the same education. When it comes to instruction and assessment, we need to stop trying to meet the needs of all learners by pretending all learners have the same needs.


Value what we Measure or Measure what we Value

It is true that it makes good sense to occasionally stop and reflect upon how well we are learning – the rest of the time we should concern ourselves with actually learning whatever it is we have set out to learn.

A short anecdote may enlighten this point: A man was seen on his hands and knees searching underneath a street light. It was late at night and very dark. When a passerby inquired what the man was doing, the man said that he was looking for his lost keys. The passerby then noted that the man was fortunate that he had lost his keys under the street light. The man quickly replied that he actually lost his keys a distance to the north, but it was too dark over there, and so he wanted to search where it was easy to see.
There is a big difference between measuring what is simply easily measurable and measuring what we actually consider important. Multiple choice tests measure a very limited and narrow kind of learning. If a great amount of importance is placed on these kinds of tests, people will come to see these limited and narrow kinds of learning as most important – sacrificing their pursuit of other valuable kinds of learning that are rarely measured on multiple choice exams.

While a lot of people concern themselves with what will be on the test, I find myself thinking more about about what can never be on these kinds of tests. Show me the multiple choice test that can assess things like sense of humor, morality, creativity, ingenuity, motivation, empathy.

***

Too many education systems have confused measurement with assessment and forgotten that the latin root for assessment is assidere which translates into "to sit beside". Assessment isn't a spreadsheet -- it's a conversation. 

Multiple choice tests were originally tools used by teachers, but today teachers are tools used by multiple choice tests. This shouldn't come as any surprise, especially if you are familiar with some of Marshal McLuhan's work who once said, "We shape our tools and thereafter our tools shape us."

Despite all these reasons for abandoning the use of multiple choice tests, their utility seems to trump their consequences. What's even more discouraging is that many teachers still choose to use multiple choice exams despite having a plethora of more authentic assessment alternatives such as performance assessments, portfolios, written response and personal, two way communications.

Teachers who continue to use multiple choice exams as their primary or default assessment tool are engaging in a kind of educational malpractice because they are reporting on their student's learning in a way that may range from being marginally inaccurate to wholly untruthful.

I asked Irmeli Halinen, head of curriculum in Finland, how often a teacher in Finland would use a multiple choice test as a way of assessing their students. Her answer said it all:

"Our teachers rarely if ever use multiple choice tests because they would rather have their students do something real."