Thursday, December 1, 2011

Real assessment for learning

Most educators have come to differentiate between assessment OF learning and assessment FOR learning. (I actually don't - I differentiate only between supportive and unsupportive assessment, but that's a different post)

Assessment OF learning is typically defined as a summative evaluation that usually takes the form of a grade that judges a student after they are done learning.

Assessment FOR learning is typically defined as a time for formative feedback that helps students learn from their successes, failures, mistakes and misconceptions. This feedback is timely and informative in nature rather than judgemental or evaluative.

While it is true that the prospect of abolishing grading from school entirely may be controversial, the idea that we should never grade or judge students during formative evaluation is not up for debate. The point here is that grading practices have no place while students are still learning.

Some forms of conventional wisdom have teachers using rubrics to provide students with feedback during the assessment FOR learning process. As a classroom teacher, I have come across a lot of different rubric designs; yet, despite their differences, there is always one common denominator:
Every rubric I have ever seen involves the use of a fixed measurement scale. 
By definition, a grade is any attempt to reduce learning to a symbol. While it's true that these scales can be represented in a number of different ways (numbers, letters, descriptors, smiley faces, stickers, stars, etc) these reductionist scales are inevitably experienced by the student as a judgment. It's important to note that 75%, B-, and "proficient" have distinctions without a difference. A grade by any other name is still a grade.

While it may seem counterintuitive to suggest that we should not evaluate and judge students while they are still learning, this is precisely what the research has been showing us. We often seriously overestimate the effectiveness of judgement and evaluation as a precondition of learning.

Assessment guru Dylan Wiliam puts it this way:
Grades cause an emotional reaction – either positive or negative. Feedback causes you to think and engage, which is reflective learning.
Alfie Kohn suggests:
Never grade students while they are still learning something and, even more important, do not reward them for their performance at that point.
Paul Dressel explains:
A mark or grade is an inadequate report of an inaccurate judgement by a biased and variable judge of the extent to which a student has attained an indefinite amount of material.
Jerome Bruner proclaims:

Students should experience their successes and failures not as reward and punishment but as information.
A more profound statement than Bruner's about how children should be assessed is hard to imagine. Bruner's law provides us with both what we should not be doing while simultaneously suggesting a superior alternative to grading and manipulating students. Because learners can only experience grading and other fixed measurement scales as a reward or a punishment, they have no constructive role to play in the learning process. This is precisely why rubrics and their fixed measurement scales have no place place in assessing children while they are still learning.

In other words, we need to fight back the urge to marinate children in our praise, disapproval, bribes, threats, rewards and punishments and dedicate ourselves to providing children with nothing more than the information they need to improve.

Here is a Prezi that explains the three-step process I use as an alternative to grading and judging students in an effort to provide them with information they need to learn.







While it is true that I use this to guide both my written and verbal interactions, the majority of the feedback that I relay to children is done verbally. The two-way nature of a conversation almost always trumps the one-way nature of written comments.

Assessment's latin root is assidere, which means "to sit beside". This is why it is critical we remember that assessment is not a spreadsheet or a judgement - it's a conversation.

18 comments:

  1. How do you feel about performance based/interactive assessments? They are my assessment of choice because they are providing a product and applying it to a unique/unfamiliar situation and I get to really see if they "get the big idea".

    ReplyDelete
  2. Great arguments! I had the idea in my head that they should get a grade on formative assessment to give them an idea of where they stand "so far" but I'm seeing more and more that the grade is just a distraction.

    ReplyDelete
  3. Your discourse always resonates with me and informs my practice. Rubrics and attendent rating scales are embedded in my system's assessment practice. Tomorrow is another day out of the classroom while we are encouraged to master data in structured teams. Rubric data configures our nacient efforts to differentiate. One way or another grading implicates learning in my classroom.

    ReplyDelete
  4. Great post. The one thing missing from your form of assessment, which I think is essential, and which, incidentally, is part of a crierion-referenced assessment system, is that students need to know beforehand, what the expectations are in the work/ assignment. And I am not referring to "organization" or any other ambiguous terms. I mean real, authentic components that are skill or content focussed. This helps students understand what they are doing and why they are doing it.
    While I agree with the approach which you have offered (sit down, verbal feedback, etc, etc.), I do not think it is fair to simply "move the bar" for every student. The bar should already be set so students have the ability to gauge their improvement. It is this level of improvement that motivates students (and humans for that matter) to work hard, seek further feedback and carry on learning.

    ReplyDelete
  5. I spent some with colleagues earlier this week exploring the difference between clarity and certainty. In the midst of the conversation, we pondered the difference between semantic discussions and the search for the clarity. What your post has highlighted for me is that we're going to constantly struggle to get clarity if we (royal, educators writ large we) cannot agree on our own jargon.

    To me "summative" doesn't require grades. Summative means only it comes at the end of learning, exploration or project. Summative assessment are opening nights, final debate, conference presentations, submission of manuscript to an editor. Summative comes at the end. Summative is the chance for a learner to say "check out what I know, can do" or a teacher to say "show off what you've learned". When we talk about or think about assessment, it happens at three moments of time: before (diagnostic), during (formative), and after. Summative does not imply evaluation - unless you use the phrase summative evaluation. Summative does not require a grade - unless it's sitting within a system that works with grades. I agree that students shouldn't be evaluated while still learning. I disagree that the only point of a summative assessment is to generate a grade. I do not understand why "judgement" is a bad thing. Learner deserves the chance to say "stop talking to me about this and let me show you what I can do." Learning isn't entirely about process, it's about application. I wonder how we can explore improving the quality of those moments if discussion of summative is perpetually linked to grades.

    Pre-Apprentice. Apprentice. Crafts(wo)man. Master Crafts(wo)man. These four levels reflect the official titles in a craftsperson union. Is this a fixed measurement scale? Perhaps - but it also authentically reflects the evolution of expertise in a given trade. I imagine the adults in this union care nothing about grades but do care about improving their skill level. Getting better, doing better, being better does not require grades. To act as if all rubrics are attached to grades is to deny what happens in life. Pre-Service. Novice. Practitioner. Mentor. Our own profession has a "fixed measurement scale" for thinking about degrees of quality. This is what a quality rubric can do. It can take the intangible attributes that differentiate the evolution levels of quality and make them explicit. Again, I struggle with how this is a bad thing. Of course, I should reiterate my caveats: rubrics are way over used and majority available online aren't all that good. Hammers are used to break car windows. That doesn't mean hammers are bad tools.

    My final point - Engaging a fellow human in a conversation about their work is amazing and powerful and hands down the best way to help a learner see the strengths and weaknesses in their work. However, sometimes learners need, want, deserve a monologue. A quality rubric can allow a learner to independently self-assess the quality of his or her work and make revisions. It frees the child from having to wait for an adult to reflect. Frankly, an appreciative isn't going to hunt down a mentor and ask "am I doing this okay" every time they want to improve. They're going to sit down, compare where they are to where a master is and take steps to improve. A 30-year-old can probably do this but even adults benefit from making the implicit, explicit. Students need, and deserve, support during this reflection process.

    If we force students to wait for an adult, or a peer, to get feedback on the quality of their work, we are communicating that the only way to get better, do better, be better is because of others. In absence of quality rubrics, what options are available to support a learner improve the quality of their work without waiting for others' feedback?

    ReplyDelete
  6. Junk again.

    Again, your arguments don't match your conclusion.

    1) "Students should experience their successes and failures not as reward and punishment but as information." OK.

    That's easy. Grade the student, let the student know -- as information -- the quality of the work or the performance, but don't factor that grade into the final report to the student/parent/school/district. Ta-Dah! Information without being a reward or punishment.

    2) "A mark or grade is an inadequate report of an inaccurate judgement by a biased and variable judge of the extent to which a student has attained an indefinite amount of material."

    That criticism is applicable to nearly all feedback given to students, regardless of it's form. It certainly is true of feedback given to students on anything approaching authentic work when given by a teacher with dozens or scores of students. That is not a condemnation of grades, that is a caution about the limits of our tools.

    3) "Never grade students while they are still learning something and, even more important, do not reward them for their performance at that point." The fact is that what we really want students to learn -- call them what you will, habits of mind, higher order thinking, deeper thinking skills, larger themes, problem solving, 21 century skills -- do not ever get all learned. All students are always still learning something. Heck, all people should be still learning something. This formulation does not allow for any grading ever.

    4) "Grades cause an emotional reaction – either positive or negative. Feedback causes you to think and engage, which is reflective learning." Are you really endorsing the idea that non-grade feedback never causes emotional reactions? Are you really endorsing the idea that grades never cause students to think, engage or reflect? Have you really never seen a student a little hurt by negative feedback or a little cheered by positive feedback? Have you really never gotten a disappointing grade and wondered what you did wrong, or what the grader missed?

    *************

    I don't like reporting grades in summary fashion as a final record of students' work in a course. I think this is bad for education, bad for learning, and therefore ultimately bad for society. But that is a different argument.

    There are plenty of good arguments against grading along the way, though none of them are dispositive. Unfortunately, none of them appear here.

    This is, as usual, a bunch of pablum masquerading as insight or argument.

    ReplyDelete
  7. Ceolaf are you an educator or just a critic? Just curious because you put an awful lot of time into something you seem disconnected from and may never apply in your own practice, whatever that practice is.

    ReplyDelete
  8. I have been an educator who has been thinking and working on these issues for over 25 years, and I am capable of recognizing an ad hominem attack when I see it.

    ReplyDelete
  9. DataDiva has me thinking about" fixed measurement scales."

    Don't we all have scales in our heads? For example, imagine a teacher who says "This is a good start," "This is shows some promise," "I'm starting to see what you can do," and "This is excellent work!"

    Think back on your own practice. Think of how students might do an impression of you. Think about the patterns in how to talk with students, and the ways that you have a set of phrases that are just comfortable for you. Think about how you sometimes try to find another way to say something -- to mix it up -- but really mean the same thing.

    These are GOOD things. Our students need structure and regularity. They should struggle to understand what we are saying, as we want that kind of effort to go into their own work and their own learning. One way that we can be more clear with them to adopt consistent ways of speaking with them. Professionals do this. It helps our students.

    We have some number of categories or ranges in our heads. And we try to be consistent in how we communicate with students.

    So, seriously, how is that not a fixed measurement scale? How is that really different from what rubies do, and you criticize?

    ReplyDelete
  10. It doesn't matter whether the feedback is in the form of a number, a letter or a phrase. What matters is the summary nature that lacks specific information on what the student did well and what student did poorly.

    And rubrics can address this problem. If reported back to students on a trait by trait level, rubrics -- even with a fixed measurement scale -- give students specific information that they can act on. Even if not reported back on a trait-by-trait level, a rubric's explicitness can help students to spot areas of potential weakness or strength -- as they focus the learner and the grader on specific aspects of the work.

    ReplyDelete
  11. ceolaf, you ask some very good questions that could be potentially answered by reading some of the research on this topic.

    Butler, R. (1988). Enhancing and undermining intrinsic motivation: The effects of task-involving and ego-involving evaluation on interest and performance. British Journal of Educational Psychology, 58,1-14.

    Pulfrey, C., Buch, C., & Butera, F. (2011). Why grades engender performance-avoidance goals: The mediating role of autonomous motivation. Journal of Educational Psychology, 103, 683-700.

    ReplyDelete
  12. Even if everything could be described as judgment on a mixed measurement scale, Richard Ryan and Edward Deci's Self Determination Theory tells us that it's critically important that learners not feel a sense of powerlessness because they feel controlled externally.

    ReplyDelete
  13. Pulfrey is about grading evaluations. Quality Rubrics are about learners self-assessing. Pulfrey's findings don't apply. Rubrics are not always about grades.

    From the Black and Wiliam article you shared, "Self-assessment will happen only if teachers help their students, particularly the low achievers, to develop the skill" and low and behold, on the next page, "taking time to help students understand scoring rubrics is also very helpful. Students can be given simplified versions of the rubrics teachers use, or they can be encouraged to create their own."

    We get better because we want to get better. Writing down the attributes of work products or people who are REALLY good at what we want to do, good at what we want to go, almost there, and just beginning can help us figure out where we are and how to get to the good. That's a quality rubric. Don't like fixed measurement scales? Then call it column 1, column 2, column 3, column 4, etc. Denying students the chance to reflect on degrees of quality because you don't like the tool seems ... antithetical to a conversation.

    ReplyDelete
  14. Shrugging at Butler or Pulfrey's findings and how they relate to rubrics isn't something I'm prepared to do.

    At the heart of their findings is that when students anticipate a grade they tend to avoid taking the necessary risks real learning requires. In other words, the potentially beneficial impact of the formative comments is overshadowed by the grade.

    While you may have found ways to use rubrics in a way that does not include a fixed measurement scale or a grade, let's not pretend that this is common practice. Rubrics are overwhelmingly used as a tool for rationalizing grading practices.

    Renaming the columns may sooth your conscience but I'm not prepared to pretend the kids won't see a 3 out of 4 as 75%. Remember that our best intentions are not as important as the students' perception.

    Ultimately, grading, scoring guides and fixed measurement scales are tools that meet the system's needs. Rather than stretching to save these tools, I'm prepared to jettison them completely so that we can focus less on meeting the needs of the system and focus more on meeting the kids' needs.

    ReplyDelete
  15. I'm not shrugging at the research, however, I do think it's compelling you've ignoring the fact an article you cited to support your point about motivation also supports my point.

    I struggle to connect research against grading to quality rubrics because in my experiences, they are not connected in lockstep to grading. Yes, the majority of rubrics you see are connected to grading, but that does not negate their benefit. "Common practice" does not dictate the quality of a tool. My conscious is fine but I appreciate your concern and I don't think you can speak to how all students see rubrics any more than I can speak to how all teachers are using rubrics.

    We can keep going around this again and again but there remains one question at the heart of my discourse with you: How do we support students' need to self-assess their work without forcing them to wait for external feedback? Without practice and support, human beings struggle to find weaknesses in our own process and products because of our desire to protect our own ego. How does a learner get better, do better, be better without relying on external feedback? Saying to child, "think about what you can do better" rarely accomplishes anything, especially for a struggling student.

    In my experience, Quality Rubrics are a way to accomplish that goal but I'm happy to hear your ideas on supporting student self-reflection on the quality of their work.

    ReplyDelete
  16. 1) You seriously misread me if you think that I defend grades. I think that letter and numeric grades are one of the blights on education that gets in the way of classrooms and schooling built around significant grades.

    2) I never said everything could be described as a judgement on a fixed measurement scale, despite your misguided implication.

    Rather, I said that fixed measurement scales are prevalent far beyond use of letter/number grades. I meant that most teachers have set set of known responses that DO comprise a fixed measurement scale. Deep and/or long form feedback is not of that form, but much of what we say as we circulate around the room, monitor and give quick feedback IS of that form.

    If you are going to criticize grades for being fixed measurement scales, I think you miss the problem with grades.

    3) One of the great things about multti-dimentional rubrics is that -- so long as they are not used holistically -- they are NOT fixed measurement scales, even though they contain fixed measurement scales. Their multidimensional nature allows for a specificity of response that fixed measurement scales cannot. Because they can address multiple aspects of work, learned are offered far greater an opportunity to learn from the assessment. Use of the rubric itself along with the judgements of the assessor (e.g. learner, peer, parent, teacher, etc.) the student has guidance in hand for what s/he may do next. The provision of those dreaded fixed measurement scales actually provide goals for students because they are pair with descriptions of the difference between where they are and where they might aim to be.

    Frankly, your whole criticism of rubrics here demonstrates that either you lack understanding of what a rubric is, how it is used or even the nature of the problems that your cited articles and sources address.

    ReplyDelete
  17. I'm not sure how someone can be profoundly against grading and then advocate for their use on a rubric. (grading = reducing learning to a symbol)

    ReplyDelete
  18. Joe,

    Thanks for sharing this again today. I've increasingly moved away from rubrics this year for many of the same reasons that you describe. The only time they come into play is either as a guide for students as we start a research project (written or oral presentation) or as a way to frame what scores the students can expect on their work when it is assessed by the International Baccalaureate. As much as I am a proponent of the IB philosophy, Learner Profile, and Diploma Program (full disclosure: I am currently a part of a pilot course program and curriculum development group), as soon as you attempt to quantify a student's work, you are grading the work. Grading, for whatever reason, has a sense of finality and judgement that is inconsistant with formative assessment, narrative feedback, conversation, and all of the other important components of learning that are essential parts of our craft.

    ReplyDelete