Monday, April 9, 2012

One average to rule them all

Sketch by Jeff Bower
Imagine if your vehicle's instrument panel took all of your gauges including your engine's temperature, tachometer, odometer, speedometer and fuel gauge and reduced all that data into one convenient number.

As silly and potentially dangerous as this sounds, this is precisely what school does when we try and reduce what matters most (creativity, imagination, literacy, numeracy, responsibility, democratic citizenry, empathy and collaboration) to a spreadsheet friendly average.

Authentic accountability is about transparency. That is, the public should have access to the information they need about the schools they pay for.

There's nothing transparent about reducing something as magnificently messy as real learning to numbers. To reduce learning to numbers is to conceal far more than it reveals.

Consider this the next time you see a student's learning reported as a single piece of data.

6 comments:

  1. Ummmm.....no. Kinda not even close.

    Nothing is more misleading than a bad analogy. Not even a lie.

    Those gauges do not measure progress. The measure performance. At a single moment. Different aspects of performance. And then, in your analogy, they are put into a single composite measure.

    What is that most analogous to in education?

    Think about the high school English writing assignment. Some sort of multi-page essay or report or argument. The teacher might grade that by considering the organization of the piece, the use of conventions (i.e. spelling, grammar, punctuation), audience awareness in the presentation, the understanding of the original text displayed and the quality/development of the argument being presented. 5 different aspects of the writing. That teacher might do all this more or less unconsciously, or s/he might have some sort of rubric to help him/her make sure s/he considered each trait/dimension wholly. Then -- as is so often the case -- s/he will assign some sort of overall grade or score for the piece.

    One performance. One point in time. Many dimensions. One overall score.

    This is not about an average. Both the teacher and set gauges in the car can compile these different dimensions of performance in VERY complex ways to arrive at that final score. Don't sell the teacher or the engineer short.

    This kinda of thing is done all the time in education, and who argues against it?

    If this is the analogy that displays your problem with big standardized assessment than you should recognize that your problem is with the reduction of ANY complex task or set of complex task to a single score. This every in education. I hated it, but it was there in my own classroom. My students had been trained to expect it, as had their parents, and the whole structure around me demanded it. I am sure that it is in your own classroom, too.

    *****************************************

    What are report card grades? Well, to stick with your analogy....

    This setup is repeated at various points over time, but not entirely regularly. When you are in a cab, in someone else's car, when you rent a car, when you drive a spouses or friend's car, at various points someone notices the master gauge number and jots it down. Sometimes when you are on the highway. But sometimes when you are just backing out of a driveway. Whenever. Sometimes when stopped at a light. Sometimes when filling up with gas. All kinds of times. Irregularly, but a whole bunch of times.

    THOSE numbers are more or less averaged. They might be a weighted average of a few categories, but still pretty damn simple.

    That's a report card grade on a single subject.

    What's a GPA? Well, that's taking more than one of that report card grade number and straight averaging them. Perhaps all the members of a family, even if they don't all live together. That's GPA.

    Pretty stupid, eh?

    Now why are you complaining about standardized assessment when all of this is right there in your own room, your own house, your own neighborhood?

    ReplyDelete
  2. There are real issues with your work choice that makes it clear how sloppy you are with this stuff.

    1) A number or a grade are NOT a single piece of data. Not all data in education are numbers, and not all every number is a piece of data. Those "a single piece[s] of data" that you refer to might be "measures." They might be averages. Or some fancily derived composite scores. They come from multiple pieces of data.

    2) Exactly whom do you think reports student learning as a single piece of data like this? Those whom you so commonly assail do not claim to be reporting learning. They usually say they are reporting "performance." If you are reading that as "learning," that is your own mistake, not theirs.

    There's actually quite a bit of attention to move towards growth scores rather that simpler scores based upon snapshots of performance. There's a lot of talk, thinking and research into how to do that. Even then, people talk of "growth," not "learning."

    So, you seem to be assailing the straw men of your own imagination as though you bravely speaking truth to power.

    ReplyDelete
  3. Yes! That's precisely the problem with averaging.

    ReplyDelete
  4. Any average or single score claiming to represent the performance of an entire school or that of the learning of a student won't likely give useful information. If a single score is used, it's important to know all of the data that goes into that single score to even begin to get an understanding of how well the student or school is doing/performing. I don't disagree with what Mr. Bauer has written, but the first two commenters on this post seem to be referring back to previous disagreements with his posts/stances on education.

    ReplyDelete
  5. A great type on my part. I meant "sloppy word choice," not "sloppy work choice." Alas, the sloppiness of a quickly written comment....

    How unfortunate.

    More importantly, the substance of my objections to this post have rather little to do with anything from Joe's past. Rather, they are objections to the post he has written and the thinking behind it. It's too bad that it seems Mr Rees would rather dismiss them out of hand than address their contents.

    ReplyDelete
  6. I love the analogy and I think it's very relevant. A cumulative grade given at the end of a term is just that, cumulative. It represents the overall academic knowledge of a student, assuming that a teacher's gradebook is reflective of learning. In other cases, it may represent a summation of academic performance including things like timeliness, participation, etc.

    However, for the purposes of trying to diagnose what a students deficiencies are, a summative grade is borderline useless. Objective (or standard) based grading is far more effective in helping teachers identify learning targets for students that should be focused. Furthermore, it's far more appreciated by the student. Reluctant learners are able to look at it and say "Well, I need to work on [x] but at least I did ok on [y]." While traditionally strong students can look at it and say "Wow, I did great on [x], [y], and [z] but I can still get better at [q]."

    Nice work Joe, thanks for sharing.

    ReplyDelete