Sunday, February 27, 2011

Conflicting data



Sometimes the data we use doesn't tell others what they think we are telling them.

And sometimes the data we read doesn't tell us what others think it should tell us.

The reductionist nature of educational data is by its very nature problematic. While somethings lend themselves well to being reduced to a number, real learning and children are definitely not one of them.

7 comments:

  1. Joe - As an educator with a large number of optimistic, powerful, delightful, and eager followers, you have a responsibility to our field to get the facts right. While in the past, I've said something when one of your hyperbolic tweets was out of line or tried to engage you in conversation regarding rubrics as you continue to attack attributes of rubrics they don't actually posses. This time, I've no qualms about confronting you directly.

    This blog post has gone too far. You're wrong.

    Data are not just numbers. This is not an opinion - this is the reality of research, of science, of our field. Presenting "educational data" as reductionist enforces the politically-created mentality that learning can be reduced to a number and that unless the data are presented in tables and charts, they are not legitimate.

    If students are being "reduced to a number", my hunch it's because the mentality that data are numbers has been adopted as a truth, rather than a half-truth. You want to rail against the over-reliance on quantitative data, it would be great to have you help advocate for multiple measures (which, incidentally, does not mean more standardized testing, lest you think that's what I'm implying).

    Just because some people are using numbers and data as synonymous, doesn't mean it's true. Just because "everyone" knows when someone says "data", they mean "quantitative" doesn't mean it's the whole story - sadly, it means our profession is lacking a systemic way of educating our members in our own vernacular.

    You've no obligation to re-post, edit, or even respond. I hope, as a fellow educator, you'll consider revising so that your language reflects what is true, rather than what sounds good.

    Jennifer (@datadiva)

    ReplyDelete
  2. I entirely agree that a single piece of data can never tell you about the complex world that is learning and education. However, a good mixture of different measures can give you an excellent tool to compare and discuss different students and groups. Context is king, and the data should only ever start the discussion, and should never be used an absolute judgement.

    ReplyDelete
  3. Data are not just numbers. This is not an opinion - this is the reality of research, of science, of our field. Presenting "educational data" as reductionist enforces the politically-created mentality that learning can be reduced to a number and that unless the data are presented in tables and charts, they are not legitimate.

    If students are being "reduced to a number", my hunch it's because the mentality that data are numbers has been adopted as a truth, rather than a half-truth. You want to rail against the over-reliance on quantitative data, it would be great to have you help advocate for multiple measures (which, incidentally, does not mean more standardized testing, lest you think that's what I'm implying).

    Just because some people are using numbers and data as synonymous, doesn't mean it's true. Just because "everyone" knows when someone says "data", they mean "quantitative" doesn't mean it's the whole story - sadly, it means our profession is lacking a systemic way of educating our members in our own vernacular.

    ReplyDelete
  4. Wow, Jennifer. Way to take it personally. I'm not an educator, I work in finance and I'm very familiar with the abuses of data. The worst offense is applying a model that doesn't reflect reality. I think that describes data-driven decision making in finance or education or any other organic system.

    Just as an aside: science is sometimes as much opinion as fact. Don't get too caught up in your dogma.

    ReplyDelete
  5. Robert - You are absolutely correct that I took it personally. It appeared as if Blogger ate my first comment and so I posted the second one with my comments to Joe removed.

    That said, this isn't about dogma. It's about definitions. Redefining words to make a point interferes with discussion of what really matters. I absolutely agree that education is too dependent upon quantitative measures. Most likely because they're seen as easier and make better soundbites. I think one of the most powerful questions we can ask in any field is: What other evidence we do have? That requires data - of both flavors.

    ReplyDelete
  6. Saying there should be balance between quantifiable and qualitative data is nice and all, but let's not pretend a balance actually exists. There are pockets of hope in classrooms where teachers focus as much or more on qualitative data, but I don't know any district that counts how ethical and creative their students are compared to how well they fill in bubbles on a standardized test.

    When the rubber meets the road, quantifiable data rules the day.

    ReplyDelete
  7. Now that is an argument I can, and do support. We're overly dependent on quantitative data. That should be the argument - not that data are bad. By perpetuating the myth that data are numbers, it almost seems as if you're enforcing the very system you are trying to change. Just because you haven't seen districts using qualitative data, doesn't mean it isn't being done.

    ReplyDelete