Wednesday, August 1, 2012

Strength of multiple choice tests

A strength of multiple choice tests is that they allow us to assess large groups of students simultaneously and identically.

A fatal flaw of multiple choice tests is that they deceive us into thinking we can (and should) assess large groups of students simultaneously and identically.

10 comments:

  1. This comment has been removed by the author.

    ReplyDelete
  2. A flaw in the argument against multiple choice is the implication that there is no shared knowledge - no facts that are common to all of us, when in truth, our realities are defined by shared understanding.

    Arguments against multiple choice deny the fact that at many times, we must chose from among the best options presented to us. That our greatest expression of civil rights, is in effect, a multiple choice test.*

    Multiple choice tests are a tool that are a best fit for a limited number of learning standards and have been around for thousand of years (H. Wainer, 'Uneducated Guesses').

    Assessment is a conversation - and conversations include many components from body language to inflection to the words we chose to use with each other. To suggest that the language can only occur in one register or in one way, is not only inaccurate, it's unfair. Or to suggest that conversations shouldn't include yelling because you don't approve of yelling, seems ... flawed.

    *Note - this is in no way suggesting multiple choice tests prepare us for the "real world" or assess the important skills of life. They obviously don't. They are an assessment tool, a limited option that can provide general information about students' skills and knowledge that is a good fit for a limited number of standards.

    ReplyDelete
  3. I had a professor who consistently said "statistics describe precisely no one person." Your comments on the fatal flaw are spot on--it operates on the assumption that all kids should be learning the same things at the same time. Jenn B makes a good point about some value in multiple choice but I do wonder about the shared understanding bit. For me, shared understanding comes from discussion and testing assumptions with each other, I wouldn't feel comfortable saying that if three students all respond to the same answer the same way at the same time on multiple choice test they have a shared understanding of that concept.

    ReplyDelete
  4. If the curriculum outlines a certain amount of knowledge that must be learned, isn't multiple choice an appropriate way to assess that knowledge? Well-constructed multiple-choice questions can even test skills up to the levels of analysis and evaluation. The problem with assessment isn't multiple choice, the problem is reliance only on multiple choice, and I honestly don't see this as a reality in actual classrooms. Standardised tests? Sure. A problem, especially if the questions are not well-written. But is it truly being suggested that teachers are only using multiple-choice questions, and nothing else, to assess their students? Or, is the suggestion that the curriculum itself is pointless, and we shouldn't be teaching our students according to it?

    ReplyDelete
  5. Jenn, when would you say that multiple choice would be inappropriate?

    ReplyDelete
  6. Multiple choice measures that have been developed by a test developer like the Education Testing Service, ETS, which develops tests for law, medical and graduate school admission, go through an engineering-like process which renders the tests standardized; that is reliable and valid. I question the validity of the tests though. Who determines that the set of intellectual properties deemed to be lawyer-like characterizes all lawyers for example? Besides isn't that the point of education, you can learn to take on the characteristic ways of thinking of the profession through study and professional practice.

    ReplyDelete
  7. Multiple choice tests don't even allow the learner to generate a response. This is a primitive form of assessment that confuses quality with convenience.

    If multiple choice tests could really evaluate higher order thinking why would we ever do projects and performances that require so much more time and effort from both teachers and students?

    And yes I am suggesting that multiple choice tests are a major problem with classroom teachers. If you surveyed how many final exams in North America are multiple choice, I would bet that a gross majority are multiple choice. Suggesting that we have anything that resembles a balance between real projects and multiple choice tests is moderation disguised as capitulation.

    ReplyDelete
  8. All curricula has content. I could care less about what students know. I want to know what can they do with what they know. Filling in a bubble is not on my list of things I care about.

    If students can do cool projects and performances that are in a context and for a purpose while constructing their own understanding while interacting with their environment -- then we will know what they know.

    ReplyDelete
  9. Joe - There are numerous times when multiple choice is inappropriate. My html isn't strong so I'm not sure how to hyperlink in this comment to an activity I share with teachers about congruence. Happy to email it to you if you'd like. I should note that 90% of the assessments I support teachers with are authentic and project-based. My goal is to have teachers choose performance or product-tasks because they are the best type for the standard under discussion, not because they don't like multiple choice. Telling an AP physics teacher she can't use multiple choice "just because" (regardless of the emotional appeal of my argument) rarely works when she's looking at 120 students. But if the professional development addresses the different types of assessment, the issues of congruence and validity, MC ceases to be the assessment of first resort. We end up with performance tasks for the important stuff and recall tasks (multiple choice) for the other stuff. We count the things we can so we can focus our energy on the stuff we can't.

    ReplyDelete
  10. Wouldn't the high level & low level stuff both be evident in projects & performances? What do we need multiple choice for?

    Bloated class sizes or inappropriate teacher workloads is not an argument for multiple choice tests -- it's an argument against large class sizes. I agree that multiple choice tests are convenient -- but don't confuse that with quality.

    ReplyDelete