Scientifically Tested Tests

The New York Times featured a brilliant article by Susan Engel on September 19. She wrote eloquently about rethinking assessment. 

Her main points are worthy of serious reflection:

• why are we so hell-bent on census testing when we could be assessing samples of students?
• why do we continue to use such limited assessment formats such as multiple choice?
• why do we continue to stress the importance of such limited measurements when there is good reason to believe that some of the worse teachers successfully hide behind such narrow goals and rigid scripts?
• why do we subscribe to such a behaviourist's approach of rewarding and punishing schools based on test scores that tend to conceal more than they reveal?
• if even a fraction of these questions have validity, then the real question is cui bono? Who benefits from the use of such limited measurements?

As much as I love to ask challenging questions, I can see the merit in offering up probable solutions, and this is exactly what I love so much about Susan Engel's article. She provides a number of substitutes to traditional, fill-in-the-bubble standardized tests:

Instead, we should come up with assessments that truly measure the qualities of well-educated children: the ability to understand what they read; an interest in using books to gain knowledge; the capacity to know when a problem calls for mathematics and quantification; the agility to move from concrete examples to abstract principles and back again; the ability to think about a situation in several different ways; and a dynamic working knowledge of the society in which they live.

This task is not as difficult as one might think. In recent years, psychologists have found ways to measure things as subtle as the forces that govern our moral choices and the thought processes that underlie unconscious stereotyping. And many promising techniques already used by child development experts could provide a starting point for improving school assessments.

For instance, using recordings of children’s everyday speech, developmental psychologists can calculate two important indicators of intellectual functioning: the grammatical complexity of their sentences and the size of their working vocabularies (not the words they circle during a test, but the ones they use in their real lives). Why not do the same in schools? We could even employ a written version, analyzing random samples of children’s essays and stories.

Psychologists have also found that a good way to measure a person’s literacy level is to test his ability to identify the names of actual authors amid the names of non-authors. In other words, someone who knows that Mark Twain and J. K. Rowling are published authors — and that, say, Robert Sponge is not — reads more. We could periodically administer such a test to children to find out how much they have read as opposed to which isolated skills they have been practicing for a test.

When children recount a story that they have read or that has been read to them, it provides all kinds of information about their narrative skill, an essential component of literacy. We could give students a book and then have them talk with a trained examiner about what they read; the oral reconstruction could be analyzed for evidence of their narrative comprehension.

Researchers have also found that the way a student critiques a simple science experiment shows whether he understands the idea of controlling variables, a key component in all science work. To assess children’s scientific skills, an experiment could be described to them, in writing, and then they would explain how they would improve upon it.

Of course, these new assessments could include some paper-and-pencil work as well. But that work would have to measure students’ thinking skills, not whether they can select a right answer from preset options. For instance, children could write essays in response to a prompt like, “Choose something you are good at, and describe to your reader how you do it.” That would allow each student to draw on his area of expertise, show his ability to analyze the process, describe a task logically and convey real information and substance. In turn, a prompt of, “Write a description of yourself from your mother’s point of view,” would help gauge the child’s ability to understand the perspectives of others.