Saturday, February 13, 2010

IQ: Misunderstood and Misused

Reflecting upon one's beliefs can be a very productive use of time, and I can think of no better time to do so than when we have come to mindlessly accept something as a given truth. When questions are no longer answered because questions are no longer being asked.

Intelligence-quotient (IQ) tests are a great place to start.

There are a lot of misconceptions surrounding IQ tests, mostly because of flat out ignorance. Most people don't understand their purpose nor do they understand how to properly use the results.

Here's some history.

Carol Dweck provides us with some insight from her book Mindset: The New Psychology of Success:

Wasn't the IQ test meant to summarize children's unchangeable intelligence? In fact, no. [Alfred] Binet, a Frenchman working in Paris in the early twentieth century, designed this test to identify children who were not profiting from the Paris public schools, so that new educational programs could be designed to get them back on track. Without denying individual differences in children's intellects, he believed that education and practice could bring about fundamental changes in intelligence.

Sir Ken Robinson expresses concern for the current day misuse of IQ tests in his book The Element:

Ironically, Alfred Binet, one of the creators of the IQ test, intended the test to serve precisely the opposite function. In fact, he originally designed it exclusively to identify children with special needs so they could get appropriate forms of schooling. He never intended it to identify degrees of intelligence or mental 'worth'. In fact, Binet noted that the scale he created 'does not permit the measure of intelligence, because intellectual qualities are not superposable, and therefore cannot be measured as linear surfaces are measured.

It is sadly ironic that Binet intended on his IQ test to be a tool used to find ways to properly include children with the opportunity to gain a better education; however, the IQ test, for the most part, has been used as a gatekeeper to exclude children from further education.

I not only find it objectionable to bastardize the purpose of IQ tests, but I also find it objectionable to assume IQ tests are a valid and reliable indicator of a person's potential. In Outliers, Malcolm Gladwell writes about Lewis Terman who made the study of the gifted his life's work. Terman took 1,470 children whose IQ's averaged between 140 and 200. Gladwell writes:

For the rest of his life, Terman watched over his charges like a mother hen. They were tracked and tested, measured and analyzed. Their educational attainments were noted, marriages followed, illnesses tabulated, psychological health charted, and every promotion and job change dutifully recorded...

"There is nothing about an individual as important as his IQ, except possibly his morals," Terman once said. And it was to those with a very high IQ, he believed, that "we must look for production of leaders who advance science, art government, education and social welfare generally."

... Terman believed that his Termites were destined to be the future elite of the United States.

Terman is not alone when it comes to placing so much faith in IQ. Gladwell points out that elite universities and high-tech companies place a great deal of importance in measuring the IQ, and that those students with the highest IQ have the greatest potential to be the best employees.

But there was a problem.

Terman's geniuses didn't prove to be all that successful in life. Gladwell explains:

By the time the Termites reached adulthood, Terman's error was plain to see. Some of his child geniuses had grown up to publish books and scholarly articles and thrive in business. Several ran for public office, and there were two superior court justices, one municipal court judge, two members of California state legislature, and one prominant state official. They tended to earn good incomes - but not that good. The majority had careers that could only be considered ordinary, and a surprising number ended up with careers that even Terman considered failures. Nor were any Nobel Prize winners in his exhaustively selected group of geniuses. His field workers actually tested two elementary students who went on to be Nobel laureates - William Shockley and Luis Alvarez - and rejected them both. Their IQ's weren't high enough.

In the end, critics of Terman's project said that had Terman simply chosen random children from the same kinds of backgrounds as Terman's Termites, and ignored IQ, he would have seen similar results.

Terman ended up reluctantly concluding:

We have seen that intellect and achievement are far from perfectly correlated.
This lack of correlation has been confirmed by others. Dan Pink's book A Whole New Mind explains:

Daniel Goleman, author of the groundbreaking book Emotional Intelligence, has examined an array of academic studies that have attempted to measure how much IQ (which, like the SAT, measure pure left-brain directed thinking prowess) accounts for career success... The answer: between 4-10 percent.

Sir Ken Robinson tends to agree with this evidence. In his book The Element, he writes about how we focus on asking 'how intelligent are you?' when we should be asking 'how are you intelligent'. Like Dweck, Goleman and Gladwell, Robinson explains that the IQ test was never to be used to determine someone's intellectual prowess - and standardized tests like the SAT measure a far too narrow defintion of intelligence. Robinson explains:

The SAT is in many ways the indicator for what is wrong with standardized tests: it only measures a certain kind of intelligence; it does it in an entirely impersonal way; it attempts to make common assumptions about the college potential of a hugely varied group of teenagers in one-size-fits-all fashion; and it drives high school juniors and seniors to spend hundreds of hours preparing for it at the expense of school study or the pursuit of other passions. John Katzman, founder of the Princeton Review, offers this stinging criticism: "What makes the SAT bad is that it has nothing to do with what kids learn in high school. As a result, it creates a sort of shadow curriculum that furthers the goals of neither educators nor students... The SAT has been sold as a snake oil it measured intelligence, verified high school GPA, and predicted college grades. In fact, it's never done the first two at all, nor a particularly good job at the third.
Robinson is not alone in his critcisms of the SAT. Alfie Kohn has written extensively on how standardized tests are at best unhelpful and at worst down right harmful. In his article Two Cheers for an End to the SAT, Kohn writes about what SAT actually does not stand for:

One imagines the folks at the College Board blushing deeply when, a few years back, they announced that the "A" in SAT no longer stood for "Aptitude." Scarlet, after all, would be an appropriate color to turn while, in effect, conceding that the test wasn't -- and, let's face it, never had been -- a measure of intellectual aptitude. For a brief period, the examination was rechristened the Scholastic Assessment Test, a name presumably generated by the Department of Redundancy Department. Today, literally -- and perhaps figuratively -- SAT doesn't stand for anything at all.
In The New York Times article Junior Meritocracy, Jennifer Senior writes that IQ tests may be more about measuring socio-economics and opportunity than intellect:

“An analogy people use a lot for this is planting corn,” says Barnett, from Rutgers. “If you want to know about the properties of different kinds of corn, you have to plant it in land that’s well fertilized and well irrigated. If you plant it in soil that’s dried up and rocky, you won’t know, because nothing will grow.” The same, he explains, goes for children. How can one possibly know anything about their minds if they’ve spent their first four years in unstimulating environments?

“People have the idea that with these tests you can cancel out socioeconomic background and get to some real thing in the kid,” agrees Nicholas Lemann, dean of the journalism school at Columbia and author of The Big Test, a history of the SAT. “That’s a chimera. If you’re a 4-year-old performing well on these tests, it’s either because you have fabulous genetic material or because you have cultural advantages. But either way, the point is: You’re doing better because of your parents.

Rather than promoting a meritocracy, in other words, these tests instead retard one. They reflect the world as it’s already stratified—and then perpetuate that same stratification.

“Instead of giving IQ tests, you could just as easily look at Zip Codes and the education levels of the parents to determine who gets the better schooling—you get a very high correlation between IQ and socioeconomic status in the first seven or eight years of life,” says Samuel J. Meisels, assessment expert and president of Chicago’s Erikson Institute, the renowned graduate school in childhood development. “Giftedness is a real thing, no question. But giftedness can be extinguished, and it can be nurtured.”

Kohn confirms what the SAT truly measures:

The SAT is a measure of resources more than of reasoning. Year after year, the College Board's own statistics depict a virtually linear correlation between SAT scores and family income. Each rise in earnings (measured in $10,000 increments) brings a commensurate rise in scores. Other research, meanwhile, has found that more than half the difference among students' scores can be explained purely on the basis of parents' level of education.

While there is plenty of evidence to question both the validity and reliability of IQ tests, there is an equal amount of reason to question IQ as a fixed quantity. Carol Dweck's entire premise behind her book Mindset is based on the idea that the view you adopt of yourself profoundly affects the way you lead your life. She makes a clear distinction between a fixed and growth mindset:

If you have only a certain amount of intelligence, a certain personality, and a certain moral character -well, then you'd better prove that you have a healthy dose of them. It simply wouldn't do do to look or feel deficient in these most basic characteristics.

If we come to believe IQ is a fixed measurement of our intelligence, then we develop a mindset that encourages us to prove how smart we are. Rather than spending our time getting better at something, we feel the need to use our time showing everyone how good we are. Even worse are the people who spend all their time focusing on ensuring they don't look dumb - rather than continuing to improve and getting smarter.

I think we can all agree that there is a huge difference between trying not to look dumb and trying to get smarter. Dweck's point here is that we shouldn't ever see our IQ as a fixed quantity - we should see our IQ as something that can shrink and grow. So is there any evidence to support the idea that IQ can shrink and grow? Sir Ken Robinson offers a rather interesting anecdote in his book The Element:

IQ tests can even be a matter of life and death. A criminal who commits a capital offense is not subject to the death penalty if his IQ is below seventy... People can also improve their scores through study and practice. I read a case recently about a death row inmate who'd at that point spent ten years in jail on a life sentence. During his incarceration, he took a series of courses. When retested, his IQ had risen more than ten points - suddenly making him eligible for execution.
Jennifer Senior's article Junior Meritocracy also speaks to the idea that IQ is not fixed:

In 2006, David Lohman, a psychologist at the University of Iowa, co-authored a paper called “Gifted Today but Not Tomorrow?” in the Journal for the Education of the Gifted, demonstrating just how labile “giftedness” is. It notes that only 45 percent of the kids who scored 130 or above on the Stanford-Binet would do so on another, similar IQ test at the same point in time. Combine this with the instability of 4-year-old IQs, and it becomes pretty clear that judgments about giftedness should be an ongoing affair, rather than a fateful determination made at one arbitrary moment in time. I wrote to Lohman and asked what percentage of 4-year-olds who scored 130 or above would do so again as 17-year-olds. He answered with a careful regression analysis: about 25 percent.

I defy anyone to call a test that can achieve consistant results from the test-taker 25% of the time reliable. If IQ tests are guilty of all this, why the hell do we continue to place so much importance on these tests? What's worse is that despite all these harmful effects, people claim that we do all of this for the children.

Placing any kind of meaning on IQ tests equates to a kind of educational malpractice.


  1. Hi Joe,
    So a tire is half flat; we know this because we can see it with our eyes, we can poke it with our finger and we can feel it as we drive the car it's installed on. In order to remedy the problem we need to put air in it. Enter pressure guage.

    In my time as a special education teacher I worked with many students that weren't functioning properly in a very obvious way, and the reason was unknown. We used IQ tests as a tool to provide data that had the potential to add some insight into what we were dealing with. I agree that the original intent of Binet's IQ test was contrary to the way the test has been, as you say, bastardized in contemporary times. That doesn't devalue the original intent.\

    In a special education environment well-trained, intuitive and insightful teachers can see when a child is struggling; they can feel it and they know something is not right when they instruct the child to do something. As the tire pressure guage is a tool to provide more specific information to the person repairing the obviously flat tire, the IQ test, when adminsitered properly, and with the correct intent, is a tool to provide more specific information about the student's challenges and/or strengths,(of course you know I am more interested in the strengths-the asset model,)so the teacher can work effectively at supporting the student's growth and development.

    Of course I agree that IQ tests should not be used as a measure to determine a student's lot in life, but I believe there is a purposeful use for these tests. When we have observed a student who is exceptional at either end of the spectrum, the IQ sub-test scores provide data that helps us plan supportive measures designed to remedy weakness, or manipulate strength.

    I know you'll have feedback... love to hear it.

  2. There's a lot to be said for an expanded vision of human mental abilities that goes way beyond IQ, and there are plenty of historical and ongoing abuses of standardized testing in general, including "intelligence" tests.

    Unless a lot of data ends up being fraudulent, there are also legitimate individual differences that are relatively stable in our own lifetime (though not neccessarily between generations) and tell us something potentially useful albeit limited.

    We can de-emphasize the g-factor and its heritability as well as its past putative importance, but we can't make legitimate observations go away in the statistical dust just because they are inconvenient. Perhaps we will have to abandon IQ, but it needs to be because we have a better explanation of existing data, not just because the concept falls out of fashion.

    Having met some remarkable geniuses in my lifetime, I suspect that they truly think differently from the rest of us, even given the same training and expertise. I think there's a very real capacity (or some cluster of factors) for grasping complexities of situations that we notice mostly when it is way out of average. That's not to say that IQ is the best way to express this at all, or even that it can be captured in a single measurement. But so far I don't see a better one.

    Perhaps "IQ" is a weak idea for capturing this kind of individual difference, which notably we've never really put adequately into somatic terms but let's not go so far as to pretend that there aren't real meaningful more or less stable differences between people. Let's just be clear that these are only a tiny part of practical mental abilities and that intelligence is an evolving and expanding concept.

    Thanks for a very interesting and thought provking article.

  3. I had an IQ of 148 last time it was measured (a long time ago).

    Which apparently made me a genius.

    Er, no. I practiced doing IQ tests for a week before I took the IQ test proper and when I did I knew how to answer most of the questions by recognition. Mostly memory and very little intelligence required.

    I can still legitimately claim I have an IQ of 148 though; and perhaps I might. :-)

  4. Thanks for the tutorial, very tests


Follow by Email