Monday, June 28, 2010

Irrational IQ tests

In his book What Intelligence Tests Miss, Keith Stanovich explains some shortcomings of IQ tests:

Although most people would say that the ability to think rationally is a clear sign of superior intellect, standard IQ tests devote no section to rational thinking as cognitive scientists would define the term. To think rationally means adopting appropriate goals, taking the appropriate action given one's goals and beliefs, and holding beliefs that are commensurate with available evidence. Although IQ tests do assess the ability to focus on an immediate goal in the face of distraction, they do not assess at all whether a person has the tendency to develop goals that are rational in the first place. Likewise, IQ tests are good measures of how well a person can hold beliefs in short-term memory and manipulate those beliefs, but they do not assess at all whether a person has the tendency to form beliefs rationally when presented with evidence. And again, similarly, IQ tests are good measures of how efficiently a person processes information that has been provided, but they do not at all assess whether the person is a critical assessor of information as it is gathered in the natural environment.


Given that IQ tests measure only a small set of the thinking abilities that people need, it is amazing that they have acquired the power that they have. IQ tests determine, to an important degree, the academic and professional careers of millions of people in the United States. University admission officers depend on indicators that are nothing but proxies for IQ scores, even if the admissions office dare not label them as such. The vaunted SAT test has undergone many name changes (from Scholastic Achievement Test, to Scholastic Aptitute Test, to Scholastic Assessment Test, to simply the letters SAT) in order to disguise one basic fact that has remained constant throughout these changes - it is a stand-in for an IQ test.

It's truly amazing how we can become so narrow minded in our measurements. When we become so focused on emprirical measurements, we place far too much emphasis on analytical thinking at the cost of intuitive and rational thinking.

People who place any kind of importance on IQ tests should listen to Mark Twain very carefully:


It ain't what you don't know that gets you into trouble. It's what you know for sure that just ain't so.
I've cited this Twain quote more than a few times. I think it embodies the underlying message of most of my blog posts. 

3 comments:

  1. It amazes me that no one seems to get exhausted with criticizing IQ tests. What really is at play is that people have differing opinions of how we define "intelligence." Not everyone agrees on what intelligence is, and that's OK. But IQ tests measure their authors' definition of intelligence, and nothing more. IQ tests actually do an excellent job of testing what they intend to assess; verbal and nonverbal skills, processing speed and attention. They have been tested, retested and retested again to determine their reliability and validity, and they do, in fact, provide accurate results based on that definition of intelligence. They don't assess emotional intelligence, and they don't pretend to. They are most used for assessing learning disabilities and/or deficits from brain injury, and there is no evidence to suggest they don't do this well.

    As for the SAT, calling it an "intelligence test" is just semantics (and in my mind, a way to sell books). Head over to Collegeconfidential.com and search for a variety of threads asking questions along the lines of "my IQ is really high, so why doesn't my SAT score show it?" Innate ability takes you only so far, and the SAT shows that smarts aren't everything. Preparation and effort can absolutely be rewarded, and those things have nothing to do with IQ as measured by IQ tests. The SAT measures critical reading skills, math skills, and grammar/writing. The test structure is unlike an IQ test in pretty much every possible way. Now, of course, it's arguable whether the SAT really predicts college performance, but it does measure a student's reading, math and writing skills at that point in time. You may feel that that's irrelevant to college admissions, but that's an entirely different issue.

    The bottom line is that tests are not at fault. They are designed to measure certain things, which they do quite well. The debate lies in how we define intelligence, and how we choose to use the results of these tests.

    ReplyDelete
  2. Jenn, I really do think you miss the whole point. I don't see how letting the tests go free of any responsibility helps the state of public education.

    ReplyDelete
  3. It is worth noting that the IQ was originally designed to separate the people that could be educated from the people that couldn't. I believe the IQ test has had a huge impact on how education has evolved and what we as a society have put value on. I look forward to reading this book.

    ReplyDelete