Want high test scores? Buy an expensive house!

In his book Measuring Up: What Educational Testing Really Tells Us, Daniel Koretz writes:

Several years ago, I received a phone call from a total stranger who was about to move into my school district and wanted me to help her identify good schools. She assumed that because of what I do for a ligivng, I ought to know this. I took her question more seriously than she wanted and told her briefly what I would look for, not only as an expert in testing and educational research but aslo as a parent of two children and a former elementary and middle-school teacher. As a first step, I suggested, she should gather as much descriptive information as she could readily obtain to get a notion of which schools she might want to consider. Test scores would be high on my list of descriptive information, but many other things might be important as well, depending on the child: the strength of the school's music or athletic programs, some special curricular emphasis, school size, social heterogeneity, and so on. Then, once she had narrowed down her list far enough (this was a very large district), I said she should visit a few schools that looked promising. A visit would allow her to get a glimpse of the characteristics of the schools, including those that might help account for their test scores. I explained some of the things that I had looked for when I had checked out schools and classrooms for my own children -- for example, a high level of student engagement, clear explanations from teachers before students undertook tasks, a level of enthusiastic activity when it was appropriate, and spirited discussion among the students. With both the observations and descriptive information in hand, she would be better able to identify schools that would be a good match for her children. 

She was not pleased. She clearly wanted an answer that was uncomplicated and that would entail less work, or at least less ambiguity and complexity. A simple answer is reassuring, especially when both your children's education and a very large amount of money are at stake. (this was in Bethesda, Maryland, where housing prices were outrageously high.) 

A few weeks later, I mentioned this conversation to a friend who at the time ran a large testing program. He repliced that he received calls of that sort all the time and that few callers wanted his answers either. They wanted something simpler: the names of the schools with the highest test scores, which the callers considered enough to identify the best schools. He told me that in one conversation he hand finally lost his patience when the caller resisted a more reasonable explanation and had told her, "If all you want is high average test scores, tell your realtor that you want to buy into the highest-income neighbourhood you can manage. That will buy the highest average test score you can afford." 

The home buyer's phone call reflected two misunderstandings of achievement testing: that scores on a single test tell us all we need to know about student achievement, and that this information tells us all we need to know about the school quality.