Knowledge is power, power is money, and I want it

Knowledge is Power Program (KIPP) is a charter school system in the United States. Mathematica Policy Research Inc. has released a study on KIPP in relation to attrition rates and test scores.

The Washington Post summarized the study:

Mathematica studied 22 KIPP middle schools, including AIM and KEY, comparing test scores of charter students to scores of selected students in regular public schools who matched their academic and demographic backgrounds. Researchers examined test data starting in third grade. KIPP middle schools begin in fifth.


By seventh grade, half of the KIPP schools studied showed growth in math scores equal to an additional 1.2 years of school. Reading gains for KIPP were not as dramatic but still significant, the researchers reported, reflecting an additional three-quarters of a year of growth.


Mathematica said it found no evidence that KIPP schools were systematically drawing students with more economic advantages from surrounding school systems. But attrition rates at the KIPP schools, measuring the portion of students who failed to complete four years at the schools, varied widely. In a third of the schools studied, attrition was significantly higher than in other local public schools. In another third of the KIPP schools, the rate was lower. Skeptics say that students who can't function in the rigorous school culture are often pushed out -- a claim that KIPP rejects.

Despite KIPP's attempts to sell this study as a feather in their cap, Gary Miron, Professor and Director at Education and the Public Interest Center at the University of Colorado at Boulder, explains some shortcomings, omissions and distortions that arise from this study:

However, an initial analysis of the report by Professor Gary Miron of Western Michigan University concludes that this initial study report misrepresents the attrition data. According to Miron, "While it may be true that attrition rates for KIPP schools and surrounding districts are similar, there is a big difference: KIPP does not generally fill empty places with the weaker students who are moving from school to school. Traditional public schools must receive all students who wish to attend, so the lower-performing students leaving KIPP schools receive a place in those schools."



In contrast, Miron explains, "The lower performing, transient students coming from traditional public schools are not given a place in KIPP, since those schools generally only take students in during the initial intake grade, whether this be 5th or 6th grade."

The KIPP study's description of attrition only considers half the equation, when comparing KIPP schools to matched traditional public schools. The researchers looked at the attrition rates, which they found to be similar - in the sense of the number of students departing from schools. But they never considered the receiving or intake rate. Even though the researchers agree that the students who are mobile are lower performing, they do not take into account the reality that KIPP schools do not generally receive these students.

Professor Miron conducted his own quick analysis, using the Common Core database, and concluded that there is a 19% drop in enrollment in KIPP schools between grades 6 and 7 and a 24% drop in enrollment between grades 7 and 8. (This analysis only included KIPP schools that had enrollments in all three grades). In comparison, traditional public schools in these grades maintain the same enrollment from year to year.

Aside from the controversy around KIPP's attrition rates, I find it important to reflect on any study that uses test scores as it's primary tool of evaluating any learning environment. We know that measurable outcomes and standardized tests may measure some of the least important aspects of education, that standardized tests encourage a great deal of shallow thinking, and that standardized tests are at best unreliable indicators of anything that resembles good learning - so what does that say about any teacher, school or charter system that holds high test scores as a badge of honour?

Perhaps we should be more than a little skeptical of schools that owe their name to a chant by Harriett Ball: “…Knowledge is power, power is money, and I want it.”