Saturday, September 4, 2010

Knowledge is power, power is money, and I want it

Knowledge is Power Program (KIPP) is a charter school system in the United States. Mathematica Policy Research Inc. has released a study on KIPP in relation to attrition rates and test scores.

The Washington Post summarized the study:

Mathematica studied 22 KIPP middle schools, including AIM and KEY, comparing test scores of charter students to scores of selected students in regular public schools who matched their academic and demographic backgrounds. Researchers examined test data starting in third grade. KIPP middle schools begin in fifth.


By seventh grade, half of the KIPP schools studied showed growth in math scores equal to an additional 1.2 years of school. Reading gains for KIPP were not as dramatic but still significant, the researchers reported, reflecting an additional three-quarters of a year of growth.


Mathematica said it found no evidence that KIPP schools were systematically drawing students with more economic advantages from surrounding school systems. But attrition rates at the KIPP schools, measuring the portion of students who failed to complete four years at the schools, varied widely. In a third of the schools studied, attrition was significantly higher than in other local public schools. In another third of the KIPP schools, the rate was lower. Skeptics say that students who can't function in the rigorous school culture are often pushed out -- a claim that KIPP rejects.

Despite KIPP's attempts to sell this study as a feather in their cap, Gary Miron, Professor and Director at Education and the Public Interest Center at the University of Colorado at Boulder, explains some shortcomings, omissions and distortions that arise from this study:


However, an initial analysis of the report by Professor Gary Miron of Western Michigan University concludes that this initial study report misrepresents the attrition data. According to Miron, "While it may be true that attrition rates for KIPP schools and surrounding districts are similar, there is a big difference: KIPP does not generally fill empty places with the weaker students who are moving from school to school. Traditional public schools must receive all students who wish to attend, so the lower-performing students leaving KIPP schools receive a place in those schools."



In contrast, Miron explains, "The lower performing, transient students coming from traditional public schools are not given a place in KIPP, since those schools generally only take students in during the initial intake grade, whether this be 5th or 6th grade."

The KIPP study's description of attrition only considers half the equation, when comparing KIPP schools to matched traditional public schools. The researchers looked at the attrition rates, which they found to be similar - in the sense of the number of students departing from schools. But they never considered the receiving or intake rate. Even though the researchers agree that the students who are mobile are lower performing, they do not take into account the reality that KIPP schools do not generally receive these students.

Professor Miron conducted his own quick analysis, using the Common Core database, and concluded that there is a 19% drop in enrollment in KIPP schools between grades 6 and 7 and a 24% drop in enrollment between grades 7 and 8. (This analysis only included KIPP schools that had enrollments in all three grades). In comparison, traditional public schools in these grades maintain the same enrollment from year to year.
Aside from the controversy around KIPP's attrition rates, I find it important to reflect on any study that uses test scores as it's primary tool of evaluating any learning environment. We know that measurable outcomes and standardized tests may measure some of the least important aspects of education, that standardized tests encourage a great deal of shallow thinking, and that standardized tests are at best unreliable indicators of anything that resembles good learning - so what does that say about any teacher, school or charter system that holds high test scores as a badge of honour?

Perhaps we should be more than a little skeptical of schools that owe their name to a chant by Harriett Ball: “…Knowledge is power, power is money, and I want it.”

5 comments:

  1. KIPP is the steroid of education - artificial inflation, big results in a purely competitive realm, but if it doesn't kill you, it's almost guaranteed to make you lose your balls.

    Sorry if that's too vulgar.

    ReplyDelete
  2. Often the truth is labelled vulgar by those who wish to avoid the conversation all together.

    I think you hit the nail on the head, John!

    ReplyDelete
  3. Perhaps you'd like to tell that to KIPP graduates succeeding in college who were, by their own account, on track to land in prison before they got into KIPP. See also the tearful parents who were unable to get their child into KIPP.

    Different children benefit from different types of education, and they deserve some choice.

    ReplyDelete
  4. I agree though, that attrition and self-selection are serious issues to consider when evaluating KIPP though.

    ReplyDelete
  5. Each individual school has their challenges - some have issues with attrition, some with self-selection, and I know they each work on this - but that is true of any school. Although KIPP schools do believe in testing, this is certainly not their only method of evaluating schools and students, as is immediately apparently once you visit a school or talk to some teachers or former KIPPsters. They simply believe that it is easy to convince yourself a child is doing well (or not doing well!) because of personal biases, and testing can act as a more objective check on that. And finally, although they did start teaching the saying as something else (I don't remember what exactly, but it now goes power is ____, not money, at least for the older students), if you really have a problem with that saying and its motivating power it makes me wonder if you've ever met a fifth grader. :)

    ReplyDelete