Bay Area KIPP schools lose 60% of their students, study confirms

Miracle solution or not?

A new study of the five Bay Area KIPP schools by the respected research firm SRI International confirms what we already knew: KIPP students overall perform well academically, usually outperforming their peers in other schools.

But it also confirms what those who look beyond the test scores have found: Those KIPP (two in San Francisco, one in Oakland, one in San Jose, one in San Leandro) schools suffer from very high student attrition. .

Sixty percent of the students who enter the Bay Area KIPP schools in fifth grade leave before the end of eighth grade  (page ix of the study, repeated in several places throughout). And the study also confirms what some might suspect -- it's consistently the lower performers who leave:

"On average, those who leave KIPP before completing eighth grade have lower test scores on entering KIPP and demonstrate smaller fifth-grade effects than those who stay," the study reports on Page ix.

To clarify one point that confuses some observers: Traditional public schools also have high turnover (called mobility in the education world). And high mobility is associated with the less-stable lives of low-income families. That is, families who move  often are more likely to be poor and lower-functioning -- meaning that their kids are more likely to be low achievers. But when students leave traditional public schools, they are replaced -- most likely by similarly high-mobility kids with similarly unstable lives. By contrast, when students leave KIPP schools, they are not replaced.

Thus, when 60% of KIPP students leave and they tend to be the lowest performers, to state the obvious, KIPP is left with the top 40% of the class. That's not what happens when students leave traditional public schools.

The study does not address the questions that information immediately raises:

• What  would the impact be on  the traditional public school down the street if its lowest 3/5 of achievers left?

• How much impact does that attrition rate have on the success of the 2/5 of students who remain at KIPP schools, if it could be separated from the impact of KIPP's distinctive culture , methods and practices?

Why did the students leave?

The study noted: 

"Although an in-depth analysis of why students (or their families) chose to leave the Bay Area KIPP schools—and how stayers and leavers experienced KIPP—was beyond the scope of this study, we did ask school leaders why students left their schools. Whereas most leaders noted that the schools lose many students to family moves, they also elaborated on the issue of fit. As one school leader explained:  I think for a cohort of students and families, it was harder than they thought it was going to be.
Our expectations were more than they had anticipated. [For example,] [w]hen we said we were going to give 2 hours of homework [a day], they didn’t really believe that it was going to be that much. " (Page 14; emphasis in the original)

One area the study would have looked at is the impact of KIPP's culture, methods and practices on the students who make it through eighth grade. But it couldn't, because the high attrition rate -- and the fact that the students who leave are likely to be the lower performers -- made that impossible, biasing the sample:

"We could not estimate longitudinal impacts because of student attrition and in-grade retention. Because of both the number of students who left and the fact that those who left are systematically different from those who stayed, longitudinal comparisons would be biased," the study stated  (page  ix).

The study confirmed two other points that have been raised about KIPP.

• KIPP schools cost more than traditional public schools: "...(I)mportant for sustainability, at least in the California KIPP schools, is a continued influx of supplemental private funding for operating costs. The Bay Area KIPP schools could not function without substantial resources above and beyond the state per-pupil expenditures they receive."  (Page 80)
• Because of the intense demands placed on KIPP teachers, faculty turnover is very high, raising questions about long-term sustainability of such programs:" Leading and teaching in a KIPP school are hard jobs, and turnover in the five Bay Area schools is high for teachers. ... How much turnover can KIPP schools tolerate and still retain the essence of their cultures? Over time, will the pool of candidates for school leaders and teachers continue to meet the schools’ needs?" (Page 80)

The study also confirms the observations that I and others have made about claims of KIPP's rate of alumni matriculation to college: The claims are based on a very small sampling. (I believe that they're largely repeated by journalists who don't grasp that fact.)

" Because college attendance begins 8 years after students enroll in fifth grade in KIPP, only students from the original two founders’ schools have reached college age. Those two schools, begun in 1995, report that 80 percent of their graduates have enrolled in college. Because most KIPP schools began in 2003 or later, large waves of potential college attendees will begin completing their senior year in 2011.' (Page 81)

I've already been interviewed about the study findings, as a "KIPP critic." It may sound like hairsplitting, but I really don't view myself as a critic of KIPP itself.

I'm a critic of the notion that KIPP schools have found the solution to educating the most challenging of our students. The study confirms that at least in those five schools, KIPP has succeeded in educating a high-functioning subgroup of the most challenging of our students, and we don't know how that subgroup -- roughly the top 40%, isolated from their lower-functioning  peers -- would have done in another setting. And we also don't know how a traditional public school would have fared in educating just that top 40%, if they could be isolated from their lower-functioning peers.

But the study does help dispel the "miracle solution!" myth, which I believe is an important start in moving toward real solutions.