Are education donors doomed to fail?

Bill Gates’s giving to K-12 schools has been widely criticized, and he himself has admitted some missteps, as noted in my last post.

But the truth may be even worse: Could it be that nearly all donors in K-12 are just not up to the task?

Education donors assume they are wise enough to identify worthy projects and schools that can transform private dollars into significant improvements in students’ learning. But a recent study suggests that philanthropists in general may lack the needed wisdom -– or, perhaps, courage.

The study comes from Andrew Coulson of the Cato Institute, who looked at a funding strategy attempted by nearly all serious education reform donors, including Gates: investments in charter schools.

Charter schools are public schools freed from at least some of the hindrances suffered by conventional public schools. Donors hope that charters will outperform their public school peers, and that the best charters can be scaled up to serve more students.

Coulson studied California, which has more charter schools and charter school networks than any other state. He aggregated data on private giving to California charter school networks over the past eight years, and compared that to the networks’ achievements on the state’s standardized tests for 2010. His analysis controlled for factors like students’ socioeconomic backgrounds.

The good news: Some charter school networks perform far better than conventional public schools, and there’s no evidence a charter network’s increasing size harms its academic performance. So we conclude that if donors help the best networks to scale up, education can be significantly improved.

The bad news: Most of the quarter-billion dollars that funders, including Gates, pumped into California charters these past eight years was invested with little regard for academic performance.

Of California’s 68 charter networks, the ones that ranked 1^st, 2^nd, and 3^rd in terms of test scores ranked 21^st, 27^th, and 39^th in private funding. All told, Coulson found no statistically significant correlation at all between a network’s performance and the grants it received. In fact, Coulson found you’d have a better chance of predicting a school’s grants by counting the number of letters in its name than by looking at its test scores.

That’s the summary picture for all funders. The stories for individual donors are more complicated.

Unfortunately, Coulson didn’t rank donors by bang for the buck; so one can only make rough judgments, but some conclusions suggest themselves. Take the Gates Foundation and the Eli and Edythe Broad Foundation, the two wealthiest donors to California charters. Both have been significant national funders of KIPP schools, which in California and elsewhere have performed far better than conventional public schools.

Both foundations have also lavished millions on the California efforts of the Aspire and Green Dot charter networks, which explains why those schools ranked 1^st and 2^nd statewide as recipients of private grants. Yet Coulson’s study shows that Aspire performed only moderately better than conventional public schools, while Green Dot’s network was indistinguishable from the poor performance of its conventional peers.

In more precise statistical terms, Coulson’s study compared the average effect that a charter network had on students’ state test scores, adjusted for individuals’ socioeconomic backgrounds and school-wide peer effects. The size of a network’s effect was measured in terms of “standard deviations” from the statewide mean for all students. Statisticians call an effect of around 0.2 standard deviations “small”; one around 0.5, “moderate”; and one around 0.8, “large.”

Coulson found Aspire schools could boast of having some effect: they achieved 0.4 standard deviations’ improvement. Green Dot, however, achieved an effect of only 0.04.

California’s KIPP schools, by contrast, made an enormous difference in students’ lives, having an effect on test scores of 1.44 standard deviations. Kudos to KIPP, and to Gates and Broad for supporting it.

But note that although the performance gap between KIPP’s California schools and the statewide mean is large, an even larger gap exists in the other direction, between KIPP and the highest-performing schools in Coulson’s study, the American Indian Model schools. Their effect on students’ scores was a staggering 4.21 standard deviations, and Oakland Charter Academies, the second-ranked network that uses an extremely similar school model, scored 3.76.

To grasp how amazing these achievements are, consider that low-income black and Hispanic students at American Indian schools outperform the statewide averages for middle- and upper-income whites and Asians. And lest you sneer at the quality of California’s state tests, Coulson also studied Advanced Placement (A.P.) tests among California schools and found that American Indian schools left the competition even further in the dust by that measure.

Despite the American Indian and Oakland Charter networks’ achievements, they both received very little private money, and none from Gates or Broad.

On the other hand, consider the far smaller Koret Foundation, which has assets of roughly $400 million, compared to the Gates and Broad foundations, which measure their assets in billions. (Disclosure: I know Koret’s giving well because they once briefly hired me to review their history of education funding.)

Koret stresses the value of bringing market forces to bear on K-12 schooling, and so as soon as charters became legal in California, it began supporting charter entrepreneurs in its San Francisco funding area. As the “market” for charters developed, Koret carefully reviewed the schools it funded and directed its support to the very best models, which meant investing in KIPP schools and then American Indian and Oakland Charter Academy schools. Indeed, Koret was the first private funder of American Indian schools. If Coulson ever crunches his numbers to rank donors according to the return on their charter school investments, I suspect Koret will easily outpace its peers.

Why does this much smaller foundation have so much more to show for its giving than Gates and Broad? I’d guess two reasons explain the disparity. First, Koret focused on academic results above everything else, including the speed at which a charter network could scale up. Second, Koret’s staff and its entire board – led by such no-nonsense figures as Tad Taube and Michael Boskin – courageously supported charter networks even when the no-nonsense leaders of those networks came under fire.

Compare that to, say, Bill Gates’s admission to the Wall Street Journal that school vouchers would work but won’t receive his support because they are too unpopular:

honestly, if we thought there would be broad acceptance in some locales and long-term commitment to do them, they have some very positive characteristics…. We haven't chosen to get behind [vouchers] in a big way, as we have with personnel systems or charters, because the negativity about them is very, very high.

And so it seems that courage and wisdom are not abundant in education philanthropy. Does civil society have any other ways to bring dramatic improvements to our schools, which underperform not only for the poor, but also for the middle and upper class? (See “Not as Good as You Think” by Lance Izumi, the Koret Senior Fellow at the Pacific Research Institute.)

Additional research by Coulson and others finds that, where market forces are brought to bear in K-12 education, excellence is identified and substantial money flows into scaling up superior models (for example, in Sweden and Chile; Coulson surveys more international findings here). Entrepreneurs don’t worry much about their popularity with their competitors, and they and their investors keep a keen eye on the return earned by their investments – not only in terms of dollars but also in terms of students' progress.

As education scholar Jay Greene concludes in his Education Next discussion of Coulson’s charter study:

foundations are no substitute for market forces in identifying what works and what doesn’t for kids. Rather than focusing on picking winners and losers, foundations should focus on pushing the idea that we need stronger market forces. In particular, foundations could back the idea that we need a broader set of options for students (including charters).... ... if [donors] start focusing the bulk of their money on picking winners and losers in the educational marketplace, they are very likely to get it wrong. Central planning doesn’t work any better for foundations than it does for governments.

FOOTNOTE: Jay Greene has also weighed in with two interesting critiques of Bill Gates’s education investments here and here. Rick Hess and Michael Petrilli discussed the Coulson study on an Education Gadfly podcast. From the anti-market forces side of school reform, the New York Times recently raised its eyebrows on Gates’s funding of advisors for New York State’s education bureaucrats. It seems those advisors – gasp! – may support the idea of evaluating teachers and principals with some reference to whether the students they’re paid to teach actually learn. Peter Meyer of the Fordham Institute defends Gates here.

---