The Department of Education Reform chair talks about philanthropic education-reform strategies—including the Gates Foundation’s—and learning from failure, or not.
Jay P. Greene is a distinguished professor and chairs the Department of Education Reform at the University of Arkansas in Fayetteville, Ark. He has long studied the various effects of school-choice programs, including in the forms of vouchers and charter schools.
Greene’s academic research was cited four times in the U.S. Supreme Court opinions in 2002’s Zelman v. Simmons-Harris case, which upheld the constitutionality of the Cleveland choice program’s inclusion of religious schools against a challenge that it violated the First Amendment’s Establishment Clause.
His other current scholarly interests also include schools’ effects on non-cognitive and civic values, as well as culturally enriching field trips. In addition to education, he has published work on political science, public policy, government, sociology, and public policy.
Previously, Greene has been a professor at the University of Texas and the University of Houston. He was also a fellow of the Manhattan Institute. He earned his bachelor’s degree in history from Tufts University and his Ph.D. in government from Harvard University.
He blogs at Jay P. Greene’s Blog and his Twitter handle is @jaypgreene. A recent provocative blog post about philanthropic failures in education reform, “What Do Education Reform Failures Have in Common?,” caught our attention at The Giving Review and, we thought, warranted further exploration.
Below is the second of two parts of an edited transcript of a discussion that Greene was kind enough to have with me in September. The first part—in which he talks about ed-reform philanthropy and that which animates it, as well as Teach for America (TFA)—is here.
At the beginning of this second part, after critiquing philanthropic support of TFA in particular, he addresses some of the difficulties facing grantmaking and grantmakers in general.
Greene: Most things are going to fail. Education reform is really hard. Philanthropy is really hard. Most things are going to fail. Everyone should be comfortable with that reality. It’s really important to learn from that failure, and I think philanthropies have not been set up as learning organizations, especially when they’re part of a very insular and driven worldview that cannot falsify itself.
Hartmann: You really do have sympathy for those in philanthropy, don’t you?
Greene: Oh, sure.
Hartmann: As opposed to contempt. I mean, isn’t there a lot of arrogance on their part?
Greene: Sure. I mean, look, it’s extremely hard to be a friendly critic of philanthropic organizations. There are tons of unfriendly critics out there. Those people are not trying to improve philanthropy. They’re not trying to have philanthropy be better. They would just like philanthropy to go away or be restricted as an independent source of power to compete with existing organized interests, like teacher unions. I’m actually trying to be a friendly critic, but it’s extremely hard because philanthropies have a hard time distinguishing any criticism from hostile criticism.
Hartmann: Are you skittish about “biting the hand that feeds you”?
Greene: I’m not trying to bite anyone’s hand. I’m trying to actually help their hand, but they don’t necessarily see it that way.
I’m a tough critic of my own organization, too. It’s not as if I’m harsher in thinking about how philanthropies are run relative to how my own department is run. We have lots of problems, too. I’m sure we can do better in lots of ways, as well. It’s healthy for organizations to be self-critical and open to the fact that they make lots of mistakes and try to learn from those mistakes.
A big difficulty is how hard it is for philanthropic organizations to learn from their failures. Part of it is this very insular worldview that they have all decided to share and coordinate on, so that they’re not able to see the shortcomings of that worldview.
Hartmann: Any examples?
Greene: I’ve been most openly critical of the [Bill & Melinda] Gates Foundation. They’ve gone through a series of big reform strategies without any obvious learning from each of these experiences. They’ve ditched promising reform strategies even when the evidence was encouraging. They’ve embraced others even when the evidence was discouraging.
That’s another part of the difficulty here. Despite all of the rhetorical commitment to evidence-based policymaking, they’re doing more of what Russ Whitehurst called policy-based evidence-making. There’s unfortunately a fair amount of intellectual corruption involved in these activities.
In particular, the Gates Foundation was pushing small-schools of choice, which seemed like it might be a reasonable strategy. They commissioned a randomized experiment to be evaluated by MDRC in New York City. Before that evaluation could produce results, they decided, based on I don’t know exactly what, that this wasn’t going well. The decision was made that they didn’t like the small school strategy anymore. So, they switched to their teacher-effectiveness strategy.
Ironically, the MDRC studies came out after they had abandoned small schools, showing that the small schools were actually doing a really great job and that that it was actually a promising reform strategy that they might have wanted to stick with and see where that went. When the MDRC study came out, they at first ignored the study. Even though they had commissioned it and paid quite a lot for it, even though it was about their own strategy, they did not put it on their website. They did not issue a press release about it. They buried it.
But word got out. MDRC itself made the report available. Eventually, the Gates folks were forced to confront it, but then they just reprocessed it to claim that small schools succeeded because they invested in teacher quality, so their switch to teacher effectiveness was still correct. This was a complete distortion of the evidence, but they had to rationalize the change in strategy that they had adopted.
The new teacher-effectiveness strategy, however, was both unlikely to be sound policy and politically impossible. It was unlikely to be sound policy because it depended on using top-down measures, like standardized testing, student surveys, and centralized teacher-observation rubrics to identify the most-effective ways to teach. The approach assumed that there were correct ways to teach that could be consistently identified across many different kinds of teachers, students, and communities using these multiple measures.
The whole thing fell apart, almost immediately. These three ways of measuring teacher effectiveness don’t correlate well with each other. They thought that classroom observations, student surveys, and student testing would all basically point in the same direction to identify what works in the classroom. As it turns out, they didn’t. There was almost no relationship between these three different ways of measuring what a quality teacher was.
Rather than saying, gee, perhaps what constitutes quality teaching differs in different contexts—that something might be good in achieving one educational goal, but bad at promoting another; that some teachers might be good at one thing, but bad at others; that some approaches might be good for certain kids and bad for other kids—they just kept pressing to find the correct way to teach, because that was the commitment they had made.
But the Gates strategy doesn’t collapse because of its policy failure. The political failure is the thing that does it in. Once Gates stopped paying the school systems involved in the effort to adopt this strategy, they quickly abandoned it. Teachers and local communities leaders had mobilized against it because it interfered with their autonomy without offering them any tangible benefits. That is, it failed to appeal to their self-interest. And local educators and community leaders had the wisdom and experience that the foundation folks lacked, so they were disinclined to believe that there were uniform, correct ways of teaching that they should impose.
Again, this goes back to my core belief that education is an extension of parenting. Being good as an educator, like being good as a parent, may require different things for different kids in different situations. Given that, these top-down solutions of finding the single, right way are doomed to fail. Instead, any sensible strategy, both in terms of policy and politics, will have to empower local actors to find what works in their various, particular circumstances.
I don’t think the big foundations like this idea, because they want there to be uniform answers that can be applied everywhere. That uniformity allows foundation folks to plan out how the solution will work, allows them to feel in control, allows them to “scale” a single solution rather than having to pursue many solutions, and allows them to say that they are promoting equity by doing the same for all. If instead they decided to empower people to find their own solutions, they would no longer feel like they were the ones devising solutions, that they were the ones in control, and that they were righteous in their pursuit of equity through uniformity.
Hartmann: I’m just checking. Have you been supported by Gates during your career at some point?
Greene: I have, yes. While these difficulties are most noticeable at Gates, almost all of the foundations seem to share at least some of these similar problems. What’s interesting is that these foundations don’t just have this scientific-management approach to schools, but philanthropies also have a scientific-management approach to their own organizations. They’re very metric-based in how they manage philanthropy.
This metric-based management, frankly, is remarkably similar to how the Soviet Union was managed. They sit in retreats where they develop their five-year plan in which they set their goals. In the Soviet Union, this was called the Gosplan. Then, they develop metrics for measuring progress toward achieving those goals.
It all feels very organized and scientific, but the problem with central planning, whether in the Soviet Union or in philanthropies, is that no one has enough information to know what goals everyone else should aspire to meet, what unintended consequences might result from narrow and distorted measures of progress toward those goals, or what new opportunities or challenges might present themselves before the next five-year plan will be promulgated. There’s nothing inherently wrong with organizations having goals and plans, but the highly centralized way in which these things are developed and the rigidity with which they are enforced can lead to things going horribly wrong, as they ultimately did in the Soviet Union.
Sizeable foundations, serious funds, some failures
Hartmann: Are there any good examples of philanthropies that have learned well from failure?
Greene: I’d have to think for a second ….
Hartmann: You just gave your answer.
Greene: I think among the big foundations, it’s not something that they’re really strong at.
I do see models of successful philanthropic organizations. I read Ron Chernow’s biography of John D Rockefeller. I was particularly interested in his philanthropic efforts. The things that he did and how he approached it were very different from how modern philanthropists approach things. In particular, I think a big strength Rockefeller had is he had a lot of experience managing large, complex organizations that included a lot of people from different backgrounds with different goals.
He didn’t even have a single organization. He had a series of trusts that he had to manage. He couldn’t impose top-down solutions in his business, so he wasn’t inclined to do the same in his philanthropy. For both, he had to get people on board so that they believed it was in their interest to work together on something. His skills in politics, or at least the politics of his organization, were actually extraordinary.
The technology-based sources of wealth are not producing people with that same skill set of understanding a broad range of people from a lot of different backgrounds and how they can be mobilized towards common goals. Compared to Standard Oil, technology companies have fewer employees who tend to be more homogenous in having high skills and sharing an elite worldview. Great success has also come to the new high-tech philanthropists at a younger age, which undermines the wisdom and experience they would need for philanthropic efforts.
I do look back on [The Lynde and Harry Bradley Foundation’s first president] Mike Joyce as an example of success in philanthropy. He had real political skills. He had ideas about how you could achieve political goals. He wasn’t trying to publicly brag about his righteousness, but he was trying to accomplish a political goal and had experience and wisdom in doing that. It’s hard to think of his current counterparts.
Hartmann: Now, when you say politics there, and when you used it before, you mean politics as broader than merely electoral politics, I assume?
Greene: Exactly. Getting stuff done.
Hartmann: Dealing with human beings.
Greene: The broad definition of politics is that it is the answer to the question of who gets what, when, and how. It’s a process by which resources are allocated.
Hartmann: Which, for a philanthropy, is relevant.
Greene: That is what they do.
Hartmann: In philanthropy, is education reform harder than supporting the arts or whatever else might be on the menu, be it low taxes or small government or whatever?
Greene: Oh, it’s most definitely harder. It’s harder because it does not lend itself easily to single, scalable, technical solutions. Some other kinds of problems do. I think philanthropies tend to be more effective at addressing those other problems.
Hartmann: What other kinds of problems are you thinking of?
Greene: Distributing mosquito nets to reduce the incidence of malaria. It’s relatively straightforward. But even that is somewhat complicated, because you have to get the mosquito nets ultimately to the people who need to use them. That is not merely a technical issue and does require political skills. I think there’s been a mixed track record. We focus a lot on the good intention of doing things, as opposed to the actual end result of succeeding in doing it.
Hartmann: So overall, you would be for pluralism in grantmaking—a lot of options, let a thousand flowers bloom—and what some might call “philanthro-localism.”
Greene: That’s right. I think that that your solutions are likely to benefit from contextual knowledge. This obsession with scaling solutions has been very unhelpful, because often there aren’t scalable solutions. Understanding one’s situation and the nature of the problem in your community is incredibly helpful for effective philanthropy.
There may just be problems inherent in the gigantism that has crept into philanthropy, but there have been giant philanthropies in the past, and they have managed to avoid some of those problems. I think it’s possible to do big philanthropy, but it’s a lot harder than to do community-based philanthropy.
Hartmann: The giant philanthropies of the past were probably headed by people who were drafted into the army. It was different—less elite, less credentialed?
Greene: Well, they did depend less on technical expertise and were more explicitly value-driven. They believed in certain values that they were promoting. They also had, again, greater political acumen.
The other thing they did, which I think was incredibly important for their success, is they didn’t try so much to convince other organizations to change how they did things. They built new organizations based on what they thought would be better. Then, they had those new organizations compete to see which ideas could work in the world.
If we look at the example of reforming higher education in the late 19th Century, most elite American universities were still ministerial training schools. People like Rockefeller, but also Stanford and Vanderbilt and Carnegie, they thought the United States needed to shift to have more universities that were built on the German scientific model. The thing they didn’t do was make a grant to Harvard to build research labs. That’s not what they did. Instead, they built new universities, and then they had those new universities compete with the old ones. Harvard, Yale, Princeton and others saw the competition and adapted.
Building new institutions is something that big philanthropy can do very effectively. That approach has the added benefit that there is ultimately a real-world test of whether those new institutions are good or not. They either succeed in the world or they don’t, and some don’t and they go away.
It strikes me as unusual that of all the big philanthropies out there, none of them is actually investing in building new institutions that could model their preferred approach and values. They could. They have that scale. They could build new universities, They could build new ed schools. They could build chains of K-12 schools. Not everywhere, but they could do it in certain places. They have that scale. But they generally aren’t.
Hartmann: It’s funny. As you’ve been speaking, I’m thinking of links I’d like to send you that agree with what you just said. Then I’m like, well, this “we’re-right” attitude is exactly the type of insularity about which you’re so properly skittish.
Greene: You know, of course everyone’s going to think they’re right. There’s nothing wrong with thinking you’re right. The thing is, you just have to have the possibility of being told you’re wrong by somebody. If philanthropies build new institutions that could fail, that is actually an openness to the world telling them they are wrong.
The old joke with people who work for philanthropy is, “Congratulations. Now that you work for us, you never have to buy a meal or be wrong again.” This is told as a joke, but like most jokes, it contains an important element of truth.
Hartmann: It’s fine for the person who hears that joke. They should, of course, discount what’s being told to them all the time. They shouldn’t lie to themselves about how smart they are. There should be more humility. You’re calling for more humility, right?
Greene: Yeah, but I mean that’s like calling for more goodness. We’re not inclined to think that we’re bad, and we’re not inclined to think that we’re arrogant. I think the world has to humble us. One of the difficulties with the strategies that big philanthropy is pursuing is they don’t really set it up so that the world could humble them. They really don’t want to be embarrassed in the world. They don’t want to be seen as failing.
What’s more is, internally, there seem to be consequences to failing in some of these philanthropies. If you make the wrong bet, you and all of your associates are gone. That’s a very unhealthy organizational culture because it does not encourage intellectual honesty and learning in an organization.
Hartmann: Or risk-taking or entrepreneurship, as they would all say they’re engaged in right now?
Greene: There’s a lot of bureaucratic compliance with the internally established, five-year Gosplan. Much like the Soviet Union, they will misreport their steel production. Who’s going to call them out? Who wants to admit to the Politburo that you’re failing?
Bold great leaps and tentative small steps
Hartmann: Well, okay, you’ve gone after Gates. They can handle it. Anyone else?
Greene: I think [the Laura and John] Arnold [Foundation, now Arnold Ventures] is another organization that has proven quite frustrating to me in their rhetorical commitment to scientific, evidence-based policymaking, but their actions are totally inconsistent with that. It’s tough, because I think John Arnold’s a good guy. I think he’s trying very hard. He’s very involved and is very smart. The trouble is that he is not as driven by scientific research as he thinks and is actually so committed to a particular vision of the world that they have not set things up so that the world can humble them.
Even when they fail, they think they’ve won. I don’t know what it would take for them to feel like their “portfolio-management” strategy is not working out—it’s not being adopted, it’s being repealed where it exists, it’s not producing improvements in other places where it’s tried. I don’t know what more you want.
Hartmann: Have you found that either liberal or conservative philanthropies are better at avoiding or learning from failure?
Greene: I know the conservative ones a lot better, but no. The liberal ones, to the extent that I know their work, don’t strike me as much better. A number of these organizations are kind of stuck in 1960s theories of poverty and social policy. A half-century of frustration and failure seem to have very little effect on them. Philanthropic ideas may fail, but the philanthropies never have to acknowledge failure.
Hartmann: Isn’t some incrementalism defensible?
Greene: Ultimately, the world is incremental. These great leaps forward that philanthropies say they want and are scalable, those are probably dangerous ideas that lead to unhealthy strategies.
This was true of school choice. We don’t know the right way to structure and regulate school-choice programs. We don’t, and there might not be a right way. There are probably a lot of different right ways in a lot of different circumstances. It seems to me like we’re going to have to try a lot of different things, and some things are going to work and some things won’t, and we’ll make incremental progress on that.
I don’t envy the people who have to think about all these things and make decisions. I understand how hard that is. I just don’t think that they have been structured effectively as learning organizations, nor are they very open to criticism or input. They’re built in every way possible to keep outsiders away, because they don’t want to be inundated with everyone asking them for stuff.
There’s some balance there between insulating oneself from being overwhelmed by requests and being an organization that’s open to input. Look at the structure of how grantmaking occurs. The way it works is if I had a really great idea, I can’t just go to a philanthropic organization and say, I’ve got a really good idea. I’d like to convince you of it, so that you should support it. Instead, they say, Is it aligned with our strategic priorities? And if it isn’t, we don’t want to hear about how great your idea is, because it’s not aligned with our strategic priorities. How do we develop those strategic priorities? We met in a retreat with a bunch of consultants who are our friends, and we wrote a five-year Gosplan.
This is not a good way to receive input and learn.