Jeff Marlow writes in the New York Times about Koraro, a Millennium Village Project village in Northern Ethiopia:

As the project’s first five years wind down, its ultimate goals remain elusive, and the five-year initiative has swelled to 10. The extension, naturally, will require more spending: The financial injections to date—over $5 million per year in a mix of cash and non-cash contributions—have not abolished poverty. Improvements in the five sectors targeted by the MVP are readily apparent, but their sustainability is still up in the air.

There are many people in the development set who are sceptical about the utility of the Millennium Village Project, for good reasons and for bad.  Village-level interventions have had a chequered past, and the conventional wisdom today is that development assistance should help to build capable and accountable states which can deliver services, from agriculture and education to security and health, and not provide these separately from the systems that are being established.

I don’t know as much about Koraro as Jeff, but G and I did visit the town, unannounced, one day when we happened to be driving past.  We struck up a conversation with a local shopkeeper which went like this:

O&G:                 What is it like being a Millennium Village?

Shopkeeper:    Very good. We have lots of things.

O&G:                 Does everything work well?

Shopkeeper:    No, not all of it.  But we are much better off now.

O&G:                 Who decides what to change? Do you have a village council, or is there an Elder who decides?

Shopkeeper:   It is all decided by a Professor in New York.

O&G:                 Really? Do you know his name?

Shopkeeper:   No. But he is a very famous man

I don’t have the same ideological objections to Potemkin Villages the Millennium Villages Project as some other people. As both Jeff Sachs and Nick Stern have arged, it seems plausible that there may be significant complementarities between interventions which mean that programmes work better if there are other successful programmes at the same time.  For example, there may be little value in increasing agricultural productivity to generate surpluses if there is no way to get those surpluses to market, which requires infrastructure.  That suggests that each community may need a big heave:  ensuring that all these things come in together may be more effective than a series of uncoordinated interventions spread thinly.

For me the most disappointing aspect of the Millennium Villages Project has been the steadfast refusal to subject it to rigorous evaluation.  (Their evaluation programme is described here.)  The most detailed study so far has been conducted by the Overseas Develoment Institute.  The problem is lack of a proper basis of comparison.  Ethiopia is changing quite rapidly, and Korkora would have changed with or without the Millennium Village Project. For example, there has been a 51 percent reduction in malaria cases in Koraro, Ethiopia. This has been touted as a success by the supporters of the project; and it sounds impressive until you find out that malaria cases have been more than halved across the whole country, not just in Koraro.  The improvement in the Millennium Village is apparently no greater than anywhere else in the country.

To evaluate the project, Millennium Villages need to be compared with some suitable control group, ideally through randomised controlled trials.   Ideally, the individual components of the project would also be randomised to test the hypothesis that the effects of interventions are complementary.  (It follows that I don’t agree with Chris Blattman’s view that it would be too hard.)

It would, as Chris says, be pretty surprising if the Millennium Villages Project does not make a difference. After all, it is spending money roughly equivalent to 100% of the villagers’ income. Furthermore, it has benefited from close personal attention from the Prime Minister, other ministers and officials, researchers and academics (and, of course, a famous Professor from New York).  A rigorous evaluation would help us to know how big that improvement is, and and what cost.  It might also give us insights into whether any particular parts of the progamme are particularly important.