The interesting question in development is not whether aid works or does not work. Not surprisingly, the answer is that some aid works and some doesn’t. A more interesting question is: what kind of aid works best?
Nick Kristof has a good article (if a little simplified) in the New York Times today about randomized trials, which he describes as ‘the hottest thing in the fight against poverty’. This new wave of rigorous evidence about impact is helping us to understand which policies and programmes in developing countries work well (whoever pays for them) and which do not.
I especially enjoyed his digression about the importance of economists:
When I was in college, I majored in political science. But if I were going through college today, I’d major in economics. It possesses a rigor that other fields in the social sciences don’t — and often greater relevance as well. That’s why economists are shaping national debates about everything from health care to poverty, while political scientists often seem increasingly theoretical and irrelevant.
Economists are successful imperialists of other disciplines because they have better tools. Educators know far more about schools, but economists have used rigorous statistical methods to answer basic questions: Does having a graduate degree make one a better teacher? (Probably not.) Is money better spent on smaller classes or on better teachers? (Probably better teachers.)
I suspect not everybody will agree with this.
21 responses to “Do economists have better tools?”
Hmm. Perhaps different disciplines just have different, possibly complementary, advantages. Sure, economists are able to answer many interesting questions. But there are also some limitations to their methods.
For example, economists often approach intrahousehold relations using a bargaining framework, thinking about game theory and exit options. But I’m not sure this model can explain recent changes in gender relations in Zambia, from where I’ve recently returned after a year’s research.
Here many men explained that since they lost their formal sector jobs in mining and manufacturing and their wives (who used to be housewives have started trading) they have come to appreciate their wives so much more, in recognition of their hard work and newly developed capacities (greater confidence and more ideas having travelled further afield, associating with different people). Most men whose wives are traders, for example, said they respect and make decisions with their wives so much more now because of appreciation, struggling together in a tricky world. An economist’s bargaining framework focuses on conflict and threat but I didnt hear those kinds of narratives.
Also, if an economist wanted to understand why some people come to think that women can do what men can do (as I did) they might test various variables and note significant associations, but I don’t think correlation shows cause. Much better to befriend, understand, develop trust with and ask a socio-economic range of people to narrate their lives and during these conversations ask how different experiences they refer to (such as co-education, mother’s livelihood, role models, household financial insecurity etc) affect their confidence in women to undertake work traditionally done by men.
I’m not saying it’s ‘better’ to enable people themselves to highlight what was important in their own lives, but it might be an equally important pathway to understanding how change happens. I think the technical term is: Q-squared
P.s. The ‘poverty’ equivalent for the kind of work I’m doing on understanding changing gender roles and relations is perhaps best found at the Chronic Poverty Research Centre – they’ve collected life histories of poverty dynamics, quantitative data on poverty levels and also map out macro-economic change over time, thereby harnessing a range of methodologies in an attempt understand how different people move in and out of poverty.
http://www.chronicpoverty.org
I was also reading the comments and they demonstrate that the financial crisis has caused a serious blow for the reputation of economists, and not just for the macro economists and finance guys that actually focus on real exchange rates and inflation figures.
It is quite sad to see that the financial crisis causes these problems for economists that work on a solution to make people vaccinate themselves as well. These people seriously challenge assumptions associated with rationality, although admittedly this opinion might be an example of self-serving bias since I am in this business myself. I guess development economists should not only worry about good research but also about their reputation to make sure that the general public still take their research seriously. This might actually be important for the dissemination of research findings to policy makers as well.
I would say that if there is one discipline in social science that knows the difference between correlation and causality it would be economics. We certainly do not have that reputation yet and we have to work on it. However, I agree with the limitations of methods that economists use. Bargaining power might be an example of a topic where economists can learn a lot from other disciplines.
I’m with skeptical secondo….
Great tools, we economists undoubtedly do have. In studying development issues, they are often used unhelpfully due to hubris and a shocking level of comfort with ignorance about the phenomenon being studied.
I’ve tried to resist, but can’t:
Three guys are alone on a desert island: an engineer, a biologist and an economist. They are starving and don’t have a thing to eat, but somehow they find a can of beans on the shore.
The engineer says: “Let’s hit the can with a rock until it opens.”
The biologist has another idea: “No. We should wait for a while. Erosion will do the job.”
Finally, the economist says: “Let’s assume that we have a can opener”.
NB: I know that making assumptions is key to making models and that models are essential 😉
Here’s the thing – RCTs are not new or unique to economists, they are old, and borrowed from medicine. Global health is also the major success story of aid and development over the past 60 years. Coincidence? http://www.cgdev.org/section/initiatives/_active/millionssaved
Back in the day when I studied Pol. Sci., I was taught that Machiavelli was a smart man due to his commitment to realpolitik; when studying economics, I have been taught that Machiavelli is an idiot due to his inability to make credible commitments due to his realpolitik.
It is thanks to the Freakonomics guys that people think RTC’s are a tool of economists. As Lee has noted, they have been used in medicine since forever. Their use in development can be helpful when used appropriately, but they are certainly not the magic bullet that some people seem to think. They are simply one (very rigorous) form of evaluation, but they do not generate solutions. And sometimes even when they provide evidence of whether something has worked or not, they are still unhelpful in understanding why the intervention did or didn’t work. Moreover, it is not clear to me that one can ever stop doing RTC’s. At least in medicine, when RTC’s prove the efficacy of a drug, patients can apply the solution without having to redo RTC’s. In development, just because an RTC shows an intervention is effective in Mexico, doesn’t mean that it will be equally effective in Ethiopia. You would need more RTC’s for every context.
Economics used to be – maybe still is – the dismal science. Not everyone makes decisions on rational economics either.
As for its role in development – clean water and sanitation are a long way ahead of economics in making real gains in peoples lives.
We’ll all take economists much more seriously when they all warn us in advance about the next credit crunch/global financial melt-down. Economists are great at post hoc but you need to work a lot harder on ante hoc.
Development economists seem to be particularly stuck in the past. You need to have a word with behavioral economists: your rational assumptions about what happens to aid bear little relation to what’s going on in the real world.
Yes Owen, I must disagree. And I would venture that when Kristof was in school, the anlaytical tools of economists were yet to be fully honed. I find it humorous and unhelpful to pit one discipline against the other, especially when they often study different subject matter. Both economists and political scientists have a dismal record of predicting the future, but this is not due to the rigor of their tools. And as others have noted, RCTs have been borrowed from the medical sciences, and I would add, have long been employed by political scientists in the comparative politics subfield. Isn’t careful case selection, done in order to isolate the variable(s) of interest, the essence of RCT? That’s something that comparativists have been refining for quite some time. RCT will also have to face what those in political science face — small n studies that provide interesting findings but lack robust explanatory power. As a Poli Sci-er happily ensconced in a think tank filled with top-notch economists, I suggest that our research would be enriched if we tackled problems together, each bringing our own research approaches to the table.
Following Alan’s lead, I also couldn’t resist:
Three econometricians went out hunting, and came across a large deer. The first econometrician fired, but missed, by a meter to the left. The second econometrician fired, but also missed, by a meter to the right. The third econometrician didn’t fire, but shouted in triumph, “We got it! We got it!”
Thanks a lot for the last joke! Love it!
Economists, just like political scientists, struggle to get their findings into policy making – irrespective of whether these findings are based on RCTs or not. The study by Kremer and Miguel was published in 2003 and worms are still on the WHO list of neglected diseases. Reasonable and fairly straightforward recommendations on distributing deworming drugs in schools are not implemented on a large scale by donors and partner countries. Why?
Another problem facing economists just like any other scientific field is that existing knowledge is not aggregated sufficiently and translated into policy formulations. In many areas both at the macro level (trade, corruption, climate) and the micro level (how to increase agricultural production, how to enhance school performance) there is at times an abundance of knowledge, but knowledge management across thousands of researchers, organisations, projects, etc. is challenging.
So, I really disagree with Kristof’s enthusiams. And “No”, economists don’t have better tools. They have complementary tools.
[…] drivers of the use of RCTs, rather than just being consulted or included in them. As a commenter on Owen Barder’s blog shared, “Great tools, we economists undoubtedly do have. In studying development issues, they are […]
[…] Owen Barder: “Do Economists Have Better Tools?” […]
Sure, economics is useful, but in the case of development, I don’t think we lack good ideas derived from theory. I think we could get much more marginal return from shifting resources into community dialog, that is spend more time talking to the poor about what they think they need, instead of us trying to impose policies based on our predictions about their behavior. The book “Portfolios of the Poor: How the Worlds Poor Live on $2 a Day” was one of the most enlightening development books I have read, and it was derived in this fashion. We also need more research into how good ideas should be implemented given existing power arrangements.
I suspect Kristof didn’t get particularly rigorous training in political science, which is not surprising given his age and the ways that the discipline has evolved since that time.
I’m a methodological universalist and believe that we need as many approaches as possible to fully understand most social problems, so I don’t have a problem with economists or their work. That said, I do think the contextual understanding that political scientists can provide is extremely valuable, particularly if we want to understand the why’s and how’s of our findings.
Economics, like accounting and a tank, can be an effective tool in the hands of the right people (i.e., those who haven’t spent their lives studying economics or Ivy-league political science). However — like tanks in a military dictatorship — when it turns into an imperialist ideology (neoclassical economics) like Kristof and Ayn Rand wish it to, then it simply destroys all that oppose it (e.g. the environment, humanity, etc.).
[…] This in turn had Owen Barder asking whether development economists have “all the best tools?” […]
This article makes me think Owen might have an excellent point:
What Matters (and What Does Not) in Households’ Decision to Invest in Malaria Prevention?
By Pascaline Dupas
American Economic Review: Papers & Proceedings 2009, 99:2, 224–230
http://www.aeaweb.org/articles.php?doi=10.1257/aer.99.2.224
A huge range of unanswered questions are about the drivers of HH level malaria decisions…