Thursday, May 7, 2009

Whither the causation?

I just finished reading Wars, Guns, and Votes: Democracy in Dangerous Places by Paul Collier. Actually, reading is not the right way to put it. I read the first chapter, became so amazed at the poor scholarship and then proceeded to flip through the rest of the book, looking for citations of his own work and looking closely at those. What follows is not a full critique of the book, but a look at a few semi-random pieces of work that he quotes in the book.

It seems that Collier still does not understand (or more likely care, as I will argue in a bit) about the problems of causation, as evidenced by his recent paper Democracy, Development, and Conflict (nor, one supposes, do the people at the European Economic Association, who published the paper). This paper is cited prominently in the first chapter. This is where my mind began to turn to mush.

The paper argues that "democracy ... constrains the technical possibilities of government repression, and that this makes rebellion easier", and purports to show empirical evidence of this for countries below an income threshold.

How does he show this? By simply regressing a measure of government repression and rebellion on lagged values of GDP and a democracy score. He takes no steps to control for reverse causality or other endogeneity beyond a lagged term, which does nothing to control for these things. He has come up with nothing but correlation, which tells us very little that is policy relevant, certainly not as much as he makes claim to in his book.

Then, to determine if there is an income threshold, he simply interacts democracy and GDP. To add to this, he clearly engages in data mining when coming up with his income threshold of $2,700. Why $2,700? In his book, even he admits that $7 a day is a rather arbitrary cutoff. (Easterly discusses some of the problems of data mining in a nice post today)

Here's two perfectly legitimate explanations of his results: (1) poverty is driving all of these mechanisms, not democracy, or (2) anticipated rebellion hurts an economy and the ability of governments to function well. We can't know which is the right interpretation, or if a third or fourth is the real reason.

While this paper may be important to motivate future research, to put it prominently in a book designed to influence policy seems rather dangerous, and innapropriate.

Then there is this paper he mentions, which looks at the effect of peacekeeping troop deployment on reversion to war. This is an important argument for the book as it ties back to previous arguments of his that military intervention is needed in some areas to prevent regression to violence. It relies on an assumption about UN peacekeeping, that it is basically random, that needs to be researched much further before being taken seriously. I have a hard time believing that the choosing of peacekeeping missions is done randomly. More like a dart board, where a target is chosen and a shaky hand throws. There'd still be a lot of correlation across reasons for mission targeting.

For those interested in this topic, I suggest a paper by Bernd Beber that uses UN recess timing to capture involvement in mediation processes, an arguably much better quasi-experimental approach. (Beber does find that mediation matters, but that is hardly the same as arguing that troops matter.)

Then there is work Collier claims to have done with Patrick McGowan from my Alma Mater, ASU. It smells of the same causality problems: tying coup incidence to democracy. I can't though find the paper anywhere online.

I think the problem is Collier is engaged in producing a lot of research to bolster his arguments and so suffers from sloppiness more than failing to understand the problems of causality and endogeneity. (Although, he does call the recent push for randomized trials in economics as a "fad", so maybe he believes the whole causality problem is simply a "fad")

One example that shows he is keenly aware of the problem of causality is a paper he does with Marguerite Duponchel, unpublished, on Sierra Leone, where he uses distance to the border with Liberia as an instrumental variable, similar to a paper by Richard Akresh Damien de Walque, who use distance to Uganda as an IV for the Rwandan genocide. It is the only reference to an IV that I could find in the whole book.

His previous book, Bottom Billion, did not suffer from these problems because he relied more on case studies. In this new book he seems to be trying to make a more generalizable argument, but fails to use good scholarship to do it.

I recently urged a reader of this blog to be cautious with regression models that do not have clear causality reasoning, such as simple lagged time periods. I think Collier, and anyone that reads this book, needs this advice too.

13 comments:

Lee said...

There are 2 pretty common comments I've seen on this book - 1 that the evidence is weak, and 2 that more specifically the evidence is too weak to endorse the policy recommendations.

I would argue that

1 - The evidence is indeed weak, but is arguably stronger than case study evidence. Rigorous economic analysis might be the firmest type of evidence going but it limits the range of questions which can be investigated. Collier is trading off between analytical rigor and interesting questions (although he doesn't tend to admit this).

2 - Academics are generally too timid rather than too bold. Bertrand Russell once said that “The biggest cause of trouble in the world today is that the stupid people are so sure about things and the intelligent folks are so full of doubts.”

Academics engage too infrequently in policy debate because their own standards of evidence are too high for them to feel confident enough in their implications. This is crazy. The vacuum is just filled by loud-mouth media commentators who base their ideas on almost no evidence or rigorous thinking whatsoever. Better to argue loudly on weak evidence than on pure prejudice.

BenRymer said...

I also just finished reading -- in the sense of 'reading' that you describe -- 'Wars, Guns and Votes', and found it to be a really frustrating experience. Collier seems to be intellectually agrophobic; he references his own work and that of a small handful of other scholars (Tim Besley is one of them) over and over.

This makes me wonder whether he has adopted a bunker mentality; the Bottom Billion and his Greed or Grievance work has been greeted with much indifference, and 'Wars, Guns and Votes' seems resolutely set on finding what it looks for, ie. evidence for the "over the horizon" military intervention Collier proposes.

One other footnote is that I heard a Collier lecture from two weeks ago at the LSE where he said that "public goods are economic goods", which got under my skin somewhat. Are human rights or public health provision essentially economic activities? I would argue not, for the most part at least.

I agree in part with Lee, in that development is about not making the better the enemy of the best, but we do need more academic debate on this. Conflict prevention and resolution is too important to leave to one man (or group's) quantitative findings.

Anonymous said...

thank you! That looks like a great resource.

web design India

Dr. Alden Kurtz said...

isn't Paul Collier the anglo-saxon pseudonym of Paulo Coelho?

zcocorporation said...
This comment has been removed by the author.
SITL said...

I have been meaning to read this and just never got a chance. Its an issue that Im very interested in, I just started reading and Im glad I did. Youre a wonderful blogger, 1 of the most effective that Ive seen. This weblog undoubtedly has some facts on topic that I just wasnt aware of. Thanks for bringing this stuff to light.
canada exporters
b2b trade leads
suppliers directory

Unknown said...

Your review lost a lot of authority when you said you only really read the first chapter...

Ideation Fashion said...

Nice share of this wonderful post. Thanks for sharing. Will come to visit again.
birkin bag

Unknown said...

Image this means to learn to read this kind of and by no means acquired a chance. Its a worry in which Internet marketing incredibly considering, I simply started out looking at as well as Internet marketing delighted I did so. Your an awesome doodle kit, hands down the best in which Ive witnessed.Best phytoceramide That blog site certainly features several information about subject matter which i just wasn't aware about. Thank you with regard to delivering these products to help gentle.

Admin Produk Nasa said...

any women experience vaginal discharge, vaginal discharge, and it is reasonable because it is certain that experienced by a woman. But whiteness is also not neglected, because whiteness is that of germs can cause serious problems in your female organs. Whitish an early symptom of cysts, myom and cervical cancer. Therefore let heal whitish and vigorous various diseases due to the whitish
and follow the steps
Crystal X Asli
Crystal X Asli Nasa
Jual Crystal X Asli
Jual Crystal X Asli Nasa
Jual Crystal X Asli Mengatasi & Menghilangkan Keputihan
Crystal X Jogja

Unknown said...

You are looking for herbal solutions to miss v? capable of dealing with vaginal discharge, itching in miss v, cure the infection and irritation of the miss v, v miss the freshness treating, curing cysts, to constrict you and miss v to prevent cervical cancer? Crystal X solutions handle, made ??from natural natural ingredients making it safe for use.
Crystal X Nasa
Crystal X Asli Nasa
Efek Samping Crystal X
Ciri-ciri Crystal X Asli
Entropy Cream
Bibit Bawor
Crystal X

Agus Putra said...

now comes herbal treatments that can help you solve problems with your female organs such as kepuithan, itching, myoma, cysts and odor. Crystal X present to help you to overcome these problems.
Crystal X Asli
Crystal X Merapatkan Vagina
Crystal X Mengatasi Keputihan, mioma dan kista
Jual Crystal X Asli
Crystal X merawat Vagina
manfaat Crystal X Untuk Wanita
mengatasi Keputihan
Cara Agar Cepat Hamil

Unknown said...

On this occasion I will explain Apa Itu Manfaat Payudara Kecil Wanita and Bagaimana Cara Memperbesar Ukuran Payudara..? payudara kecil actually very beneficial for women because women can be members freedom to perform various kegitana without having to feel uncomfortable because the breasts are rubbed or touched. Also in the field of women's fashion can benefit because they can wear almost all kinds of clothes and bra. But if you want to add a little or a lot ukurang breast. We can do a Olahraga Untuk besarkan Payudara such as push-ups, pull-ups, etc. In addition you can also use Ayla Breast Care Untuk Besarkan Payudara , This cream can accelerate the growth of the breast becomes firmer and larger. This method can NDA combine.
Apart from the appearance of the inside of the course should be treated not the True Way Vaginal care is crucial for intimate areas are not attacked by diseases such as vaginal discharge, itching "and odor. Even Kanker Serviks . But no need to panic now Cara Mencagah Kanker Serviks already available. Such foods do routine maintenance and using Crystal X . Manfaat Crystal X Atasi Keputihan and cervical cancer has been proven in addition to the Efek Samping Crystal X a positive effect, namely the effect penyemubuhan. Thus and thanks Crystal X