Whither the causation?

I just finished reading Wars, Guns, and Votes: Democracy in Dangerous Places by Paul Collier. Actually, reading is not the right way to put it. I read the first chapter, became so amazed at the poor scholarship and then proceeded to flip through the rest of the book, looking for citations of his own work and looking closely at those. What follows is not a full critique of the book, but a look at a few semi-random pieces of work that he quotes in the book.

It seems that Collier still does not understand (or more likely care, as I will argue in a bit) about the problems of causation, as evidenced by his recent paper Democracy, Development, and Conflict (nor, one supposes, do the people at the European Economic Association, who published the paper). This paper is cited prominently in the first chapter. This is where my mind began to turn to mush.

The paper argues that "democracy ... constrains the technical possibilities of government repression, and that this makes rebellion easier", and purports to show empirical evidence of this for countries below an income threshold.

How does he show this? By simply regressing a measure of government repression and rebellion on lagged values of GDP and a democracy score. He takes no steps to control for reverse causality or other endogeneity beyond a lagged term, which does nothing to control for these things. He has come up with nothing but correlation, which tells us very little that is policy relevant, certainly not as much as he makes claim to in his book.

Then, to determine if there is an income threshold, he simply interacts democracy and GDP. To add to this, he clearly engages in data mining when coming up with his income threshold of $2,700. Why $2,700? In his book, even he admits that $7 a day is a rather arbitrary cutoff. (Easterly discusses some of the problems of data mining in a nice post today)

Here's two perfectly legitimate explanations of his results: (1) poverty is driving all of these mechanisms, not democracy, or (2) anticipated rebellion hurts an economy and the ability of governments to function well. We can't know which is the right interpretation, or if a third or fourth is the real reason.

While this paper may be important to motivate future research, to put it prominently in a book designed to influence policy seems rather dangerous, and innapropriate.

Then there is this paper he mentions, which looks at the effect of peacekeeping troop deployment on reversion to war. This is an important argument for the book as it ties back to previous arguments of his that military intervention is needed in some areas to prevent regression to violence. It relies on an assumption about UN peacekeeping, that it is basically random, that needs to be researched much further before being taken seriously. I have a hard time believing that the choosing of peacekeeping missions is done randomly. More like a dart board, where a target is chosen and a shaky hand throws. There'd still be a lot of correlation across reasons for mission targeting.

For those interested in this topic, I suggest a paper by Bernd Beber that uses UN recess timing to capture involvement in mediation processes, an arguably much better quasi-experimental approach. (Beber does find that mediation matters, but that is hardly the same as arguing that troops matter.)

Then there is work Collier claims to have done with Patrick McGowan from my Alma Mater, ASU. It smells of the same causality problems: tying coup incidence to democracy. I can't though find the paper anywhere online.

I think the problem is Collier is engaged in producing a lot of research to bolster his arguments and so suffers from sloppiness more than failing to understand the problems of causality and endogeneity. (Although, he does call the recent push for randomized trials in economics as a "fad", so maybe he believes the whole causality problem is simply a "fad")

One example that shows he is keenly aware of the problem of causality is a paper he does with Marguerite Duponchel, unpublished, on Sierra Leone, where he uses distance to the border with Liberia as an instrumental variable, similar to a paper by Richard Akresh Damien de Walque, who use distance to Uganda as an IV for the Rwandan genocide. It is the only reference to an IV that I could find in the whole book.

His previous book, Bottom Billion, did not suffer from these problems because he relied more on case studies. In this new book he seems to be trying to make a more generalizable argument, but fails to use good scholarship to do it.

I recently urged a reader of this blog to be cautious with regression models that do not have clear causality reasoning, such as simple lagged time periods. I think Collier, and anyone that reads this book, needs this advice too.