Lies, damned lies, and statistics

My favorite quote on the unreliability of statistics comes from Homer Simpson, who once said something like: "Oh, people can come up with statistics to prove anything, Kent. 14% of people know that."

It's true that people have become somewhat jaded when it comes to statistics these days, and who can blame them. So many numbers are thrown out, and a little research will find a lot of it just isn't true.

For instance, after spending 6 months last year in northern Uganda studying the incredible problems displaced people have been facing, imagine my surprise to find a research paper presented at Oxford that claims to have found evidence that "displacement by war into an Internally Displaced People‘s (IDP) camp does not have a significant impact on expenditures per adult equivalent. However, using quantile regressions at each decile of the expenditure distribution, we find that living in an IDP camp has no impact on the expenditures per adult equivalent at any decile except the top two, where it has a significantly negative impact. Thus, it is the better off households who lose most (indeed, lose at all) when forced to move to a camp."

A decile is of course a 10% cutoff, meaning they found that only the richest 20% of people were affected by displacement.

Not believing their results, I looked at the data myself and found some disturbing results. Rather than affecting only the richest 20%, displacement has had a massive impact on everyone BUT THE BOTTOM 20%.

Why did I find such different results? For those of you that may get lost in the statistics of my paper, the most important differences are that (1) the Oxford presentation focuses on consumption, which in fact identifies the service delivery quality of the World Food Program, who is delivering food aid in northern Uganda, while I look at the effect of displacement on assets and food quality, and (2) the authors use a simple dummy variable in their regression for being an IDP (1 for yes, 0 for no), and for being Acholi. The conflict is centered in Acholiland, and 95% of the Acholi are displaced. At the minimum, all of this means its really not clear how to interpret any of the results.

While my solution may not be perfect, it is the best one can do with the data. The situation in northern Uganda actually offers an interesting study as the rebel group was not well organized on the ground, and so attacks (and eventual displacement) was not systematic. This means a good identification strategy, as I try to do, can give meaningful results.

Keep in mind that getting unbiased estimates of the effect of displacement is very difficult, and I am aware of only one other person that is really trying to answer this question well. Until better work starts coming out, try to be in the 14% and stay suspicious of any stats.