Monday, November 10, 2008

Scientific bias and distortion

Scientific publishing, including the social, natural and medical sciences, comes with its own set of human biases, even in areas that seem so important this shouldn't happen. Publication bias is when journals prefer to publish significant results over insignificant, and researchers, knowing this, then focus their work in the same way. It's a serious problem that can distort the intent of science, and can have important implications for the real world. Here are some examples:

  • Disclosure: I am an ardent anti-death penalty proponent. An interesting post on another blog discusses how research on the effect of the the death penalty as a deterent to crime shows some funny tendencies. I am not very familiar with the method used here, but it seems an interesting check for publication bias.
  • I just finished reading an article on the economics of scientific publication. Publishing is a business, and it has its own incentives. Given the power of the internet for disseminating information, why are journals still in print? Why are they so expensive most developing country institutions can't afford them? When business trumps science, probelms can arise.
  • Don't think medical trials avoid these problems. There is plenty of evidence that pharmaceutical companies have been making sure such bias continues.

Update: I just finished reading a paper from Djankov and Reynal-Querol that argues that the link between poverty and civil war is not a correct. They aregue that once you include simple dummy variables into cross-country regressions, the effect everyone in the literature is crazy about goes away. So, they find zero effect.

This is the exact opposite argument of a paper from the Brookings Institute that states in the intro that "an important element of [the debate over what determines civil war] has been resolved: recent academic research on the causes of conflict demonstrates compellingly that countries with low income per capita are at increased risk of civil conflict."

Djankov and Reynal-Querol will probably get their paper published because it will lead to plenty of interesting debate, but would it have been published if it was the first paper on the topic? I vote probably not.

No comments: