Tuesday, December 9, 2008

Correlation is not causation

It is really depressing to find researchers at top universities that still don't seem to understand this. James Fowler of UC San Diego has amassed a body of research that shows he has no clue about the difference.

Ever heard of the Colbert Bump? Here's a paper Fowler wrote where he tried to prove it. (In a previous edit of the paper, he used the word "scientific" three times in his abstract. If you have to use the word "scientific", its probably not.) He finds that politicians who were on the show received more donations right afterward. But there's a serious problem: what if politicians are fund raising around the time they are on the show? Certainly no politician appears on TV randomly; they do it as part of a larger campaign financing strategy. We can't be sure its Colbert, or general campaigning strategy.

Fowler also has a couple of papers using a 20 year data set where he looks at the effect of social networks on smoking, obesity and, most recently, happiness. In all three papers he finds that your social network, or who you know and spend time with, is very correlated with how much you smoke, how much weight you gain, and how happy you are. His conclusions in all three papers is not just that your social network matters, but that there are contagion effects so that a change somewhere in your social network will affect these things about you as well.

He arrives at these conclusions through logical nonsense. While he tries to control for selection effects, where you choose the people you know based on similar characteristics, but he does it by only using lagged values, which does not actually solve the problem. More importantly, he can't control for shared experiences. As Justin Wolfers at the blog Freakonomics puts it when discussing the happiness paper:

"if you and I are friends, we are often subject to similar influences. If a buddy of ours dies, we’ll both be less happy. Or, less dramatically, if our favorite football team wins, we’ll both be happier. But this isn’t contagious happiness — it is simply a natural outcome of the shared experiences of people in the same social circles."

To drive the point home even harder, Ethan Cohen-Cole and Jason Fletcher have a paper out (in the same journal no less) using the same methods that Fowler uses and find that social networks also determine height, headaches, and acne. So, if someone in your social network gets taller, you do to. Or, if someone gets a headache, so do you. Obviously, they are using this as an example of when confusing causation with correlation is dangerous.

There is probably a contagion effect to social networks. We are social people, and we feed off of others. Its probably no where near as big of an effect though as Fowler keeps finding. Its also sad to see that this kind of research keep being published.

No comments: