I’ve become more skeptical through experience. Whenever I read a survey or study, I have a number of stock questions I ask before accepting the findings.
Was the study peer reviewed? Can the results be repeated and verified? What was the breadth of the study? How many subjects were tested? And, perhaps most importantly, who paid for the research?
Unless you can answer these questions satisfactorily the study is best treated as either very preliminary or unverified.
An interesting article in the Chicago Tribune pointed a recent study that said children who eat candy weigh less than those who don’t.
Think about that statement. It is completely counter-intuitive to my way of thinking.
But with the barest analysis, you can find out who might be motivated to publish a study such as this.
The paper, it turns out, was funded by a trade association representing the makers of Butterfingers, Hershey and Skittles. And its findings were touted by the group even though one of its authors didn’t seem to think much of it.
So, we have a study funded by an industry group selling a product, using a study that even the researchers felt was dubious, to support a position that clearly benefits their business.
Another article, at Food & Wine hits the nail on the head.
Much of this research is funded by food manufacturers with clear agendas.
I’ve already posted on why you need to read popular coverage of science stories carefully. Whenever I see a story pointing to a particular piece of research or an academic paper I look for the source document and read it.
I’ll admit some of the content can be very dense and technical but you can learn a lot by reading an abstract and conclusion – sections that are usually easier to read.