By Dan Vergano
USA Today: Science Fair
Some scientific disciplines are reporting far fewer experiments that didn't work out than they did twenty years ago, suggests an analysis of the scientific literature.In particular, economists, business school researchers and other social scientists, as well as some biomedical fields, appear increasingly susceptible to the "file-drawer" effect -- letting experiments that fail to prove an idea go unpublished -- suggests the Scientometrics journal study by Daniele Fanelli of Scotland's University of Edinburgh.
"Positive results in research studies overall, became 22% more likely to appear in scientific journals from 1990 to 2007," says the study, which looked at a sample of 4,656 papers over this time period, looking for trends in science journals.
"One of the most worrying distortions that scientific knowledge might endure is the loss of negative data. Results that do not confirm expectations—because they yield an effect that is either not statistically significant or just contradicts an hypothesis—are crucial to scientific progress, because this latter is only made possible by a collective self-correcting process. Yet, a lack of null and negative results has been noticed in innumerable fields. Their absence from the literature not only inflates effect size estimates in meta-analyses, thus exaggerating the importance of phenomena, but can also cause a waste of resources replicating research that has already failed, and might even create fields based on completely non-existent phenomena," says the analysis.
The analysis looked at studies where authors proposed a hypothesis and then sought to test it, either confirming it for a positive result, or not. Overall, 70.2% of papers were positive in 1990–1991 and 85.9% were positive in 2007. "On average, the odds or reporting a positive result have increased by around 6% every year, showing a statistically highly significant trend," says the study.
Japan produced the highest rate of positive results, followed by U.S. results. Some regions, such as Europe, and disciplines, such as Geosciences and Space Science, didn't show the positive-result increases:
"The average frequency of positive results was significantly higher when moving from the physical, to the biological to the social sciences, and in applied versus pure disciplines, all of which confirms previous findings. Space science had not only the lowest frequency of positive results overall, it was also the only discipline to show a slight decline in positive results over the years, together with Neuroscience & Behaviour," says the study.
Geosciences and Plant & Animal sciences showed a flat rate of positive vs. negative results since 1990.
The entire piece can be read here.
