Some high-profile scientific studies have cast substantial doubt on the veracity of, well, scientific studies. This is rather a big deal for those of us who want to believe that science is the best way to find the truth about important things in the world.
Recently, the New York Times reported on a recently completed analysis by a psychologist at the University of Virginia supported by the Laura and John Arnold Foundation. He asked 250 researchers to reproduce the results of 100 previously published psychology and social science papers—reproducibility being, of course, a core tenet of the scientific method. Okay, you'd probably expect that a few of the experiments might not work out the second time around. It was not a few. Testers were unable to reproduce a whopping 60 percent of the studies. This wasn't because of fraud, but because the original studies simply weren't as conclusive as the authors claimed.
(On the other hand, the University of Virginia study may have been flawed, if the University of Virginia study is to be believed. Just saying.)
In any case, not-quite-solid conclusions aren't restricted to the so-called soft sciences. It's also a recognized problem in medical and other research. Fortunately, some institutions are taking steps to strengthen this critical plank in the scientific platform. And the Arnold Foundation has become the top funder behind such work.
We've written before about the LJAF's evidence-based approach to combat problems in areas like criminal justice and education. Meanwhile, with their initiative on research integrity, they're seeking to strengthen the very notion of scientific evidence.
- The Billionaire Couple Targeting Shoddy and Secretive Science
- Why This Funder is Spending Big Money to Clean Up Bad Research
Most recently, LJAF announced a $3.8 million give to Stanford University to establish the Center for Reproducible Neuroscience (CRN), which says it will work to increase transparency and improve reliability in neuroscience research on brain disorders such as Alzheimer's disease, schizophrenia and depression. The CRN will attempt to boost research reliability through the implementation of a free, online platform using supercomputers to provide scientists with advanced tools to better study serious brain and mental illnesses.
The LJAF says neuroscience is a particularly fitting target of efforts to improve research reliability. Not only is the brain disproportionately complex, the foundation points out, it's especially hard to study. People, as a rule, don't care to have researchers poking around in their gray matter, and neuroscientists can't always grab brain cells and look at them under a microscope. So they mostly rely on non-invasive imaging methods like functional Magnetic Resonance Imaging (fMRI).
CRN says its platform will enable neuroscientists to conduct more open, reproducible research. This will help investigators build a framework of solid evidence that will ultimately provide better guidance for the development of new treatments. Through the platform, scientists will be able to share their data as well as the methods used to produce the findings. The platform is scheduled for launch in 2017.
But the LJAF makes an even more serious point. It says the scientific challenges of studying the brain are "compounded by misaligned incentives within the research community, which have led some neuroscientists to employ flawed research practices in order to produce results that are more likely to be published in high-profile journals or attract grant funding."
This has already occurred. According to the LJAF, many small studies in the last decade purportedly identified genes related to schizophrenia, memory, and prefrontal cortex function. Those studies—and many others claiming that particular genes are related to certain psychological traits or brain systems—have since been proved unreliable.
In medical research, dead ends like that can slow the development of needed medical treatments and cures. And again, it's not restricted to any single field: Researchers need to publish to advance their careers, or even to keep their jobs. Institutions, the research community and even the government need to intensify efforts to eliminate the pressures that lead to the practice and publication of bad science.