Recreating scientific studies that have been funded by large corporations is very difficult and disproving or countering any findings are less common because to apply the scientific method properly is beyond skill and know how, it’s down to money.
I’m a researcher myself, so I feel like I can weigh in on the “reproducibility crisis”. There are several facets to it: One is of course money, but that’s not just related to corporately funded research. Good like finding or building an independent lab capable of reproducing the results at CERN. It basically boils down to the fact that some (a lot of) research is insanely expensive to do. This primarily applies to experiments and to some degree to computationally expensive stuff.
Another side is related to interest. Your average researcher is fired up by the thought of being the first person to discover and publish something no one has seen before. It’s just not as fun to reproduce something someone else has already done. Even if you do, you’re likely to try to improve on it somehow, which means the results may change without directly invalidating the old results. It can be hard work to write a good paper, so if you don’t feel your results are novel enough that they’re worth the effort (because they’re basically just equivalent to previously published values) you might not bother to put in the effort to publish them.
Finally, even without direct reproduction of previously published results, science has a way asymptotically approaching some kind of truth. When I develop and publish something, I’m building on dozens of previously published works. If what they did was plain wrong, then my models would also be liable to fail. I’ve had cases where we’ve improved on previously published work, not because we tried to reproduce it, but because we tried to build on their results, and found out that their results didn’t make sense. That kind of thing is fairly common, but not reported as a “reproduction study”.
There’s also review articles that, while they don’t do any reproduction themselves, collect and compare a bunch of comparable work. They usually have some conclusions regarding what results appear trustworthy, and what appear to be erroneous.
I’ve always considered sciences like psychology to be more susceptible to the reproducibility crisis. It seems if someone decides to pursue a career in academia the criteria becomes publishing, and well publish or perish as is goes.
I think some researchers areocing towards things like prerigistering hypothesis and open data+publishing source code for calculations and using that as references in there paper so it can be updated afterwards.
They’re have definitely been a lot of papers where results were later determined to be wrong but is still referenced because well you can’t update a paper from the 1970s.
This is hearsay from friends I’ve never done any serious research or published in journals. As a side note I do enjoy reading taking a scroll through https://retractionwatch.com/
Replicating results is a problem across the board; I’m sure money is a factor but it’s not just the chocolate-sponsored-by-Hershey studies that have replication challenges.
Recreating scientific studies that have been funded by large corporations is very difficult and disproving or countering any findings are less common because to apply the scientific method properly is beyond skill and know how, it’s down to money.
I’m a researcher myself, so I feel like I can weigh in on the “reproducibility crisis”. There are several facets to it: One is of course money, but that’s not just related to corporately funded research. Good like finding or building an independent lab capable of reproducing the results at CERN. It basically boils down to the fact that some (a lot of) research is insanely expensive to do. This primarily applies to experiments and to some degree to computationally expensive stuff.
Another side is related to interest. Your average researcher is fired up by the thought of being the first person to discover and publish something no one has seen before. It’s just not as fun to reproduce something someone else has already done. Even if you do, you’re likely to try to improve on it somehow, which means the results may change without directly invalidating the old results. It can be hard work to write a good paper, so if you don’t feel your results are novel enough that they’re worth the effort (because they’re basically just equivalent to previously published values) you might not bother to put in the effort to publish them.
Finally, even without direct reproduction of previously published results, science has a way asymptotically approaching some kind of truth. When I develop and publish something, I’m building on dozens of previously published works. If what they did was plain wrong, then my models would also be liable to fail. I’ve had cases where we’ve improved on previously published work, not because we tried to reproduce it, but because we tried to build on their results, and found out that their results didn’t make sense. That kind of thing is fairly common, but not reported as a “reproduction study”.
There’s also review articles that, while they don’t do any reproduction themselves, collect and compare a bunch of comparable work. They usually have some conclusions regarding what results appear trustworthy, and what appear to be erroneous.
I’ve always considered sciences like psychology to be more susceptible to the reproducibility crisis. It seems if someone decides to pursue a career in academia the criteria becomes publishing, and well publish or perish as is goes.
I think some researchers areocing towards things like prerigistering hypothesis and open data+publishing source code for calculations and using that as references in there paper so it can be updated afterwards.
They’re have definitely been a lot of papers where results were later determined to be wrong but is still referenced because well you can’t update a paper from the 1970s.
This is hearsay from friends I’ve never done any serious research or published in journals. As a side note I do enjoy reading taking a scroll through https://retractionwatch.com/
Replicating results is a problem across the board; I’m sure money is a factor but it’s not just the chocolate-sponsored-by-Hershey studies that have replication challenges.