Sometimes, open science seems to be the thing that is going to solve all the nasty bits that are part of the current scientific enterprise. No added value from peer review? Registered report! Duplicate work? Preprints and preregistration! People working on the same problem as you are? Share data!
But recently I came across a stark reminder that that is not the whole story. I was asked to review a short research letter from people who were looking to gather all evidence to answer a particular clinical question. While doing so, they actually stumbled upon a deluge of meta-analyses: 20 meta-analyses were published in 17 different journals. What a body of evidence is then perhaps the first reaction, but the problem was that is actually all these analyses were based on only 4 trials.
The authors pointed towards the enormous waste of energy and even talked about an epidemic of meta-analyses. I see in this a great example how the root cause lies in that science is being done for publications and not publications are made for the betterment of science. This is the root cause of many problems in the scientific enterprise. And as long as that is not addressed, open science practices are just a stopgap – something which might work for a bit, but it does not address the root cause and therefore does not solve the problem. If you want to read the whole argument, with some details on what open science practices were actually in place with these 20 meta-analyses, the paper is only a minute 5-minute read.
This commentary is a joint production with FRR. This paper was a preprint first on our OSF page and is now published in the journal of thrombosis and hemostasis and can be accessed here (open access).