Publications and open science experiences in times of COVID-19 – two wildly different experiences

SARS-CoV-2 and COVID-19 has brought out both the best and worse in medical research. There are numerous examples that highlight this, such as the deplorable state of COVID-19 prediction models and the controversy around a quite well known meta-researcher. I won’t go into those topics. Instead, I will share two of my own experience that will illustrate both sides of the coin when it comes to going the extra mile by adopting some open sciences practices.

On the good side of things there is our effort of implementing an ordinal scale to measure the functional outcome after COVID-19. After our letter to the editor in which we only propose the idea, we kept the project going by publishing a manual to standardize efforts. On top of that, we were lucky that many researchers and clinicians were willing to contribute time and effort with translations and adaptations for their own patients. To this day, we have 20 translations, and many more in the making. Next to this, the scale is now included in several clinical guidelines and relevant clinical studies. This is a story of success – not for us, but for the open character of science in pandemic times. We had an idea, were able to share it quickly in a traditional journal with a letter to the editor, and afterwards supported by OSF to keep the project going.

The bad side of things is the little call-to-action I wrote with my colleague DM. He observed that there was an interesting pattern in the percentage of patients that were admitted to the hospital with COVID-19 – with up to a twofold difference, as you can see in the graph above. The lower the weekly SARS-CoV-2 infection numbers, the higher the percentage of patients that were admitted in the hospitals. Our argument at the time was that, in preparation for the third wave of infections in the Netherlands, medical professionals (GP, ER docs, home nurses, etc) should try to keep the hospitals as empty as possible and keep the beds ready for the large numbers of patients that we knew were coming our way. Because it was more of an opinion piece, we skipped a detailed methods section, but we provided everything (code, data, graph, methods description) on a dedicated OSF page which we kept up-to-date. We offered our thoughts as a comment to the Dutch journal “huisarts en wetenschap”, who after a two-week review period decided to turn it down. Two thoughts on this: 1) I am not sure why there was peer review as it was a call-to-action and not original research and 2) the reason they gave us for the rejection (after revision!) is that the message was perhaps too complicated in these already confusing times – I will leave this without further comment. Next up was the journal NTVG, a more general medical journal in the Netherlands. The peer review process (again!) gave us another delay of two weeks, before the paper was accepted. But you might have guessed it – the third wave had already started and left our call-to-action redundant. In the end, we talked to the editor and mutually decided that we would withdraw our paper, and that at a later moment in time, we might actually write a research paper on the causes behind the variation in percentage of positive tested patients admitted to the hospital.

All in all a very dissatisfying experience, especially in comparison with the PCFS story. The weird thing here is that in both cases we did the same thing – we had an idea, wanted to share with those that might be triggered by it. We started with some information on OSF with dedicated project pages with their content growing overtime as the story developed. In both cases we were commended for using OSF to share additional material in support of the letter to the editor. Still, the outcome is very different. This can of course just be that the ideas are inherently different, but the delay in the publishing process not only killed the second project but also killed some of my enthusiasm for open science. Luckily, it is only some, and I hope / am sure that some future experiences will help me regain some of that.

new paper: pulmonary dysfunction and CVD outcome in the ELSA study

 This is a special paper to me, as this is a paper that is 100% the product of my team at the CSB.Well, 100%? Not really. This is the first paper from a series of projects where we work with open data, i.e. data collected by others who subsequently shared it. A lot of people talk about open data, and how all the data created should be made available to other researchers, but not a lot of people talk about using that kind of data. For that reason we have picked a couple of data resources to see how easy it is to work with data that is initially not collected by ourselves.

It is hard, as we now have learned. Even though the studies we have focussed on (ELSA study and UK understanding society) have a good description of their data and methods, understanding this takes time and effort. And even after putting in all the time and effort you might still not know all the little details and idiosyncrasies in this data.

A nice example lies in the exposure that we used in this analyses, pulmonary dysfunction. The data for this exposure was captured in several different datasets, in different variables. Reverse engineering a logical and interpretable concept out of these data points was not easy. This is perhaps also true in data that you do collect yourself, but then at least these thoughts are being more or less done before data collection starts and no reverse engineering is needed. new paper: pulmonary dysfunction and CVD outcome in the ELSA study

So we learned a lot. Not only about the role of pulmonary dysfunction as a cause of CVD (hint, it is limited), or about the different sensitivity analyses that we used to check the influence of missing data on the conclusions of our main analyses (hint, limited again) or the need of updating an exposure that progresses over time (hint, relevant), but also about how it is to use data collected by others (hint, useful but not easy).

The paper, with the title “Pulmonary dysfunction and development of different cardiovascular outcomes in the general population.” with IP as the first author can be found here on pubmed or via my mendeley profile.