On the value of data – routinely vs purposefully

I listen to a bunch of podcasts, and the podcast “The Pitch” is one of them. In that podcast, Entrepreneurs of start-up companies pitch their ideas to investors. Not only is it amusing to hear some of these crazy business ideas, but the podcast also help me to understand about professional life works outside of science. One thing i learned is that it is ok if not expected, to oversell by about a factor 142.

Another thing that I learned is the apparent value of data. The value of data seems to be undisputed in these pitches. In fact, the product or service the company is selling or providing is often only a byproduct: collecting data about their users which subsequently can be leveraged for targeted advertisement seems to be the big play in many start-up companies.

I think this type of “value of data” is what it is: whatever the investors want to pay for that type of data is what it is worth. But it got me thinking about the value of data that we actually collect in medical. Let us first take a look at routinely data, which can be very cheap to collect. But what is the value of the data? The problem is that routinely collected data is often incomplete, rife with error and can lead to enormous biases – both information bias as well as selection bias. Still, some research questions can be answered with routinely collected data – as long as you make some real efforts to think about your design and analyses. So, there is value in routinely collected data as it can provide a first glance into the matter at hand.

And what is the case for purposefully collected data? The idea behind this is that the data is much more reliable: trained staff collects data in a standardised way resulting in datasets without many errors or holes. The downside is the “purpose” which often limits the scope and thereby the amount collected data per included individual. this is the most obvious in randomised clinical trials in which often millions of euro’s are spent to answer one single question. Trials often do no have the precision to provide answers to other questions. So it seems that the data can lose it value after answering that single question.

Luckily, many efforts were made to let purposefully collected keep some if its value even after they have served their purpose. Standardisation efforts between trials make it now possible to pool the data and thus obtain a higher precision. A good example from the field of stroke research is the VISTA collaboration, i.e the Virtual International Stroke Trials Archive”. Here, many trials – and later some observational studies – are combined to answer research questions with enough precision that otherwise would never be possible. This way we can answer questions with high quality of purposefully collected data with numbers otherwise unthinkable.

This brings me to a recent paper we published with data from the VISTA collaboration: “Early in-hospital exposure to statins and outcome after intracerebral haemorrhage”. The underlying question whether and when statins should be initiated / continued after ICH is clinically relevant but also limited in scope and impact, so is it justified to start a trial? We took the the easier and cheaper solution and analysed the data from VISTA. We conclude that

… early in-hospital exposure to statins after acute ICH was associated with better functional outcome compared with no statin exposure early after the event. Our data suggest that this association is particularly driven by continuation of pre-existing statin use within the first two days after the event. Thus, our findings provide clinical evidence to support current expert recommendations that prevalent statin use should be continued during the early in-hospital phase.1921

link

And this shows the limitations of even well collected data from RCT: as long as the exposure of interest is potentially provided to a certain subgroup (i.e. Confounding by indication), you can never really be certain about the treatment effects. To solve this, we would really need to break the bond between exposure and any other clinical characteristic, i.e. randomize. That remains the golden standard for intended effects of treatments. Still, our paper provided a piece of the puzzle and gave more insight, form data that retained some of its value due to standardisation and pooling. But there is no dollar value that we can put on the value of medical research data – routinely or purposefully collected alike- as it all depends on the question you are trying to answer.

Our paper, with JD in the lead, was published last year in the European Stroke Journal, and can be found here as well as on my Publons profile and Mendeley profile.

The story of a paper on the relationship between cancer and stroke that is both new and not so new.

Science is not quick. In fact, it is slow most of the time. Therefore, most researchers work on multiple papers at the same time. This is not necessarily bad, as parallel activities can be leveraged to increase the quality of the different projects. But sometimes this approach leads to significant delays. Imagine a paper that is basically done, and then during the peer review process, all the lead figures in the author team get different positions. Perhaps a Ph.D. student moves institutes for a post-doc, or junior doctors finish their training and set up their own practices, or start their demanding clinical duties in an academic medical center. All these steps are understandable and good for science in general but can hurt the speediness of individual papers.

This happened for example with a recently published paper in the Dutch PSI study. I say recently published because the work started > 5 years ago and has been finished more or less for the majority of that time. In this paper, we show that cancer prevalence is higher for stroke patients. But not all cancers are affected: it is primarily bladder cancer and head and neck type of effect. This might be explained by the shared risk factor smoking (bladder cancer, repository tract) and perhaps cancer treatment (central nervous system/ head and neck cancer). Not world shocking results with direct clinical implications, but relevant if you want to have a clear understanding of the consequences of cancer.


Now don’t get me wrong, I am very glad we, in the end, got all their ducks in a row and find a good place for the paper to be published. But the story is also a good warning: It was the willpower of some in the team to make this happen. Next time such a situation comes around, we might not have the right people with will right amount of power to keep on going with a paper like this. 

How to avoid this? Is “pre-print” the solution? I am not sure. On the surface, it indeed seems the answer, as it will give others at least the chance to see the work we did. But I am a firm believer that some form of peer review is necessary – just ‘dumping’ papers on a pre-print server is really a non-solution and I am afraid that a culture like that will only diminish the drive to get things formally published is even lower when manuscripts are already in the public domain. Post-publication peer review then? I am also skeptical here, as I the idea of pre-publication peer review is so deeply embedded within the current scientific enterprise that  I do not see post-publication peer review playing a big role anytime soon. The lack of incentive for peer review – let alone post-publication peer review – is really not helping us to make the needed changes anytime sooner. 


Luckily, there is a thing called intrinsic motivation, and I am glad that JW and LS had enough to get this paper published. The paper, with the title “Cancer prevalence higher in stroke patients than in the general population: the Dutch String-of-Pearls Institute (PSI) Stroke study. is published in European Journal of Neurology and can be found on Pubmed, as well as on my Mendeley and Publons profile.

Helping patients to navigate the fragmented healthcare landscape in Berlin: the NAVICARE stroke-atlas

the cover the Berlin Stroke Atlas

Research on healthcare delivery can only do so much to improve the lives of patients. Identifying the weak spots is important to start off with, but is not going to help patients one bit if they don’t get information that is actually useful let alone in time.

It is for that reason that the NAVICARE project not only focusses on doing research but also to provide information for patients, as well as bringing healthcare providers together in the NAVICARE network. The premise of NAVICARE is that somehow we need to help patients to navigate the fragmented healthcare landscape. We do so by using the diseases stroke and lung cancer as model diseases, prototypical diseases that help us focus our attention.

One deliverable is the stroke atlas: a document that provides different healthcare providers – and people and organizations who can help you in the broadest sense possible once you or your loved one is affected by a stroke. This stroke atlas, in conjunction with our personal approach at the stroke service point of the CSB/BSA, will help our patients. You can find the stroke atlas here (in German of course).

But this is only a first step. the navigator model is currently being further developed, for which NAVICARe has received additional funding this summer. I will not be part of those steps (see my post on my reshuffled research focus), but others at the CSB will.

Five years in Berlin and counting – reshuffling my research

I started to work in the CSB about 5 years ago. I took over an existing research group, CEHRIS, which provided services to other research groups in our center. Data management, project management and biostatisticians who worked on both clinical and preclinical research where all included in this team. My own research was a bit on the side, including old collaborations with Leiden and a new Ph.D. project with JR.

But then, early summer 2018 things started to change. The generous funding under the IFB scheme ran out, and CSB 3.0 had to switch to a skeleton crew. Now, for most research activities this had no direct impact, as funding for many key projects did not come from the CSB 2.0 grant. However, a lot of services to make our researchers perform at peak capability were hit. this included my team. CEHRIS, the service group ready to help other researchers was no longer.

But I stayed on, and I used the opportunity to focus my efforts on my own interest. I detached myself from projects I inherited but were not so engaged with, and I engaged myself with projects that interested me. This was, of course, a process over many months, starting end 2017. I feel now that it is time to share with you that I have a clear idea of what my new direction is. It boils downs to this:

My stroke research focuses on three projects in which we collect(ed) data ourselves: PROSCIS, BSPATIAL, BELOVE. The data collection in each of these projects is in different phases, and more papers will be coming out of these projects sooner later than later. Next to this, I will also help to analyze and publish data from others – that is after all what epidemiologists do. My methods research remains a bit of a hodgepodge where I still need to find focus and momentum. The problem here is that funding for this type of research has been lacking so far and will always be difficult to find – especially in Germany. But many ideas that come to from stroke projects have ripened into methodology working papers and abstracts, hopefully resulting in fully published papers quite soon. The third pillar is formed by the meta-research activities that I undertake with QUEST. Until now, these activities were a bit of a hobby, and always on the side. That has changed with the funding of SPOKES.

SPOKES is a new project that wants to improve the way we do biomedical research, especially translational research. Just pointing towards the problem (meta-research) or new top-down policy (ivory tower alert) is not enough. There has to be time and money for early and mid-career researchers to chip in as well in the process. SPOKES is going to facilitate that by making both time and money available. This starts with dedicated time and money for myself: I now work one day a week with the QUEST team. I will provide more details on SPOKES in a later post, but for now, I will just acknowledge that looking forward to this project within the framework of the Wellcome Trust Translational Partnership.

So there you have it, the three new pillars of my research activities in a single blog post. I have decided to lose the name CEHRIS to show that the old service focussed research group is no more. I have been struggling with choosing a new name, but in the end, I have settled for the German standard “AG-Siegerink”. Part lack of imagination, part laziness, and part underlining that there are three related but distinct research lines within that research group.

Up to the next 5 years!?

STEMO, our stroke ambulance, has had a bumpy ride…

STEMO in front of our clinic, source.

Pfew, there has been quite some excitement when it comes to the STEMO, the stroke ambulance in Berlin. The details are too specific -and way too German- for this blog, but the bottomline is this: during our evaluation of the STEMO, we noticed that STEMO was not always used as it should be. And if you do not use a tool like you should, it is hot half as effective. So we keep on trying to improve how STEMO is used in Berlin, even though the evaluation is going on.

We need to take these changes into account, so we wrote a new plan to evaluate STEMO, which was published open access the new BMC journal Neurological Research and Practice. The money to continue the evaluation was secured and we thought we were ready to go. But then reality set in: during budget negotiations a lower committee from the Berlin Senate said simply “NO” to STEMO. A day later however, the Major of Berlin used a “Machtword”, an informal veto to say that STEMO will be kept in the budget in order to finish the formal evaluation.

A true rollercoaster, which will show how directly our research has an impact on the society. The numerous calls, tweets and emails we have received in support of our now 3 STEMO ambulances over last couple of weeks underlines this even more (just the fact that a complete stranger started a petition with all nuances of the case taken into account is just mind boggeling !). But the science has to speak, and we still need to definitively evaluate the effectiveness of STEMO when used like it should be – something we will do over the next months with renewed energy in the whole team.

Auto-immune antibodies and their relevance for stroke patients – a new paper in Stroke

KMfor CVD+mortatily after stroke, stratified to serostatus for the anti-NMDA-R auto-antibody. taken from (doi: 10.1161/STROKEAHA.119.026100)

We recently published one of our projects embedded within the PROSCIS study. This follow-up study that includes 600+ men and women with acute stroke forms the basis of many active projects in the team (1 published, many coming up).

For this paper, PhD candidate PS measured auto-antibodies to the NMDAR receptor. Previous studies suggest that having these antibodies might be a marker, or even induce a kind of neuroprotective effect. That is not what we found: we showed that seropositive patients, especially those with the highest titers have a 3-3.5 fold increase in the risk of having a worse outcome, as well as almost 2-fold increased risk of CVD and death following the initial stroke.

Interesting findings, but some elements in our design do not allow us to draw very strong conclusions. One of them is the uncertainty of the seropositivity status of the patient over time. Are the antibodies actually induced over time? Are they transient? PS has come up with a solid plan to answer some of these questions, which includes measuring the antibodies at multiple time points just after stroke. Now, in PROSCIS we only have one blood sample, so we need to use biosamples from other studies that were designed with multiple blood draws. The team of AM was equally interested in the topic, so we teamed up. I am looking forward to follow-up on the questions that our own research brings up!

The effort was led by PS and most praise should go to her. The paper is published in Stroke, can be found online via pubmed, or via my Mendeley profile (doi: 10.1161/STROKEAHA.119.026100)

Update January 2020: There was a letter to the editor regarding our paper. We wrote a response.

Now hiring!

The text below is the English version of the official and very formal German text.

The QUEST center is looking for a project manager for the SPOKES project. SPOKES is part of the Wellcome Trust translational partnership program and aims to “Create Traction and Stimulate Grass-Root Activities to Promote a Culture of Translation Focused on Value”. SPOKES will be looking for grassroots activities from early and mid-career scientist who want to sustainably increase the value of the research in their own field.

The position will be located within the QUEST Center for Transforming Biomedical Research at the Berlin Institute of Health (BIH). The goal of QUEST is to optimize biomedical research in terms of sound scientific methodology, bio-ethics and access to research.

SPOKES is a new program organized by the QUEST Team at the Berlin Institute of Health. SPOKES enables our own researchers at the Charité / BIH to improve the way we do science. Your task is to identify and support these scientists. More specifically, we expect you to:

  • Promote the program within the BIH research community (interviews, newsletters, social media, events, etc)
  • Find the right candidates for this program (recruiting and selection)
  • Organize the logistics and help prepare the content of all our meetings (workshops, progress meetings, symposia, etc)
  • Support the selected researchers in their projects where possible (design, schedule and execute)

Next to this, there is an opportunity to perform some meta-research yourself.

We are looking for somebody with

  • A degree in biomedical research (MD, MSc, PhD or equivalent)
  • Proficiency in both English and German (both minimally C1)
  • Enthusiasm for improving science – if possible demonstrated by previous courses or other activities

Although no formal training as a project manager is required, we are looking for people who have some experience in setting up and running projects of any kind that involve people with different (scientific) backgrounds.