What will happen when after an ICH? A summary of the current state of prediction models

Figure 2 from the paper, showing the number of prognostic models that use a certain combination of outcome (rows) and the timing of outcome assessment (columns)

The question seems to be straightforward: “what bad stuff happens when after somebody develops an intracerebral hemorrhage, and how will I know whether that will also happen to me now that I have one”? The answer is, as always, “it depends”. It depends on how you actually specify the question. What does “bad stuff” mean? Which “when” are you interested? And what are your personal risk factors? We need all this information in order to get an answer from a clinical prediction model.

The thing is, we also need a good working clinical prediction model – that is it should distinguish those who develop the bad stuff from those who don’t, but it should also make sure that the absolute risks are about right. This new paper (project carried by JW) discusses all ins and outs when it comes to the current state of affairs when it comes to predictions. Written for neurologist, some of these comments and points that we rise will not be new to methodologists. But as it is not a given that methodologist will be involved somebody decides that a new prediction model needs to be developed, we wrote it all in up in this review.

The paper, publishes in Neurological Research and Practice, has a couple of messages:

  • The number of existing prediction models for this disease is already quite big – and the complexity of the models seem to increase overtime, without a clear indication that the performance of these models gets better. A lot of these models use different definitions for the type of outcome, as well as the moment that the outcome is assessed – all leading to wildly different models, which are difficult to compare.
  • The statistical workup is limited: The performance is often only measured in a simple AUC- calibration and net benefit is not reported on. Even more worryingly, external validation not always possible, as the original publications do not provide point estimates.
  • Given the severity of the disease, the so-called “withdrawal of care bias” is an important element when thinking and talking about prognostic scores. This bias, in which those with a bad score do not receive treatment can lead to a self-fulfilling prophecy type of situation in the clinic, captured in the data.

In short – when you think you want to develop a new model, think again. Think long and hard. Identify why the current models are working or are not working. Can you improve? Do you have the insights and skill set to do so? Really? If you think so, please do so, but just don’t add another not so useful prediction model to the already saturated literature.

Messy epidemiology: the tale of transient global amnesia and three control groups

Clinical epidemiology is sometimes messy. The methods and data that you might want to use might not be available or just too damn expensive. Does that mean that you should throw in the towel? I do not think so.

I am currently working in a more clinical oriented setting, as the only researcher trained as a clinical epidemiologist. I could tell about being misunderstood and feeling lonely as the only who one who has seen the light, but that would just be lying. The fact is that my position is one privilege and opportunity, as I work with many different groups together on a wide variety of research questions that have the potential to influence clinical reality directly and bring small, but meaningful progress to the field.

Sometimes that work is messy: not the right methods, a difference in interpretation, a p value in table 1… you get the idea. But sometimes something pretty comes out of that mess. That is what happened with this paper, that just got published online (e-pub) in the European Journal of Neurology.  The general topic is the heart brain interaction, and more specifically to what extent damage to the heart actually has a role in transient global amnesia. Now, the idea that there might be a link is due to some previous case series, as well as the clinical experience of some of my colleagues. Next step would of course to do a formal case control-study, and if you want to estimate true measure of rate ratios, a lot effort has to go into the collection of data from a population based control group. We had neither time nor money to do so, and upon closer inspection, we also did not really need that clean control group to answer some of our questions that would progress to the field.

So instead, we chose three different control groups, perhaps better referred as reference groups, all three with some neurological disease. Yes, there are selections at play for each of these groups, but we could argue that those selections might be true for all groups. If these selection processes are similar for all groups, strong differences in patient characteristics of biomarkers suggest that other biological systems are at play. The trick is not to hide these limitations, but as a practiced judoka, leverage these weaknesses and turn them into a strengths. Be open about what you did, show the results, so that others can build on that experience.

So that is what we did. Compared patients with migraine with aura, vestibular neuritis and transient ischemic attack, patients with transient global amnesia are more likely to exhibitsigns of myocardial stress. This study was not designed – nor will if even be able to – understand the cause of this link, not do we pretend that our odds ratios are in fact estimates of rate ratios or something fancy like that. Still, even though many aspects of this study are not “by the book”, it did provide some new insights that help further thinking about and investigations of this debilitating and impactful disease.

The effort was lead by EH, and the final paper can be found here on pubmed.

FVIII, Protein C and the Risk of Arterial Thrombosis: More than the Sum of Its Parts.

maxresdefault
source: https://www.youtube.com/watch?v=jGMRLLySc4w 

Peer review is not a pissing contest. Peer reviewing is not about findings the smallest of errors and delay publication because of it. Peer review is not about being right. Peer review is not about rewriting the paper under review. Peer review is not about asking for yet another experiment.

 

Peer review is about making sure that the conclusions presented in the paper are justified by the data presented and peer review is about helping the authors get the best report on what they did.

At least that what I try to remind myself of when I write my peer review report. So what happened when I wrote a peer review about a paper presenting data on the two hemostatic factors protein C and FVIII in relation to arterial thrombosis. These two proteins are known to have a direct interaction with each other. But does this also translate into the situation where a combination of the two risk factors of the “have both, get extra risk for free”?

There are two approaches to test so-called interaction: statistical and biological. The authors presented one approach, while I thought the other approach was better suited to analyze and interpret the data. Did that result in an academic battle of arguments, or perhaps a peer review deadlock? No, the authors were quite civil to entertain my rambling thoughts and comments with additional analyses and results, but convinced me in the end that their approach have more merit in this particular situation. The editor of thrombosis and hemostasis saw this all going down and agreed with my suggestion that an accompanying editorial on this topic to help the readers understand what actually happened during the peer review process. The nice thing about this is that the editor asked me to that editorial, which can be found here, the paper by Zakai et al can be found here.

All this learned me a thing or two about peer review: Cordial peer review is always better (duh!) than a peer review street brawl, and that sharing aspects from the peer review process could help readers understand the paper in more detail. Open peer review, especially the parts where peer review is not anonymous and reports are open to readers after publication, is a way to foster both practices. In the meantime, this editorial will have to do.

 

BEMC has a Journal Club now

cropped-favicon_bemc1

After a year of successful BEMC talks and seeing the BEMC grow,  it was time for something new. We are starting a new journal club within the BEMC community, purely focussed on methods. The text below describes what we are going to to do, starting in February. (text comes from the BEMC website)

BEMC is trying something new: a journal club. In february, we will start a monthly journal to accompany the BEMC talks as an experiment. The format is subject to change as we will adapt after gaining more experience in what works and what not. For now, we are thinking along the following lines:

Why another journal club?

Aren’t we already drowning in Journal clubs? Perhaps, but not with this kind of journal club. BEMC JClub is focussed on the methods of clinical research. Many epidemiological inclined researchers work at departments who are not focussed on methodology, but rather on a disease or field of medicine. This is reflected in the topics of the different journal clubs around town. We believe there is a need for a methods journal club in Berlin. Our hope for the BEMC JClub is to fulfill that need through interdisciplinary and methodological discussions of the papers that we read.

Who is going to participate?

First of all, please remember that the BEMC community focussed on researchers with a medium to advanced epidemiological knowledge and skill set. This is not only true for our BEMC talks, but also for our JClub.

Next to this, we hope that we will end up with a good group that reflects the BEMC community. This means that we are looking for a group with a nice mix in background and experience. That means that if you think you have unique background and focus in your work, we highly encourage you to join us and make our group as diverse as possible. We strive for this diversity as we do not want the JClub sessions to become echo chambers or teaching sessions, but truly discussions that promote knowledge exchange between methodologist from different fields.

What will we read?

Anything that is relevant for those who attend. The BEMC team will ultimately determine which papers we will read, but we are nice people and listen carefully to the suggestions of regulars. Sometimes we will pick a paper on the same (or related) topic of the BEMC talk of that month.

Even though the BEMC team has the lead in the organisation, the content of the JClub should come from everybody attending. Everybody that attends the Jclub is asked to provide some points, remarks or questions to jumpstart the discussion.

What about students?

Difficult to say. The BEMC JClub is not designed to teach medical students the basics in epidemiology. Then again, everybody who is smart, can keep up and contribute to the discussion is welcome.

Are you a student and in doubt whether the BEMC JClub is for you? Just send us an email.

Where? When?

Details like this can on the BEMC Jclub website. Just click here.

New paper: Contribution of Established Stroke Risk Factors to the Burden of Stroke in Young Adults

2017-06-16 09_26_46-Contribution of Established Stroke Risk Factors to the Burden of Stroke in Young2017-06-16 09_25_58-Contribution of Established Stroke Risk Factors to the Burden of Stroke in Young

Just a relative risk is not enough to fully understand the implications of your findings. Sure, if you are an expert in a field, the context of that field will help you to assess the RR. But if ou are not, the context of the numerator and denominator is often lost. There are several ways to work towards that. If you have a question that revolves around group discrimination (i.e. questions of diagnosis or prediction) the RR needs to be understood in relation to other predictors or diagnostic variables. That combination is best assessed through the added discriminatory value such as the AUC improvement or even more fancy methods like reclassification tables and net benefit indices. But if you are interested in are interested in a single factor (e.g. in questions of causality or treatment) a number needed to treat (NNT) or the Population Attributable Fraction can be used.

The PAF has been subject of my publications before, for example in these papers where we use the PAF to provide the context for the different OR of markers of hypercoagulability in the RATIO study / in a systematic review. This paper is a more general text, as it is meant to provide in insight for non epidemiologist what epidemiology can bring to the field of law. Here, the PAF is an interesting measure, as it has relation to the etiological fraction – a number that can be very interesting in tort law. Some of my slides from a law symposium that I attended addresses these questions and that particular Dutch case of tort law.

But the PAF is and remains an epidemiological measure and tells us what fraction of the cases in the population can be attributed to the exposure of interest. You can combine the PAF to a single number (given some assumptions which basically boil down to the idea that the combined factors work on an exact multiplicative scale, both statistically as well as biologically). A 2016 Lancet paper, which made huge impact and increased interest in the concept of the PAF, was the INTERSTROKE paper. It showed that up to 90% of all stroke cases can be attributed to only 10 factors, and all of them modifiable.

We had the question whether this was the same for young stroke patients. After all, the longstanding idea is that young stroke is a different disease from old stroke, where traditional CVD risk factors play a less prominent role. The idea is that more exotic causal mechanisms (e.g. hypercoagulability) play a more prominent role in this age group. Boy, where we wrong. In a dataset which combines data from the SIFAP and GEDA studies, we noticed that the bulk of the cases can be attributed to modifiable risk factors (80% to 4 risk factors). There are some elements with the paper (age effect even within the young study population, subtype effects, definition effects) that i wont go into here. For that you need the read the paper -published in stroke- here, or via my mendeley account. The main work of the work was done by AA and UG. Great job!

Berlin Epidemiological Methods Colloquium kicks of with SER event

A small group of epi-nerds (JLR, TK and myself) decided to start a colloquium on epidemiological methods. This colloquium series kicks off with a webcast of an event organised by the Society for Epidemiological Research (SER), but in general we will organize meetings focussed on advanced topics in epidemiological methods. Anyone interested is welcome. Regularly meetings will start in February 2017. All meetings will be held in English.
More information on the first event can be found below or via this link:

“Perspective of relative versus absolute effect measures” via SERdigital

Date: Wednesday, November 16th 2016 Time: 6:00pm – 9:00pm
Location: Seminar Room of the Neurology Clinic, first floor (Alte Nervenklinik)
Bonhoefferweg 3, Charite Universitätsmedizin Berlin- Campus Mitte, 10117 Berlin
(Map: https://www.charite.de/service/lageplan/plan/map/ccm_bonhoefferweg_3)

Description:
Join us for a live, interactive viewing party of a debate between two leading epidemiologists, Dr. Charlie Poole and Dr. Donna Spiegelman, about the merits of relative versus absolute effect measures. Which measure of effect should epidemiologists prioritize? This digital event organized by the Society for Epidemiologic Research will also include three live oral presentations selected from submitted abstracts. There will be open discussion with other viewers from across the globe and opportunities to submit questions to the speakers. And since no movie night is complete without popcorn, we will provide that, too! For more information, see: https://epiresearch.org/ser50/serdigital

If you plan to attend, please register (space limited): https://goo.gl/forms/3Q0OsOxufk4rL9Pu1