Bandwidths for dummies

Thanks to Herman Steigstra and Jillis Kriek

In discussions about excess mortality, reference is sometimes made to the bandwidth, which allows for abnormal deaths on a weekly basis. It is the light blue area around the predicted weekly value. If that margin is taken into account, there would be no significant excess mortality on a weekly basis. In this article I hope to make it plausible (also for non-statisticians like myself) why this is not true if we consider longer periods. Even if there is excess mortality during an entire calendar year, it is still possible that there is no excess mortality in any week because the weekly figures always remained within the bandwidth. Statisticians then say: "There is never significant excess mortality".

Every expectation has a degree of uncertainty, and that is certainly true when it comes to demographic data such as mortality expectancy. The extent of this uncertainty depends on all kinds of factors. For example, there is uncertainty in the estimation of an average, uncertainty in the number of deaths counted and weather influences that make the fluctuation greater or shifts in the composition of the population. It is a mix of statistical uncertainties and the difficult to predict variability of the subject of that statistic.

De werkelijke sterfte zit zelden of nooit exact op de verwachting maar dat is geen reden tot zorg. Wanneer in een week 2.600 doden zijn voorspeld, kunnen het er ook 2.500 zijn of 2.700. Als de voorspelling juist is, zal de werkelijke sterfte een slingerbeweging rond de verwachting oftewel de "basislijn" maken. Binnen een paar weken wordt een afwijking wel weer gecompenseerd door een afwijking de andere kant op.

How far that pendulum movement may go before there is an alarm signal is a statistical matter, but also depends on agreements about what we can and want to live with. All kinds of circumstances determine the extent to which many people die at the same time and what we find acceptable. Because more people die in winter than in summer and because it is accompanied by stronger swings, not only is the baseline higher in winter, but the bandwidth is also considerably larger (as much as 2x as large as in summer). So we find that acceptable.

Die fluctuatie is normaal - gegeven normale omstandigheden, zonder hittegolven of zware epidemie. De gemeten sterfte mag dus elke week afwijken van de verwachting. Daarvoor worden weekmarges gehanteerd.

Excess mortality or not?

Excess mortality per week

CBS hanteert de termen oversterfte en ondersterfte voor de weken waarin de donkerblauwe lijn (de sterfte) uit het lichtblauwe vlak (de weekmarge) breekt. De marge is het gebied tussen twee grenswaardes die we hebben afgesproken voor een te verwachten aantal overlijdens in een week. Uiteraard gelden ook hierbij weer onzekerheden maar dat voert nu te ver. De signaalwaardes zijn duidelijk en CBS publiceert ze. Zie bijvoorbeeld de "Download CSV" knoppen in de maandberichten op this page.

There may be increased or decreased or increased mortality, excess mortality, decreased mortality etc. etc. that also remains within the light blue area that indicates the weekly margins, so inside and outside the signal values. In this article, we use the terms for this increased mortality and reduced mortality.

Without a bandwidth in mortality expectancy, we would not be able to interpret fluctuations properly. The bandwidth therefore mainly has a signal function: within it there is no reason to worry for that week. Under normal circumstances, we can see in a moment when something excessive happens that needs to be explained. In the Netherlands, this is usually a heat wave or a flu epidemic.

Excess mortality per month

It seems obvious to use the successive weekly margins as a signal value when evaluating a longer period. No matter how far back we look, we continue to consider those weekly margins to be normative. That is not justified.

You can't just stick weekly margins one after the other to include them in longer periods.

Het is gemakkelijk te zien dat ruwweg optellen van de weekmarges -want dat doe je dan- leidt tot absurde resultaten. Als je alle oversterftemarges van 2021 bij elkaar optelt, kom je op een oversterftemarge van 13.413 extra sterfgevallen. Binnen die marge zou dus geen sprake zijn van oversterfte. Dat klopt niet met het feit dat zowel CBS als RIVM spreken van een "gemiddelde jaarlijkse oversterfte van 6.500 overlijdens meer dan verwacht". Dat is nog niet de helft van die 13.413.

Now this seems like a sought-after example, but this is exactly what happens when you look back on a year and check each week separately whether the weekly margin has been exceeded or not. Continuing to assess whether or not excess mortality is a weekly basis will therefore be misleading. After all, there is the possibility of getting a more precise picture because of the longer period of a month or six months.

So how can we stick those weekly margins together to a longer period?

Combining such weekly margins into a period is fascinating. On the one hand, you know that after several weeks it is about bigger and bigger numbers. On the other hand, you also expect your estimate to be more accurate after a longer period of time. You expect that what actually happens will be more in the direction of your estimate. If you take into account that you can be off by 1% after a year, that's very little. But in a short period of time, such as a random winter week, you can be off by as much as 20%. So: the longer the period, the narrower the bandwidth. But at the same time, the number of statements you make about increases every week and larger groups make the margins larger in absolute terms.

Vergelijk het met een muntje waarvan je verwacht dat je er net zoveel keer kop als munt mee zal gooien: als je die twee met elkaar vergelijkt, moet je na heel veel worpen steeds dichter bij 50 : 50 komen. Als je drie keer opgooit en je hebt drie keer kop, dan zegt dat eigenlijk niets. Gooi je nog eens drie keer op dan is de kans dat je wéér drie keer kop gooit hetzelfde, maar de kans dat je zes keer achter elkaar kop gegooid zou hebben is veel kleiner. Elke keer dat je weer drie worpen doet, is die kans op 3x "kop" steeds hetzelfde, maar de kans dat je voor alle worpen kop zult gooien is voor het steeds groter wordend aantal worpen op voorhand steeds kleiner.

Gooi je 365 keer het muntje op en je krijgt 365 keer kop, dan is dat een signaal dat er iets met je munt aan de hand is. Dan zul je ofwel je verwachting wat deze munt betreft moeten bijstellen ofwel de munt afkeuren (of gaan onderzoeken: misschien heeft die aan beide kanten kop?). Als je dan stug volhoudt " ja maar elke keer dat ik gooide was het toch echt een kans van een op twee dus er is niets bijzonders te zien" dan pas je de verkeerde regel toe om een reeks van gebeurtenissen te kunnen beoordelen.

The fact that a science journalist or a fact-checker does not immediately see this is perhaps justifiable. This would surprise me from statisticians. We will soon be reviewing a monthly CBS report on this subject.

Back to excess mortality: if, for example, we take an excess mortality of 300 per week as the limit value for excess mortality, this does not mean that the margin is 600 after two weeks and 900 after three weeks. That line is increasing more and more slowly and only reaches 600 after 4 weeks (depending on the chosen reliability limit, but that is beyond the scope of this blog. Usually 95% probability is chosen).

To calculate this decreasing increase in probabilities, there is a formula that comes from binomial statistics. The spread (statistical name for the deviations upwards and downwards) in the number of deaths (in this case) increases with the square root of the total number.

How the formula works out is best illustrated if we assume a constant weekly margin and a fixed number of deaths per week. The bandwidth at a constant weekly margin of 300 looks like a curve that gets flatter and flatter:

Week 1: 300
Week 1 to 2: 424
Weeks 1 to 3: 520
Weeks 1 to 4: 600
etc.

How does that relate to an excess mortality of, for example, 75 per week? We see that below.

In de eerste week zit de 75 meersterfte ruim onder de bovengrens van 300, in de tweede week op de helft van de margelimiet. Na 16 weken is het nulpunt bereikt en begint de 'officiële' oversterfte. Kijken we in week 26 terug naar het voorbije half jaar, dan zien we dat er oversterfte is over die halfjaarlijkse periode. De paarse lijn breekt uit de marge.

For example, from week 17 we are in a period of growing excess mortality.

An excess mortality situation can therefore arise with an excess mortality of only 75 deaths per week, despite the fact that the bandwidth indicates a signal value of 300 per week (that is 4 times as much!).

The light yellow dotted line in the graph indicates what happens when the weekly margins are simply added together, as is sometimes suggested.

CBS reporting example

CBS collects, manages, evaluates, interprets and plausibises our national data. In short, CBS tells us what the figures mean.

An example: May 2022.

In de kop van het artikel zien we al dat de oversterfte weer de veilige bandbreedte in duikt, in de laatste week van de maand. Fijn toch? We zitten alweer in de veilige zone, het gaat de goede kant op, de oversterfte neemt af. Die kop van het artikel klopt dus helemaal - als je alleen naar de afzonderlijke weekmarges kijkt.

But if you look back at the past month in your monthly report once a month, can't you get a little more out of your data, trends perhaps? Signs that require attention?

With the basic knowledge about period margins, you should also look at May as a whole and the weekly mortality in relation to each other. And on the basis of that, perhaps the connection with previous months, if there is reason to do so. This produces a different picture: over the entire month of May there was excess mortality, ultimately more than twice as high as the monthly signal value!

The real numbers of the past six months

So let's take a look back at the past six months. The weekly margins vary. In the first weeks, the weekly margins are still high (winter), the bandwidth is rapidly increasing. The arc is therefore not as regular as the example with a constant weekly margin. In the last weeks of the graph (the summer), the margins are more than half smaller, so we see the graph flattening out faster there. If we apply the same period adjustment again, but this time to the actual weekly margins as determined by CBS, the half-year margin looks like this:

The kink where winter mortality changes into summer mortality is clearly visible.

In the graph below, the purple line is the cumulative mortality deviation compared to the expected excess mortality (the baseline). Where the line decreases, there is less mortality and just no undermortality. After week 18, there is excess mortality. Week 18 is the first week of May 2022.

As early as August 2021, I drew attention to excess mortality. As a result of my articles and attention from Maurice de Hond, parliamentary questions were asked about this. Thierry Baudet tried to get it on the agenda. It was only months later that it was picked up more widely, Pieter Omtzigt submitted a motion (which was complied with through a surrogate exercise) and now structural excess mortality is a point of attention worldwide that is curiously ignored by the authorities.

Below is that first visual with the summer highlighted for 0-80 years: excess mortality or not? The article was from August 25, 2021. I am aiming for next week, August 25, 2022, to release the above check at least on this period. To celebrate the excess mortality anniversary.