A reaction to the report "Systematische literatuuranalyse en internationale vergelijking 'Oversterfte'", Cochrane Netherlands, 3 april 2023
Authors (in alphabetical order): Bram Bakker, Bastiaan Geelhoed, Maurice de Hond, Jillis Kriek, Fritsander Lahr, Ronald Meester, André Redert, Herman Steigstra, Anton Theunissen
Also found on the LinkedIn page from Ronald Meester.
Date: April 24, 2023
Introduction and context
On 14 April 2023, Minister Kuipers sent the above report[1]to the House of Representatives. This gave the report a status that warrants a response. The report concerns a meta-analysis, which describes or compares several international studies on excess mortality. In principle, meta-analyses are useful, although conclusions are often difficult or impossible to formulate, precisely because of the fact that causality is extremely difficult to establish in an epidemiological setting in such a large case.
In the accompanying letter to the House of Representatives, Kuipers provides the following summary of the results:
"The study describes that it shows a trend of higher excess mortality (the difference between observed and expected mortality (per 1 million population)) in Eastern European countries and lower excess mortality in Scandinavian countries, Australia and New Zealand. In addition, the study also includes a review of COVID-19 mortality. The highest COVID-19 mortality (per 1 million population) was seen in Western European countries, while COVID-19 mortality was also the lowest in Australia and New Zealand. The differences in the trends of excess mortality and specific COVID-19 mortality could possibly be explained by differences in the reporting of COVID-19-related deaths as well as differences in the ability to test individuals for COVID-19. The researchers also indicate that international differences in excess mortality may be explained by measures against coronavirus, organisation of and access to care, population characteristics, behaviour, geographical location and differences between countries in data quality. Finally, the researchers point out that a number of studies show that excess mortality is lower among the population of fully vaccinated people."
There is much to be said about this quote, and in principle it is good that the uncertainties and difficulties in comparing studies are mentioned. However, the last comment in particular is rather misleading because it suggests causality. Because what does the report say? We quote on page 8: "Vaccinations appear to be associated with lower excess mortality, but since higher vaccination rates were associated with the emergence of the Omikron variant, no causal relationship can be established." Cochrane's wording thus paints a different picture from the minister's suggestion. Not only that: as we will explain below, no conclusion about a relationship between vaccination coverage and the extent of excess mortality can be drawn from the report.
Of course, we are not just concerned with how Kuipers informs the House of Representatives. Rather, we make some observations about the report itself. We first present a list of some comments on the report. Our list of points of interest (which we present below) is by no means exhaustive - in fact, it would require a much larger time investment. We then show that the Cochrane report uses and cites sources incorrectly. We end with a brief conclusion.
Some comments on the report
- The period 2020-2021 was probably chosen because of the Omtzigt motion. Since then, however, a wealth of information has emerged, in particular on the side effects and damage of vaccines and on the consequences of measures such as lockdowns. It is also precisely the over mortality that persists after vaccination that worries us, and which will not disappear after the arrival of the milder Omikron variant. Claims that may have seemed reasonable until 2021 do not hold up in the light of current insights. Thus, the report is already outdated when published and therefore of little or no use.
- Comparisons between different studies are almost impossible if methodologies differ. There is no systematic methodological assessment of the various studies in the report. In any case, it is not clear what system was actually followed in selecting and assessing the various studies. So the study is not very systematic at all. If you don't look very deeply at the methodological strengths and weaknesses of individual studies, don't think very carefully and deeply about them including all the pitfalls and biases, and only say that "several studies say this", then you don't really have much use for it.
- Even the term "excess mortality" itself has different definitions in different studies. [2] That in itself should be enough reason not to draw any conclusions from this report.
- The report writes on page 6 that excess mortality has increased in Eastern Europe. This is completely false, seeExcess mortality in 34 countries, failing vaccines? (virusvaria.nl)for a discussion on this claim, which can only result from a naive and incomplete view of the data. Furthermore, countries like Belarus, Iceland and Greenland are not suitable for a study like this for various reasons. That analysis also shows that what was mistaken for a correlation between vaccination coverage and excess mortality cannot simply be that; based on historical data.
- On page 7, we read that no studies were found in which vaccination and excess mortality were positively correlated. That can only mean that those studies were not included. Such studies do exist - in fact, there are many.[3]Indeed, here we see that the report was already dated when it was published.
- It is completely unclear how is dealt with the various sources of bias, as for example recently described inSources of bias in observational studies of covid-19 vaccine effectiveness - PubMed (nih.gov), in the various studies as well as in this report.
- Death certificates from at least CBS are shrouded in mystery, for various reasons. [4] For example, the WHO prescribes that when covid is listed somewhere on the death certificate, "Covid" must be recorded as the cause of death. There are many reasons why these data are not suitable as input for scientific research.
- It is now also clear that there are large differences between batches of vaccines. In Denmark, about 70% of adverse events were found to come from 4% of the batches; a strong indication of a relationship between vaccines and mortality. [5] This may make a meta-analysis rather useless.
- There are a lot of omissions in this report that are very relevant to this discussion on excess mortality, for instance with regard to the remarkably short protection against infection by the vaccines and the increasingly short protection by boosters. But we also mention the disappointing protection of the vaccines against new virus variants (a possibly important partial explanation of why the Omikron period was associated with high over-mortality rates), the now well-known side effects of the vaccines in the literature, which are quite worrying, the now well known differences between the various vaccine types and brands, and the striking over-mortality rates in the second half of 2021, all of 2022, and early 2023 in many highly vaccinated countries.
- The exclusion criteria on pages 11 and 12 cannot be reconciled with research questions 5 and 6.
- On page 13, we read, "In this way, different countries could be compared, and differences in follow-up duration between studies were corrected." The differences in follow-up duration are corrected by Cochrane by dividing the relevant variables (mortality/survival etc.) by the follow-up duration, so they were presented as rate (mortality per year). If a publication only zooms in on one wave, that wave gets hugely magnified when the values are converted to rates. A distinction should be made to individual waves. For each wave, you can convert values to rates so that countries can be compared. But now there is even the danger of comparing wave x in country A with wave y in country B on the basis of mortality rates, while in the meantime vaccinations may even have been rolled out.
- For baseline question 5 (international differences), only studies that presented data from more than 5 countries were selected. However, even if studies cover only 1 country, these studies can still be used to compare countries, provided appropriate normalisations and standardisations are done. Thus, we think that with this selection criterion, there is an unnecessarily restrictive selection of studies.
- Finally, we also note that there is an interesting difference between the version of the report sent to the House of Representatives and the version found on ZonMw's site.[6]On page 45, the ZonMw version reads: "Differences between excess mortality and COVID-19 mortality may be caused by different data quality." This sentence is not in the House of Representatives version. We find it confusing that there are apparently different versions in circulation, especially when it comes to comments that matter. This obviously does raise questions.
The use of resources
As a sample, we checked whether the Cochrane report used all sources correctly. This turned out not to be the case at all, and we illustrate this with some examples:
Example I
“Eén onderzoek presenteerde oversterfte in relatie tot vaccinatiegraad voor vier regio’s in de Verenigde Staten en voor drie verschillende tijdsperiodes (Tabel 11 en Excel bijlage). [62] Oversterfte was voor alle regio's het hoogste in de periode met de laagste vaccinatiegraad van ongeveer 7%.”
The cited publication is (we follow the numbering from the Cochrane report) [62]Stoto MA, Schlageter S, Kraemer JD. COVID-19 mortality in the United States: It's been two Americas from the start. PLoS ONE 2022; 17(4 April) Doi:https://dx.doi.org/10.1371/journal.pone.0265053.
Echter, de uitspraak in het Cochrane rapport is incorrect, omdat er waarschijnlijk over het hoofd is gezien dat er periodes waren waarin de vaccinatiegraad niet gegeven wordt (een leeg vakje in de tabel), maar waarin deze naar alle waarschijnlijk wel lager was dan in de overige periodes in de tabel waarin de vaccinatiegraad wel gegeven wordt (omdat vaccinatiegraad in de tijd toeneemt). Zo zien we dan dat bijvoorbeeld regio Northeast de laagste oversterfte (0,111) per capita heeft in de periode 31 mei 2020 - 3 oktober 2020, terwijl de vaccinatiegraad voor die periode niet gegeven wordt, maar waarschijnlijk fors lager was dan 7%, omdat de massavaccinaties toen nog niet begonnen waren (het zal dus 0% zijn of in ieder geval dicht bij 0% liggen). Dit zou dan juist weer op de tegenovergestelde conclusie (dan die in het Cochrane rapport) duiden. In het Cochrane rapport wordt er dus een conclusie gesuggereerd (dat lagere vaccinatiegraad tot hogere oversterfte leidt) die niet door de auteurs van het artikel wordt gemaakt. In Tabel 11 in het Cochrane rapport wordt dus selectief geciteerd.
Example II
The Cochrane report continues on page 42 with: "Seven studies described the relationship between excess mortality and vaccinations. Multiple studies showed that increased vaccine availability and use correlated with lower excess mortality (Appendix 11B). [17, 31, 62, 70, 73]".
From reference [62] we have already shown that the Cochrane report does not interpret it properly. Do the other references show the connection that the authors of the Cochrane report point out? We'll check.
[17]Bell E, Brassel S, Oliver E, et al. Estimates of the Global Burden of COVID-19 and the Value of Broad and Equitable Access to COVID-19 Vaccines. Vaccine 2022; 10(8) doi:https://dx.doi.org/10.3390/vaccines10081320
The conclusion of the Bell paper does not state that increased vaccine availability and use are correlated with lower excess mortality. It may be that the authors of the Cochrane report have come to their conclusion based on the following statement in the Bell paper: "Our model's results under Scenario 1 show that in 2021 1.4 million direct deaths, 4.3 million excess (direct and indirect) deaths and 6.0 million hospitalisations have been averted and hospital resources worth USD 59 billion have been saved by COVID-19 vaccinations." Regardless of whether this conclusion is correct (after all, it is based on a model/scenario), it is simply different from what is claimed in the Cochrane report.
Source [31] then? This is aboutFazekas-Pongor V, Szarvas Z, Nagy ND, et al. Different patterns of excess all-cause mortality by age and sex in Hungary during the 2nd and 3rd waves of the COVID-19 pandemic. GeroScience 2022 Doi:https://dx.doi.org/10.1007/s11357-022-00622-3.
Deze studie laat helemaal niet eenduidig zien dat (in de woorden van het Cochrane rapport) “een verhoogde beschikbaarheid en gebruik van vaccins een correlatie met een lagere oversterfte” zou hebben. Het laat een kleine afname van de “excess absolute mortality” zien van de tweede naar de derde golf terwijl gelijktijdig de vaccinatiegraad steeg. Maar dit effect hebben de auteurs van het Fazekas paper ook uitgesplitst naar leeftijd, en lijkt voornamelijk gedreven te worden door de groep van 65 jaar en ouder. De leeftijdsgroepen 35 – 44 jaar, 45 - 54 jaar en 55 - 64 jaar lieten alle drie een toename van de “excess absolute mortality” zien, die gepaard ging met een eveneens toenemende vaccinatiegraad.
Source [70] then? That'sWatson OJ, Barnsley G, Toor J, et al. Global impact of the first year of COVID-19 vaccination: a mathematical modelling study. The Lancet Infectious Diseases 2022; 22(9):1293-302. Doi:https://dx.doi.org/10.1016/S1473-3099%2822%2900320-6.
This is a mathematical modelling of how COVID-19 vaccination can reduce excess mortality. The model does not take into account any deaths due to vaccination, adverse events or vaccine (in)safety. While interesting, this study should not be used to draw conclusions regarding mortality due to vaccination, adverse events or (in)safety of the vaccines, as these are assumed a priori to play no role by the model used.
Source [73] then? This is aboutZhou F, Hu TJ, Zhang XY, et al. The association of intensity and duration of non-pharmacological interventions and implementation of vaccination with COVID-19 infection, death, and excess mortality: Natural experiment in 22 European countries. Journal of Infection and Public Health 2022; 15(5):499-507. Doi:https://dx.doi.org/10.1016/j.jiph.2022.03.011.
Deze publicatie laat juist het omgekeerde effect zien dan wat in het Cochrane rapport beweerd wordt. In Figuur 35 is er duidelijk te zien dat het “vaccine effect” op de oversterfte toeneemt met toenemende “vaccination coverage”. Dat de relative risks (RR) onder de 1 liggen is hierbij niet van belang. Van belang voor de correlatie tussen oversterfte en vaccinatiegraad is alleen het feit dat de RR's toenemen met toenemende vaccinatiegraad.
Discussion and conclusion
We did not have time to study ALL sources of the Cochrane report, but where we did, there were invariably problems. Incidentally, in our research into problems with sources, we have encountered many more problems than those we mention here. Of course, the question is how this is possible, but we're unable to answer that question.
For context, it might be worth noting that Cochrane's Dutch team, including the director of Cochrane Netherlands, consists exclusively of employees of the UMC Utrecht. The director of contractor Cochrane Nederland also performs work for the final client (VWS).
Based on the foregoing, our quality judgment is clear: this Cochrane report hardly makes a meaningful contribution to research on excess mortality.
References
[2] See for exampleComparison of pandemic excess mortality in 2020–2021 across different empirical calculations - ScienceDirect, as well as some other references in the Cochrane report.
[3] See for exampleIs there a Link between the 2021 COVID-19 Vaccination Uptake in Europe and 2022 Excess All-Cause Mortality? [v1] | Preprints.org,(16) (PDF) Australian COVID-19 pandemic: A Bradford Hill analysis of iatrogenic excess mortality (researchgate.net),(16) (PDF) Causal effect of covid vaccination on mortality in Europe (researchgate.net),(16) (PDF) A likelihood analysis of COVID-19 mRNA vaccine safety in the third booster campaign in The Netherlands (researchgate.net). There is much more to be found of course.
[4] See for exampleIs er echt oversterfte? - Onverklaarde oversterfte.
[5]Batch-dependent safety of the BNT162b2 mRNA COVID-19 vaccine - PubMed (nih.gov)
[6]Mortality report (zonmw.nl)
Ruin. They do it expressly. What failure settings there are.
And in the meantime, still not recognizing that lockdowns and face masks are very bad for public health. That closing schools is even worse than already thought and that there is certainly something wrong with having your own body cells generate Spike protein with engineered mRNA.
John Campbell on spike in organs nav Prof. Arne Burkhardt:
https://www.youtube.com/watch?v=kEE5OfiVS7o&t=1s
A meta-study like this is full of confounding variables (both by method and by time/phase of the pandemic), and is indeed hardly meaningful. To finally give a political answer to the question of whether vaccination is the driver of excess mortality (perhaps differentiated by age group), one MUST do targeted research: comparing cohorts – vaccinated (status, which jabs, how many, when?) versus unvaccinated, over a longer period of time. Take 'serious adverse events' with you, not just death. Only then will there be a reliable answer, everything else (as it is now used by Kuipers and van Gaalen) is, sorry to put it this way: rubbish! Ripe for the trash. So, Ernst, if you're reading along: finally set a study in motion, via Sun-MW, in which we include vaccination status, and compare cohorts 1 to 1. Surely that should be possible? Or am I too naïve in this?
That is already happening, research via ZomMW. We would like to see open data, and independent research, not from a VWS satellite.
Observational research has too many biases (See also Fung et al 2020 with last author Doshi last one I think was expelled from Cochrane for criticism)
Some systematic meta-analysis based on RCTs conducted by vaccine pharmacy itself is with me
by Stabell Benn et al (preprint with the Lancet, April 2022) which seems to show that the mRNA vaccines used worldwide in particular cause more harm (in terms of mortality) in the medium term than good. I don't think this article was ever published in any magazine and stuck in preprint. Would also have placed bomb under the mRNA.
The presumably harmful long-term effects have been killed by being found on the basis of a generically found relative!! risk reduction of contamination! (applicable to the relatively young pricked population) hastily decided to inject the placebo group with 'vaccine' as soon as possible.
People don't want to know.
The Cochrane report is a missed opportunity, it is a fairly simple literature review at candidate level university, and certainly not a meta study.
Comparison of international figures is especially important for limited groups of regions that already have many characteristics in common, but differ in some areas. The Netherlands, Belgium, east of England, west of Germany for example.
The comparison is meaningless for a 'hodgepodge' of countries, here and there around the world, even located in different parts of the world.
A real meta-study could therefore have been an exploration of international (statistical) modelling. A modeling mainly focused on policy evaluation. After all, the extent and development of excess mortality says something about the performance of the national healthcare system and the effectiveness of policy measures.
That can help answer questions. For example, the question: why is the pandemic mortality in the Netherlands over 2020-2022 no less than 1.5-2X as high as in North Rhine Westphalia, right across the border, a region with an almost equal population size, but with more people over 80?
But yes, policymakers should be interested in the answers to such questions...
North Rhine-Westphalia has done worse than the Netherlands for age (ASMR).
Deaths in NL all wrongly pushed under C19?
https://www.mortality.watch/?q=%257B%2522c%2522%253A%255B%2522Netherlands%2522%252C%2522DEU%2520-%2520Nordrhein-Westfalen%2522%255D%252C%2522t%2522%253A%2522asmr%2522%252C%2522ct%2522%253A%2522fluseason%2522%252C%2522cs%2522%253A%2522line%2522%252C%2522df%2522%253A%25222009%252F10%2522%252C%2522dt%2522%253A%25222021%252F22%2522%252C%2522sb%2522%253A1%252C%2522ce%2522%253A0%252C%2522m%2522%253A0%252C%2522pi%2522%253A1%252C%2522sl%2522%253A1%252C%2522v%2522%253A1%257D
Ward, maybe geography, further east, less coastal? Mortality in our northern provinces was also almost Swedish.
But compare NRW with the whole of Germany. Doesn't make sense either!
Anton, Ward thank you for your comments.
It is true that the annual total mortality in North Rhine-Westphalia (NRW) is greater than the total mortality in the Netherlands (NL). In the period 2016-2019 it is approximately 36% higher.
This is mainly because the ageing wave in NRW is somewhat more advanced than in the Netherlands. For example, there are about 40% more over-80s in NRW. Although the total population size is almost the same and the composition by age groups does not differ much, the group over 80 is very decisive for mortality.
Furthermore, I don't see many factors that could potentially lead to higher mortality rates in NRW compared to NL, except for the influence on the health of heavy industry and mining in the past, especially in the 60s and 70s of the last century.
Incidentally, the higher total mortality in NRW does not make the problem I mentioned earlier smaller but bigger: why is the Covid-19 mortality in NRW much lower than the excess mortality in the Netherlands? Surely you would expect that this mortality should have been much higher?
We can also roughly express the problem in numbers. Based on the 32K Covid-19 deaths in NRW (see: https://www.mags.nrw/coronavirus-statistiken), you would expect the mortality for NL to be either approx. 23.5K or 26K. In the first case we divide 32K by a factor of 1.36 (see above), in the second estimate we only divide the 20K deaths in the 80-plus group in NRW by 1.40 (after all, 40% more over-80s).
In both cases, we "standardize" the NL Covid-19 mortality on that of NRW.
If you compare these estimates with cbs' 45K excess mortality for the Netherlands at the moment, the problem is clear: you end up with an unexplained or perhaps avoidable excess mortality of about 20K.
Conclusion: further research desirable, including other regions and countries such as Belgium, standardization per phase of the pandemic.
Nice analysis, JVI!