Mortality differences between ICUs that are regarded as ‘in control’: a longitudinal register-based study in the Netherlands, 2013–2023

BMJ Open, vol. 16, pp. e107572

Abstract

OBJECTIVES: Funnel plots are used to identify intensive care units (ICUs) with a higher than expected risk-adjusted mortality. ICUs with a standardised mortality ratio (SMR) within pre-defined control limits (often the 99.8% CL) are regarded as 'in control' and not labelled as a potential outlier for a particular calendar year. However, increased mortality rates not due to random fluctuations within and across the calendar years may be overlooked. We examined whether statistically significant and relevant differences in mortality over time between ICUs regarded as 'in control' are present. DESIGN: A longitudinal register-based study. SETTING AND PARTICIPANTS: 88 ICUs in the Netherlands registering the admissions of all critically ill patients in the National Intensive Care Evaluation registry in the Netherlands from 2013 to 2023. PRIMARY OUTCOME MEASURE: Hospital death analysed in a multivariable logistic regression analysis with a random intercept for ICU. The random intercept variance was translated to the median OR (MOR). RESULTS: 877 ICU-calendar year combinations were included, covering 759 498 unique admissions. The MOR increased from 1.12 (95% CI 1.10 to 1.15) for ICU-calendar year combinations with an SMR within the narrowest 95% CL (N=677) to 1.20 (1.17 to 1.24) for combinations with an SMR within the expanded 99.8% CL (including adjustment for overdispersion) (N=194) and to 1.21 (1.17 to 1.25) when including all ICU-calendar year combinations. Similar results were found for separate calendar years and separate diagnostic groups. CONCLUSIONS: These results show differences in mortality between ICUs that were not labelled as outliers. Assessment of mortality performance should integrate cross-sectional funnel plots, the MOR and longitudinal trends in the SMR to better capture persistent patterns of excess risk.

Authors 6

1. Fabian Termorshuizen corresponding

   Amsterdam UMC Location University of Amsterdam · Nationale Intensive Care Evaluatie Stichting · Amsterdam Public Health · Amsterdam University Medical Centers

   Affiliation as printed

   Department of Medical Informatics, Amsterdam UMC Locatie AMC, Amsterdam, The Netherlands

   National Intensive Care Evaluation (NICE) Foundation, Amsterdam, The Netherlands

   Quality of Care, Amsterdam Public Health Research Institute, Amsterdam, The Netherlands

2. Sylvia Brinkman

   Amsterdam UMC Location University of Amsterdam · Nationale Intensive Care Evaluatie Stichting · Amsterdam Public Health · Amsterdam University Medical Centers

   Affiliation as printed

   Department of Medical Informatics, Amsterdam UMC Locatie AMC, Amsterdam, The Netherlands

   National Intensive Care Evaluation (NICE) Foundation, Amsterdam, The Netherlands

   Quality of Care, Amsterdam Public Health Research Institute, Amsterdam, The Netherlands

3. M. Sesmu Arbous Aachen Department of Clinical Epidemiology Department of Intensive Care Medicine

   Leiden University Medical Center · Nationale Intensive Care Evaluatie Stichting

   Affiliation as printed

   Department of Clinical Epidemiology, Leiden University Medical Center, Leiden, The Netherlands

   Department of Intensive Care Medicine, Leiden University Medical Center, Leiden, The Netherlands

   National Intensive Care Evaluation (NICE) Foundation, Amsterdam, The Netherlands

4. Dave Anton Dongelmans

   Amsterdam UMC Location University of Amsterdam · Nationale Intensive Care Evaluatie Stichting · Amsterdam University Medical Centers

   Affiliation as printed

   Department of Intensive Care Medicine, Amsterdam UMC Locatie AMC, Amsterdam, The Netherlands

   National Intensive Care Evaluation (NICE) Foundation, Amsterdam, The Netherlands

5. Nicolette F. de Keizer

   Amsterdam UMC Location University of Amsterdam · Nationale Intensive Care Evaluatie Stichting · Amsterdam Public Health · Amsterdam University Medical Centers

   Affiliation as printed

   Department of Medical Informatics, Amsterdam UMC Locatie AMC, Amsterdam, The Netherlands

   National Intensive Care Evaluation (NICE) Foundation, Amsterdam, The Netherlands

   Quality of Care, Amsterdam Public Health Research Institute, Amsterdam, The Netherlands

6. Ferishta Bakhshi‐Raiez

   Amsterdam UMC Location University of Amsterdam · Nationale Intensive Care Evaluatie Stichting · Amsterdam Public Health · Amsterdam University Medical Centers

   Affiliation as printed

   Department of Medical Informatics, Amsterdam UMC Locatie AMC, Amsterdam, The Netherlands

   National Intensive Care Evaluation (NICE) Foundation, Amsterdam, The Netherlands

   Quality of Care, Amsterdam Public Health Research Institute, Amsterdam, The Netherlands