Auteur: Jan van der Laan, Corine Witvliet-Penning, Suzanne Gerritsen, Agnes de Bruin
HSMR 2022 Methodological Report

4. Evaluation of the HSMR 2022 model

This chapter presents and evaluates the HSMR model results. Summary outcomes of the 158 logistic regressions are presented, with in-hospital mortality as the dependent variable and the variables mentioned in section 3.4 as explanatory variables. More detailed results are presented in the table “Statistical significance of covariates HSMR 2022 model” published together with this report. The regression coefficients and their standard errors are presented in the file “Coefficients HSMR 2022.xlsx”, published together with this report. 

4.1 Target population and data set

All hospitals that register complete records of inpatient admissions and prolonged observations without overnight stay in the LBZ are included in the HSMR model. In 2019, 2020 and 2021 all general hospitals and university hospitals were included in the model, as well as three short-stay specialised hospitals (two cancer hospitals and an eye hospital). In 2022 a fourth short-stay specialised (orthopaedic) hospital registered admissions in the LBZ and was included in the model. All of the hospitals had completely registered all admissions in the LBZ. 

The total number of hospitals included in the HSMR model of 2019-2022 is 75 and includes 64 general hospitals, 7 university hospitals and 4 short stay specialised hospitals. These numbers are based on the number of hospital units in 2022, counting hospitals that merged in 2022 as one unit for all years (2019-2022). 

4.1.1 Admissions in HSMR 2022 model
2019202020212022total
Excluded admissions not meeting the NZa criteria1)138,310123,889131,521143,399537,119
Excluded admissions of foreigners9,8256,2315,9937,97330,022
Excluded admissions due to COVID-192)40,43060,228100,658
Excluded admissions of healthy persons3)19,15118,20219,19618,94375,492
Total number of admissions included in model1,617,3711,395,5451,407,7491,481,6485,902,313
Number of inpatient admissions1,504,1301,288,2911,295,2551,367,7005,455,376
Number of observations113,241107,254112,494113,948446,937
Number of deaths included in model32,64729,15129,77936,207127,784
Crude mortality (in admissions in model)2.0%2.1%2.1%2.4%2.2%
1) Admissions that do not meet the billing criteria of the Dutch Healthcare Authority (NZa) for inpatient admissions, and for prolonged observations, unplanned, without overnight stay. The number of these admissions in the LBZ varies over the years, due to different registration instructions of DHD.
2) Admissions with COVID-19 as the main diagnosis (ICD-10 codes U07.1 (COVID-19, virus identified (lab confirmed)), U07.2 (COVID-19, virus not identified (clinically diagnosed)) and, from 2021 onwards, U10.9 (Multisystem inflammatory syndrome associated with COVID-19, unspecified)).
3) Admissions of healthy persons are admissions of healthy newborns, healthy parents accompanying sick children, or other healthy boarders.

Table 4.1.1 lists some characteristics of the admissions included in the HSMR 2022 model per year. Admissions not meeting the criteria of the Dutch Healthcare Authority and admissions of foreigners were excluded. For the years 2020 and 2021, admissions with COVID-19 as the principal diagnosis were also excluded. In addition, from 2019 onwards, all admissions of healthy persons were excluded. 

The number of admissions included in 2022 was 5% higher than in 2021, mostly due to the inclusion of admissions with COVID-19 as the principal diagnosis. However, the number of admissions was still 8% lower than in 2019, due to the ongoing overall impact of the COVID-19 pandemic on regular hospital care in 2022. The total number of admissions over all four years in the 2022 model was also 13% lower than in the 2019 model (6,777,534). 
Crude mortality of the admissions included in the HSMR model was increased in 2022 compared to mortality of previous years. This was partly due to the inclusion of the COVID-19 admissions, that had had a much higher in-hospital mortality rate (10.0%, data not shown), but mortality of the non-COVID-19 admissions was also increased (2.3%, data not shown) in 2022. 

The total number of admissions in the 2022 model (5,902,313; over all four years) was not only lower than in 2019, but was also 4% lower than in the 2021 model (6,124,777). This decrease is mostly due to the exclusion of the healthy persons: if those admissions had been included in the 2022 model, the number of admissions would have been only 2% lower when compared to that of the 2021 model. The additional decrease of 2% is probably due to the fact that the 2022 model is based on data of three pandemic years (2020-2022), while the 2021 model was based on data of two pandemic years (2020-2021) and two regular years (2018-2019). 

4.2 Hospital exclusion

Hospitals were only provided with (H)SMR outcomes if the data fulfilled the criteria stated in paragraph 3.5. In order to qualify for a three-year report (2020-2022) hospitals had to fulfil these criteria for the three consecutive years.
Of the 75 hospitals included in the model, all had registered (valid) data over 2022. The four short stay specialised hospitals and one general hospital had not been asked to grant authorization for providing HSMR numbers because their case mix was very different from that of the other hospitals. In fact, all of these hospitals had participated in the LBZ but the data of these hospitals did not meet one or more of the previously stated criteria, such as a minimum number of 60 registered deaths or on average a minimum number of 0.5 comorbidities per admission. All of the other 70 hospitals that had granted authorization fulfilled the criteria and were provided with a HSMR figure for 2022. For these 70 hospitals the data of 2020 and 2021 was additionally investigated in order to determine if a three-year report could be provided. The data of all 70 hospitals met the criteria in all years considered and so all hospitals were also provided with three-year HSMR figures.

4.3 Impact of the covariates on mortality and HSMR

The table “Statistical significance of covariates HSMR 2022 model” published together with this report shows which covariates have a statistically significant (95 percent confidence) impact on in-hospital mortality for each of the 158 diagnosis groups: “1” indicates (statistical) significance, and “0” non-significance, while a dash (-) means that the covariate has been dropped as the number of admissions is smaller than 50 (or as there are no deaths) for all but one category of a covariate; see section 3.6.2
The last row of the table “Statistical significance of covariates HSMR 2022 model” published together with this report gives the numbers of significant results across the diagnosis groups for each covariate. These values are presented again in table 4.3.1, as a summary, but ordered by the number of times a covariate is significant. As an additional model (COVID-19) was added this year, the numbers in the following tables have increased slightly compared to the previous year. Age, urgency of the admission, and severity of the main diagnosis are significant for a large majority of the diagnosis groups. This is also true for several of the comorbidity groups, especially for group 2 (Congestive heart failure and cardiomyopathy). Comorbidity 15 (HIV) is only rarely registered as a comorbidity; most diagnosis groups had fewer than 50 admissions with HIV comorbidity. In only one of the models comorbidity 15 was statistically significant. The number of times month of admission is significant varies over the years (CBS, 2018, 2019, 2020, 2021): it increased from 27 in 2017 to 43 in 2018, but in 2019 it decreased to 37, to 35 in 2020, to 33 in 2021 and to 28 in 2022. The number of times year of discharge is significant stabilized on 22 in the previous three HSMR models (CBS, 2020, 2021, 2022), but in the HSMR 2022 model it increased to 32, which was the largest increase of all covariates in the 2022 model. Compared to the HSMR 2021 model, other large changes were observed for the covariates comorbidity 3 (Peripheral vascular disease, significant in eight additional models) and comorbidity 12 (Hemiplegia or paraplegia, no longer significant in six models). The changes are smaller for the other covariates. The total number of significant covariates decreased from 1,632 in 2021 to 1,613 in 2022.

4.3.1 Statistical significance  of the covariates for the 158 logistic regressions (summary), HSMR 2022 model
CovariateNo. of significant results
Age140
Urgency125
Comorbidity 2123
Severity main diagnosis114
Comorbidity 13108
Comorbidity 3103
Comorbidity 16102
Source of admission98
Comorbidity 990
Comorbidity 688
Comorbidity 1467
Comorbidity 467
Comorbidity 559
Comorbidity 1748
Comorbidity 143
Sex40
Year of discharge32
Comorbidity 1030
Month of admission28
SES27
Comorbidity 1226
Comorbidity 723
Comorbidity 1122
Comorbidity 89
Comorbidity 151
Comorbidity groups: 1 Myocardial infarction, 2 Congestive heart failure and cardiomyopathy, 3 Peripheral vascular disease, 4 Cerebrovascular disease , 5 Dementia, 6 Pulmonary disease, 7 Connective tissue disorder, 8 Peptic ulcer, 9 Liver disease, 10 Diabetes, 11 Diabetes complications, 12 Hemiplegia or paraplegia, 13 Renal disease, 14 Cancer, 15 HIV, 16 Metastatic cancer, 17 Severe liver disease.

The significance only partially reflects the effect of the covariates on mortality. This is better measured using the Wald statistic. Table 4.3.2 presents the total of all Wald statistics summed over the diagnosis groups with the respective sum of the degrees of freedom, for each of the covariates ordered by value. It shows that severity of the main diagnosis has the highest explanatory power. Age and urgency of admission are also important variables. Of the comorbidities in the model, comorbidity groups 2, 16, and 13 are the groups with the most impact on mortality. Compared to the outcome of the 2021 model (CBS, 2022), the order of the covariates with regard to explanatory power is almost identical. Note that the values of the Wald statistics themselves cannot be compared directly between years as these values depend on the number of admissions used in the models. 

4.3.2 Wald chi-square s tatistics for the 158 logistic regressions, HSMR 2022 model
CovariateSum of Wald statisticsSum of df
Severity main diagnosis38,786401
Age30,0111,984
Urgency16,237155
Comorbidity 29,356146
Comorbidity 164,210140
Comorbidity 133,605151
Source of admission2,984283
Comorbidity 32,468147
Comorbidity 62,085152
Comorbidity 91,762125
Month of admission1,535790
Comorbidity 171,45164
Comorbidity 41,293123
Comorbidity 141,263143
Comorbidity 51,221120
SES1,023696
Year of discharge884469
Sex 773149
Comorbidity 1275398
Comorbidity 1638149
Comorbidity 10428153
Comorbidity 11328117
Comorbidity 7298130
Comorbidity 811432
Comorbidity 151812
Comorbidity groups: 1 Myocardial infarction, 2 Congestive heart failure and cardiomyopathy, 3 Peripheral vascular disease, 4 Cerebrovascular disease , 5 Dementia, 6 Pulmonary disease, 7 Connective tissue disorder, 8 Peptic ulcer, 9 Liver disease, 10 Diabetes, 11 Diabetes complications, 12 Hemiplegia or paraplegia, 13 Renal disease, 14 Cancer, 15 HIV, 16 Metastatic cancer, 17 Severe liver disease.

The explanatory power of year of discharge stabilized in the HSMR 2019 and HSMR 2020 models, increased slightly (+5%) in the HSMR 2021 model and shows a further increase of 13% in the HSMR 2022 model. This implies that the differences in mortality between the years in the model (corrected for differences in patient characteristics) has slightly increased, probably because of the influence of the COVID-19 pandemic on an increasing number of years in the model and the higher mortality of non-COVID-19 admissions in 2022.

Similar to previous years, the impact of comorbidity 1 (myocardial infarction) is decreasing, with a 45% decrease over the past seven models. In addition, the impact of comorbidity 14 (cancer) has decreased by 31% since the 2016 model. A decreased impact of a comorbidity could reflect a decreased effect of the comorbidity (e.g. the likelihood of dying in hospital when having this condition as comorbidity has decreased), and/or a decreased number of patients with this comorbidity resulting in less accurate estimates of the effect of this comorbidity (which also decreases the Wald statistic). The opposite applies in the case of an increased impact of a variable. 

The impact of the SES variable was relatively stable up to the 2020 model. In the 2021 model the explanatory power of SES increased by 12%, and the HSMR 2022 model shows a further increase of 10%. In total the impact of SES has increased by 23% compared to the 2020 model. This is probably a result of the use of updated values of the SES variable on the data of 2021 in the HSMR 2021 model and on the data of all years (2019-2022) in the 2022 model (see section 2.2 for more details on the update of the SES variable).

In addition, the impact of month of admission has also increased in the 2022 model (1,535) compared to the 2021 model (1,375). This is due to the inclusion of COVID-19 in the 2022 model, as month of admission is the second most important covariate in the COVID-19 model with a Wald of 270. This is probably caused by the monthly variation in the number of COVID-19 admissions and/or the likelihood of in-hospital mortality, which are in turn affected by the COVID-19 transmission waves in the population (four waves in 2022). During the influenza epidemic in 2018, the impact of month of admission was also increased compared to previous and consecutive models.

As was mentioned in section 3.6.2, when the hospitals differ little on a covariate, the effect of this covariate on the HSMR can still be small even if this covariate is a strong predictor for mortality. Table 4.3.3 shows the impact of each covariate on the HSMR, as measured by the formula in paragraph 3.6.2, for the hospitals for which HSMRs are calculated. The comorbidities, which are considered here as one group (17 comorbidities together), have the largest effect on the HSMR. This is caused by differences in case mixes between hospitals, but possibly also by differences in coding practices (see Van der Laan, 2013). Deleting sex as a covariate hardly has an impact on the HSMRs, whereas SES has a reasonable impact on the HSMR. This is probably because hospitals differ more in terms of SES categories of the postal areas in their vicinity than in terms of the sex distribution of their patients. In the HSMR 2022 model, the impact of SES increased compared to the 2021 model and now exceeds that of source of admission. For the other covariates in table 4.3.3 the order (ranked by magnitude of the effect) is the same as in the previous model. 

4.3.3 Average shift in HSMR 2022 by inclusion/deletion of covariates
CovariateAverage shift in HSMR
Comorbidity1)4.52
Age3.75
Severity main diagnosis2.40
Urgency1.95
SES1.03
Source of admission0.98
Month of admission0.20
Sex0.11
1) The comorbidities were deleted as one group and not separately.

4.4 Model evaluation for the 158 regression analyses

Table 4.4.1 presents numbers of admissions and deaths, and C-statistics for the 158 diagnosis groups. The C-statistic is explained in section 3.6.2. Overall the C-statistics have changed little compared to the previous model. Only two C-statistics showed a moderate change: after last year’s increase from 0.94 to 0.98, the C-statistic of “Cancer of testis and other male genital organs” decreased to 0.94. In addition, the C-statistic of “Other and unspecified benign neoplasm” decreased from 0.88 to 0.84. All other changes are smaller than 0.04 with most of them below 0.02. For 66 diagnosis groups the C-statistic did not change compared to last year. 
Only three of the 158 diagnosis groups have a C-statistic below 0.70: “Congestive heart failure, nonhypertensive” (70), “Chronic obstructive pulmonary disease and bronchiectasis” (82) and “Aspiration pneumonitis; food/vomitus” (84). For the diagnosis groups with a C-statistic below 0.7, the model’s ability to explain patient mortality is less than ‘good’. This increases the risk that the model does not completely correct for population differences between the hospitals. For the highest scoring diagnosis groups (above 0.9) the covariates strongly reduce the uncertainty in predicting patient mortality. In 2022, 56 diagnosis groups had a C-statistic above 0.9.

As mentioned in section 2.2, admissions of healthy persons (such as healthy newborns and other healthy boarders) are excluded in the present HSMR model. The large majority of these admissions were formerly part of diagnosis groups 157 (“Residual codes: unclassified”) and group 118 (”Complications of pregnancy, childbirth, and the puerperium; liveborn”). Smaller numbers of these admissions were previously part of diagnosis groups 131 (“Other perinatal conditions”) and 130 (“Intrauterine hypoxia, perinatal asphyxia, and jaundice”) mainly. The C-statistics of these four groups have not changed much in the HSMR 2022 model compared to the 2021 model (resp. -0.021, 0.018, 0.005 and 0.004). Therefore, excluding the admissions of healthy persons did not affect the ability of the models to predict mortality. 

4.4.1 C-statistics for the logistic regressions of the 158 main diagnosis groups, HSMR 2022 model
Diag. group no.Description of diagnosis groupNumber of admissionsNumber of deathsC-statistic
1Tuberculosis1,468450.88
2Septicemia (except in labor)12,3203,4900.73
3Bacterial infection, unspecified site9,7966060.80
4Mycoses2,4582770.79
5HIV infection717260.83
6Hepatitis, viral and other infections23,5712720.93
7Cancer of head and neck14,7852410.90
8Cancer of esophagus10,3505930.79
9Cancer of stomach11,2674390.82
10Cancer of colon42,6731,1250.84
11Cancer of rectum and anus17,6693840.86
12Cancer of liver and intrahepatic bile duct7,4994330.82
13Cancer of pancreas17,2959060.82
14Cancer of other GI organs, peritoneum8,6733900.79
15Cancer of bronchus, lung50,4744,2240.81
16Cancer, other respiratory and intrathoracic2,3331300.85
17Cancer of bone and connective tissue8,227970.91
18Melanomas of skin and other non-epithelial cancer of skin6,3141080.93
19Cancer of breast39,2163750.97
20Cancer of uterus9,0441270.93
21Cancer of cervix and other female genital organs10,597930.94
22Cancer of ovary8,2212530.85
23Cancer of prostate25,4903500.94
24Cancer of testis and other male genital organs6,46160.94
25Cancer of bladder53,9754440.93
26Cancer of kidney, renal pelvis and other urinary organs15,6492710.88
27Cancer of brain and nervous system11,9552640.79
28Cancer of thyroid5,955510.98
29Hodgkin`s disease1,880410.91
30Non-Hodgkin`s lymphoma22,7249640.83
31Leukemias21,5151,2110.80
32Multiple myeloma10,3784560.79
33Cancer, other and unspec. primary, maintenance chemotherapy
and radioth.
4,7011160.90
34Secondary malignancies79,0764,2450.77
35Malignant neoplasm without specification of site2,3863140.82
36Neoplasms of unspecified nature or uncertain behavior11,4102160.89
37Other and unspecified benign neoplasm64,352940.84
38Thyroid and other endocrine disorders23,0922200.91
39Diabetes mellitus without complication13,5091270.87
40Diabetes mellitus with complications23,5105040.85
41Nutritional deficiencies and other nutritional, endocrine,
and metabolic disorders
56,7704730.95
42Fluid and electrolyte disorders33,1168960.85
43Cystic fibrosis1,90050.88
44Immunity and coagulation disorders, hemorrhagic disorders10,6071760.87
45Deficiency and other anemia48,0154830.80
46Diseases of white blood cells8,8322330.75
47Mental, affective, anxiety, somatoform, dissociative,
and personality disorders
20,370700.88
48Senility and organic mental disorders10,0935350.71
49Schizophrenia, mental retardation, preadult disorders
and other mental cond.
6,209200.95
50Other psychoses2,763270.86
51Meningitis, encephalitis, and other central nervous system infections9,1625300.87
52Parkinson`s disease5,182900.84
53Multiple sclerosis and other degenerative nervous system conditions11,5442760.91
54Paralysis and late effects of cerebrovascular disease3,633600.86
55Epilepsy and convulsions42,8465870.88
56Coma, stupor, and brain damage2,1062010.90
57Headache and other disorders of the sense organs59,246430.93
58Other nervous system disorders52,3133520.94
59Heart valve disorders36,1288230.77
60Peri-, endo-, myocarditis, and cardiomyopathy21,6196690.87
61Essential hypertension, hypertension with compl.,
and secondary hypertension
12,7501060.93
62Acute myocardial infarction136,7593,3990.85
63Coronary atherosclerosis and other heart disease104,2787660.85
64Nonspecific chest pain143,800320.92
65Pulmonary heart disease30,8191,0430.80
66Other and ill-defined heart disease1,3931110.88
67Conduction disorders (heart disease)24,9033670.88
68Cardiac dysrhythmias175,0857700.90
69Cardiac arrest and ventricular fibrillation15,6625,7460.75
70Congestive heart failure, nonhypertensive126,1499,9860.67
71Acute cerebrovascular disease153,88912,9590.80
72Transient cerebral ischemia, and other cerebrovascular disease43,6253220.90
73Peripheral and visceral atherosclerosis45,6611,9240.91
74Aortic and other artery aneurysms27,0192,4460.89
75Aortic and arterial embolism or thrombosis11,1144440.85
76Other circulatory disease29,3555600.87
77Phlebitis, varicose veins, and hemorrhoids10,5641440.87
78Pneumonia107,7679,0950.75
79Influenza17,2867720.80
80Tonsillitis and upper respiratory infections49,684880.92
81Acute bronchitis26,597630.94
82Chronic obstructive pulmonary disease and bronchiectasis102,7576,2930.69
83Asthma23,518960.91
84Aspiration pneumonitis, food/vomitus8,3071,7100.64
85Pleurisy, pneumothorax, pulmonary collapse22,2866130.83
86Respiratory failure, insufficiency, arrest5,3461,6290.74
87Lung disease due to external agents1,7491630.77
88Other lower respiratory disease24,2309970.87
89Other upper respiratory disease49,1883030.90
90Intestinal infection44,8945420.89
91Disorders of mouth, teeth, and jaw20,399560.96
92Esophageal disorders12,1621150.91
93Gastroduodenal ulcer4,9742330.92
94Gastritis, duodenitis, and other disorders of stomach and duodenum7,312850.87
95Appendicitis and other appendiceal conditions70,092770.96
96Peritonitis and intestinal abscess5,1253590.80
97Abdominal hernia43,3064580.91
98Regional enteritis and ulcerative colitis17,343630.95
99Intestinal obstruction without hernia33,2451,5090.82
100Diverticulosis and diverticulitis35,2045060.92
101Anal and rectal conditions21,761500.94
102Biliary tract disease112,7621,0450.90
103Liver disease, alcohol-related7,3459180.72
104Other liver diseases17,8141,0580.82
105Pancreatic disorders (not diabetes)36,2308430.84
106Gastrointestinal hemorrhage36,5901,0660.80
107Noninfectious gastroenteritis11,1052180.80
108Other gastrointestinal disorders34,2557740.94
109Nephritis, nephrosis, renal sclerosis14,5061100.91
110Acute and unspecified renal failure15,1239980.76
111Chronic kidney disease13,4754570.87
112Urinary tract infections96,7272,7730.77
113Calculus and other diseases of urinary tract79,8231850.91
114Genitourinary symptoms and ill-defined conditions25,8471180.87
115Hyperplasia of prostate and other male genital disorders41,436490.92
116Non-neoplastic breast conditions12,90040.98
117Prolapse and other female genital disorders59,737360.98
118Complications of pregnancy, childbirth, and the puerperium,
liveborn
545,091160.86
119Skin and subcutaneous tissue infections52,8857860.89
120Other skin disorders, chronic ulcer of skin16,7942320.92
121Infective arthritis and osteomyelitis13,7133090.89
122Osteoarthritis, rheumatoid arthritis, and other musculoskeletal
deformities
227,381830.93
123Other non-traumatic joint disorders10,528350.92
124Spondylosis, back problems, and osteoporosis81,7221910.96
125Pathological fracture5,075730.83
126Other connective tissue disease31,2633040.97
127Cardiac and circulatory congenital anomalies8,9371670.89
128Noncardiac congenital anomalies26,1631990.95
129Short gestation, low birth weight, and fetal growth retardation76,2434690.88
130Intrauterine hypoxia, perinatal asphyxia, and jaundice61,4172130.95
131Other perinatal conditions225,5012150.95
132Joint disorders and dislocations, trauma-related, sprains and strains20,737300.98
133Fracture of neck of femur (hip)87,6462,7290.80
134Skull and face fractures, spinal cord injury11,3391960.90
135Fracture of upper limb41,7691330.95
136Fracture of lower limb49,7422880.94
137Other fractures43,8581,0320.85
138Intracranial injury39,4632,6990.77
139Crushing injury or internal injury20,3004370.91
140Open wounds of head, neck, and trunk5,595460.87
141Open wounds of extremities5,124370.95
142Complication of device, implant or graft96,6991,3360.88
143Complications of surgical procedures or medical care98,7808480.86
144Superficial injury, contusion58,1044750.92
145Burns3,743850.92
146Poisoning by psychotropic agents, drugs, or other medications36,2443480.85
147Other injuries and conditions due to external causes13,3606850.89
148Syncope42,0431260.84
149Fever of other and unknown origin19,2151010.82
150Lymphadenitis and gangrene3,883210.97
151Shock1,2734800.74
152Nausea and vomiting11,529700.85
153Abdominal pain37,8911110.95
154Malaise and fatigue8,8201430.82
155Allergic reactions10,868340.96
156Rehabilitation and other aftercare, medical examination/
evaluation/screening
75,2862190.85
157Residual codes, unclassified29,8311510.96
158COVID-1927,9882,8030.75

4.5 Evaluation of the COVID-19 model

As this is the first year for which admissions with main diagnosis COVID-19 are included in the HSMR, this section will specifically describe the COVID-19 model. Last year (CBS, 2022), a separate model for COVID-19 was estimated but the COVID-19 admissions were not included in the HSMR of 2021. The previous COVID-19 model was estimated using data from 2021. The current model is estimated using data from 2022. 

The current model has a C-statistic of 0.748. Although this is lower than the value 0.803 that was found for 2021, a C-statistic of 0.748 is still good. Between 2021 and 2022 there have been a number of changes in the data and admissions that might explain the difference in C-statistic. 

First, the amount of COVID-19 admissions has dropped from 60,228 to 27,988 and the crude mortality rate has dropped from 14% to 10%. Higher levels of acquired immunity (by vaccination or earlier exposure to COVID-19) in 2022 may be one of the factors that can have contributed to this.

Second, we see that some of the effects of the covariates have changed. Table 4.5.1 shows the Wald statistics and the statistical significance for each of the covariates in the current model. 
Age is still the most important predictor. However, where age explained approximately 75 percent of the variance in 2021, it explains about 50 percent in 2022. In 2022 a higher percentage of the COVID-19 admissions were in the age group >70 years and these admissions had considerably lower mortality than in 2021. Also there were more COVID-19 admissions of children in 2022, with almost zero mortality in both years. In the age-group 40-70 there were relatively less admissions in 2022, with a slightly higher mortality. Compared to 2021 the covariate ‘month of admission’ has become more important. Especially in the first months of 2022 the number of admissions was relatively high. The crude mortality rate was also somewhat higher in these months. Furthermore, there were four distinct COVID-19 admission waves in 2022 while in 2021 the number of monthly admissions was more constantly high. There are also some differences in other covariates. For example, with regard to the covariate ‘source of admission’ there were less admissions with source of admission equal to ‘other hospital’ in 2022 than in 2021. This is probably due to the fact that in 2022 there were less transfers of COVID-19 patients between hospitals.

All in all there have been several significant changes in the patient and admission characteristics in 2022 compared to 2021. The net result is that predicting mortality has become somewhat more difficult in 2022 for the COVID-19 admissions. 

4.5.1 Wald chi-square statistic and statistical significance of the covariates, COVID-19 2022 model
CovariateWaldSignificant
Age9521
Month of admission2701
Comorbidity 21241
Comorbidity 13711
Source of admission701
Comorbidity 14681
Sex401
Comorbidity 3281
Severity diagnosis211
Comorbidity 7191
Comorbidity 17131
Comorbidity 9121
Comorbidity 12111
SES50
Comorbidity 451
Comorbidity 651
Urgency30
Comorbidity 1130
Comorbidity 1630
Comorbidity 120
Comorbidity 510
Comorbidity 1000
Comorbidity 8--
Comorbidity 15--

The HSMR figures based on the model of 2021 did not include the COVID-19 admissions, whereas the current HSMR does, but only for the year 2022. To assess the impact of including the COVID-19 admissions in the HSMR 2022, we calculated the change in the HSMR outcomes with and without COVID-19. On average the HSMR for 2022 changes by 1.1 points (mean absolute difference between the HSMR of 2022 and that without including COVID-19), so removing the COVID-19 admissions has a small effect on the HSMR 2022. As the COVID-19 admissions are only included for 2022, the effect of removing COVID-19 admissions on the total HSMR for the three-year period 2020-2022 is smaller: 0.4 points. Figure 4.5.2 shows the change in the HSMR of 2022 for the individual hospitals. For most hospitals the change is smaller than 2 points. The maximum difference is 3.4 points.

4.5.2 Distribution of the changes in the HSMRs 2022 when COVID-19 would be excluded from the HSMR
diff2022_catn (Number of hospitals)
-4 to -31
-3 to -22
-2 to -114
-1 to 021
0 to 115
1 to 214
2 to 32
3 to 41
 

As shown in table 4.5.3, there are three hospitals that change from a significantly low HSMR 2022 to a non-significant HSMR 2022, or vice versa, when removing COVID-19 admissions. For the HSMR of 2020-2022, the HSMR of only two hospitals change from significantly low or high to non-significant (table 4.5.4). A comparison of HSMRs with and without COVID-19 admissions was also described in the methodological report of the HSMR 2021 (CBS, 2022) and showed larger differences (with 8 hospitals changing in significance categories). Thus the impact of in or excluding COVID-19 admissions on the HSMRs has decreased in 2022, which may be due to the lower number of COVID-19 admissions in 2022 and possibly the treatment of COVID-19 has also become more integrated in regular hospital care. 

4.5.3 Changes in significance (95% confidence) of the HSMRS for 2022 when removing COVID-19 from the HSMR
HSMR 2022 without COVID-19
HSMR 2022LowNot-significantHigh
Low1220
Not-significant1450
High0010

4.5.4 Changes in significance (95% confidence) of the HSMRS for 2020-2022 when removing COVID-19 from the HSMR
HSMR 2020-2022 without COVID-19
HSMR 2020-2022LowNot-significantHigh
Low1910
Not-significant0370
High0112

4.6 Regression coefficients

The file “coefficients HSMR 2022.xls” contains the estimated regression coefficients (columns “Estimate”), also called “log-odds”, for each of the 158 logistic regressions, as well as their standard errors (columns “Std. Err.”). The estimated regression coefficients are the elements of the estimate of vector 𝛽in the formula for p̂dhi (see section 3.6.1), for each diagnosis d. Notice that a β-coefficient has to be interpreted as the difference in log-odds between the category in question and the reference category (first category of the same covariate). For the sake of clarity, the reference categories are given in the first row of the corresponding covariates and these have by definition a zero coefficient for each regression. 

In many cases categories are collapsed (see section 3.6.2). This results in equal coefficients for the collapsed categories. If all categories were collapsed into one category for a certain variable and for a certain diagnosis group (i.e. if there was only one category with ≥50 admissions and ≥1 death), the variable was dropped from the model and all associated coefficients were set to zero. Therefore, the coefficients in the file “coefficients HSMR 2022.xls” can be used directly to calculate mortality probabilities, with the exception of two of the Charlson comorbidities (Comorbidity 17 and Comorbidity 11). If Charlson comorbidity 17 (Severe liver disease) contains <50 admissions or no mortality, it is collapsed with Charlson comorbidity 9 (Liver disease). In this case the coefficient of Comorbidity 17 is set to zero. When a patient has both comorbidities, it counts as only one comorbidity. Therefore, when the coefficient of Comorbidity 17 is zero in the coefficients file, one should first recode all Charlson 17 comorbidities to Comorbidity 9 and use the coefficient of Comorbidity 9. The same holds for Charlson 11 (Diabetes complications) when it is collapsed with Charlson 10 (Diabetes).