3. Outcome of the 2023 model

3.1 Dataset

Table 3.1.1 shows the number of hospitals that were included in the model. All general and university hospitals could be included in both study periods (2022 and 2023). Two university hospitals merged in 2022 and data of a specialised hospital that started registering data in the LBZ in 2022 were included. Specialised hospitals where patients are mostly treated for oncological disease (two in 2022 and 2023) were excluded. 

The number of index admissions included in the model, the total number of identified readmissions and the unadjusted readmission rate for both study periods are listed in Table 3.1.2. The number of admissions per year is comparable to the number of admissions in 2021 (1,039,097), but higher than in 2020 (742,733) (Van der Laan et al. 2023). The number of admissions did not return to the number before the COVID-19 pandemic (1,223,296 in 2018; Van der Laan et al. 2020). The unadjusted readmission rates, however, are quite stable: 9.0% in 2017, 8.9% in 2018, 8.8% in 2020, 8.6% in 2021, 8.7% in 2022 and 8.8% in 2022.

3.2 Impact of the covariates on readmission rate

Appendix I shows which covariates have a statistically significant (95 percent confidence) impact on the readmission rate for each of the 123 regression models (one for each diagnosis group, including the diagnosis group COVID-19). Tables 3.2.1 and 3.2.2 show the total number of significant covariates and the total Wald statistics for the 123 regression models. The tables are sorted in descending order (most important variables at the top). The first table shows the number of diagnosis groups in which a variable is significant in the model. The effect of variables on the predicted probabilities, and, therefore, the importance of the variables for the case mix correction performed by the models, is better measured with the Wald-statistics (shown in the second table).

The order of the variables differs somewhat in both tables, but in both tables age, Charlson comorbidity 13 (Renal disease), urgency and severity are in the top 5 of the most important variables for model estimation. Except for Charlson comorbity 13, this is also the case for the HSMR 2023 model (Statistics Netherlands, 2024), indicating that these variables are relevant for both predicting readmissions and in-hospital mortality.

For the 2023 readmission model, sex and Charlson comorbidity 10 (Diabetes) are more important than for the 2023 HSMR model. Vice versa, the Charlson comorbidities 16 (metastatic cancer) and 2 (congestive heart failure) are more important for the 2023 HSMR model than for the 2023 readmission model. Apparently congestive heart failure has a stronger influence on estimating in-hospital mortality, while diabetes has a stronger influence on estimating readmissions. The difference in importance of Charlson group 16 (metastatic cancer) in both models can be explained by the fact that cancer-related main diagnoses are excluded from the readmission model, since planned readmissions for those diagnoses are frequent.

Compared to the 2021 readmission model (Van der Laan et al. 2023), the current model was based on more data. As a result, the number of significant covariates and Wald statistics was slightly higher in the current model than in the 2021 model. However, as the number of admissions did not return to pre-2020 values, the number of significant covariates and Wald statistics is lower than in the 2018 model. The order of the variables remained largely comparable.

3.3 Model evaluation for the 123 regression analyses

Appendix II shows the Areas Under the Curve (AUCs; also known as C-statistics) for each of the 123 regression models. From these AUCs it can be concluded that most models have weak predictive power. This is comparable with earlier models. Of the 123 diagnosis groups, only 20 have an AUC of 0.70 or above. This is one more than for the 2021 model. The diagnosis groups with a AUC of 0.70 or above are largely (14/20) the same as in 2021. The average AUC is 0.649, which is nearly the same as for 2021 (0.651). However, compared to 2021 there are a number of models for which the AUC has changed substantially: both in positive and negative direction. There are 22 models for which the AUC has changed more than 0.02 and 9 models for which the AUC has changed more than 0.04. See table 3.3.1 for an overview.

It is not always clear why the predictive power of the models has changed. The COVID-19 pandemic has had a significant impact on admissions in hospitals. For example, there were substantially less influenza admissions in the 2021 model compared to 2018. However, the number in the 2023 model has returned to the level in the 2018 model. The AUC of influenza is also back to the value it had in 2018. However, in general there was no clear relationship between (changes in) the number of admissions and the AUC of a diagnosis group.

3.4 Regression coefficients

Appendix III contains the estimated regression coefficients (columns ‘Estimate’) for each of the 123 logistic regressions as well as their standard errors (columns ‘Std. Err.’). For the sake of clarity, the reference categories are given in the first row of the corresponding covariates, and by definition have the coefficient zero for each regression. In many cases categories are collapsed, which results in equal coefficients for the collapsed categories. If all categories were collapsed into one category for a certain variable and for a certain diagnosis group (i.e. if there was only one category with ≥50 admissions and ≥1 readmission), the variable was dropped from the model and all associated coefficients were set to zero. The significance of each of the coefficients is shown in Appendix I.

3.5 Limitations

The readmission indicator has largely the same limitations as the HSMR. Some issues that are specific to the readmission indicator are addressed below.

• In principle all readmissions are included in the model: planned and unplanned; related and not related to the index admission. Ideally, only unplanned readmissions should be included, but those are not registered as such in the LBZ. Although the LBZ contains the variable urgency (acute versus not acute; an admission is registered ‘acute’ if care is needed within 24 hours), this variable does not accurately reflect the difference between planned and unplanned readmissions. To avoid the inclusion of planned readmissions, some diagnosis groups where planned readmissions are likely (for example the various groups concerning cancer) are excluded as index and readmissions. Diagnoses that are likely planned readmissions (for example follow-up care and rehabilitation) are also excluded as potential readmissions. Furthermore, in the present model (planned) transfers are excluded as readmissions. However, some planned readmissions will likely remain in the dataset.
• Unlike with the HSMR, Statistics Netherlands does not provide readmission ratios for 2023, based on the model of 2023. DHD will use the estimated models to calculate the ratios using hospital data from 2024. This means that the models are applied to a different year than that on which they were estimated. As was shown for the readmission model 2015 (Van der Laan et al. 2017), this results in a bias and extra variance.
• It is difficult to predict readmissions using the variables present in the models: the models explain only a small part of the observed variation. This makes it more likely that there are unobserved population differences that are not corrected for, that influence the readmission probability. This means that some of the differences in the current readmission ratio can be caused by unobserved population differences.
• The model identifies intra-hospital readmissions only and readmissions that occur in another hospital are not identified. As a result, the readmission ratio for hospitals where patients are often readmitted in another hospital could be an underestimation, and vice versa.