Hospital Readmission Ratio: Methodological report 2023 model

1.1 Indicators of quality of hospital care

Overall quality of hospital care can be estimated using several types of quality indicators based on hospital admission data. Such indicators for identifying potentially suboptimal quality of hospital care might focus for example on unexpected in-hospital or post-discharge mortality, potentially preventable hospital readmissions or unexpected long duration of admissions. In the Netherlands, hospital admission and discharge data is registered in the LBZ, the national hospital discharge register covering all general, university and a few specialised hospitals. Other specialised clinics, independent treatment centres and private clinics are not included. Inpatients as well as day cases and prolonged observations without overnight stay are registered. Administrative data of the admission as well as diagnoses and procedures are registered for each hospital discharge.

In the Netherlands, Dutch Hospital Data (DHD) annually provides hospitals participating in the LBZ registration with a set of indicators based on their performance in the previous year. Up to 2016 this set included the (unadjusted) hospital readmission rate, which is the ratio of the number of observed readmissions to the total number of hospital admissions. However, this ratio does not correct for case mix differences and might therefore not correctly reflect differences between hospitals in the true number of potentially preventable readmissions. DHD has therefore asked Statistics Netherlands in 2017 to develop a model to estimate the expected readmission risks adjusted for relevant covariates, in a fashion similar to the estimation of the hospital standardized mortality rates (HSMR). Since then, the model has been updated in 2018, 2019, 2020 and 2023. This report describes the most recent update conducted in 2025, based on model years 2022-2023.

1.2 Predictive value of the hospital readmission model

Internationally, models for estimating hospital readmission rates are used for the purpose of risk stratification but also as a quality indicator. Previous studies show that several patient characteristics contribute to the risk of being readmitted to the hospital. An overview in a systematic review by Kansagara et al. (2011) shows the various validated models that have been used internationally, the covariates included in those models and their overall predictive value. Common covariates include comorbidity indexes, age, sex and/or prior use of medical services (hospitalizations). Regardless of the number of included covariates, only a small fraction of the models are moderately discriminative (AUC/C-statistic>0.70). The model developed by Statistics Netherlands includes similar covariates as well as additional covariates such as severity of the main diagnosis, urgency of the admission and socio-economic status. However, the overall predictive value of the model did not exceed previously published values (AUC=0.69). The level of case mix correction applied by the model did however significantly improve comparability of outcomes of the individual hospitals with the national average. In other words: although the case mix correction is probably incomplete, it does, to some extent, reduce the confounding effect of differences between hospital patient populations. As such, applying the model to calculate adjusted readmission ratios for individual hospitals is an improvement over calculating crude rates (Van der Laan et al. 2017). Additionally, despite the limited discriminative ability, the adjusted readmission ratio might provide insight in e.g. diagnoses or specialties within hospitals where quality of care can be improved (Hekkert et al. 2018).

1.3 Development of the hospital readmission model in the Netherlands

The initial hospital readmission model, developed by Statistics Netherlands in 2017, was based on the linkage of admissions and readmissions that occurred within the same hospital (intra-hospital readmissions). In 2018 Statistics Netherlands improved this intra-hospital readmissions model by excluding planned transfers to and from neighbouring or specialized hospitals (‘2016 model’; this model was based on LBZ data of 2015 and 2016 and was named after the most recent year of included data). It is common practice for hospitals to refer inpatients to other hospitals for specific procedures, such as coronary interventions. Such planned transfers should not be labelled as readmissions.

The results of this improved intra-hospital model were compared to that of a newly developed inter-hospital model, that also took into account readmissions in other hospitals, while excluding planned transfers. Since readmissions can also take place in other hospitals, including inter-hospital readmissions in the model might improve its predictive value. However, the predictive value of both models was largely comparable, and it was concluded that apart from views regarding the relevance of inter-hospital readmissions for measuring quality of care, practical considerations might determine which of both models will be used for calculating the readmission ratios of the individual hospitals (Van der Laan et al. 2018). A practical disadvantage of the inter-hospital ratio is that hospitals need patient information from other hospitals to study the files of the patients with readmissions and to calculate the ratios themselves. Hekkert et al. (2019) also found a limited overall impact of including readmissions in other hospitals in the Netherlands. For these reasons, DHD decided to use the intra-hospital model (excluding planned transfers) in their regular hospital indicators reports.

1.4 Aim of the current project

This year (2025), Statistics Netherlands updated the intra-hospital model (‘2023 model’), excluding planned transfers, based on LBZ data of 2022 and 2023. The outcome is described in chapter 3.

1.5 Output

Statistics Netherlands only calculates the model for estimating the hospital readmission risks, adjusted for relevant covariates, not the outcomes for the individual hospitals. DHD will use the present model, based on LBZ data of 2022-2023, to estimate the expected readmission risk for each individual primary (index) hospital admission in 2024. For each hospital, the standardized (adjusted) readmission ratio can be calculated as the observed number of readmissions (x 100) divided by the sum of the expected readmission risks of the index admissions of that hospital. DHD will use these outcomes for their indicators reports for each hospital.

2.1 Changes compared to the previous intra-hospital model

In the present ‘2023 model’ we largely used the same methods as in the previous intra-hospital model (‘2021 model’), which excludes transfers as readmissions (Van der Laan et al. 2023).

• All admissions that met general criteria for index admissions were included for the full two-year period (in the previous model index admissions in the first wave of the COVID-19 pandemic in 2020 were excluded as hospital care was disrupted during this period, and all COVID-191 index admissions in 2020 were excluded).
• The covariate ‘month of admission’ of the COVID-19 model was the same as for the models of other diagnosis groups, consisting of two-month categories. It was no longer necessary to use the more detailed ‘month of admission’ variable that was used in the previous COVID-19 model.
• In line with the 2023 HSMR-model, socio-economic status (SES) scores were based on 2021 data of Statistics Netherlands.

2.2 Readmission ratio

The (hospital) readmission ratio is calculated using the observed readmission as the numerator and the expected (hospital) readmission risk as the denominator. The expected readmission risk is predicted for each individual admission within a given period, adjusted for patient and admission characteristics of that admission as covariates. Readmission risk was predicted for all (index) admissions that could potentially be followed by a readmission, excluding admissions for diagnoses with complex care paths where planned readmissions are often involved. Readmissions are defined as those admissions that occurred within 30 days of the discharge date of the preceding index admission. More information on the criteria of index admissions and readmissions is given in paragraphs 2.3.5 and 2.3.6 respectively.

The expected readmission risk is determined separately for each of the included diagnosis groups, based on the CCS (Clinical Classifications Software). The CCS clusters ICD codes of the main diagnoses of the admissions into 259 clinically meaningful categories (see https://www.cbs.nl/-/media/_excel/2024/43/classification-of-variables-hsmr-2023.xlsx), which were clustered further into 158 diagnosis groups. This clustering is in accordance with the HSMR and similar to the SHMI (Summary Hospital-level Mortality Indicator) in the UK (HSCIC, 2016). To determine the readmission risk we used logistic regression models, with an observed readmission as the target (dependent) variable and various patient and admission characteristics available in the LBZ as covariates.

The methodology for estimating the expected readmission risk is very similar to that used for estimating expected mortality rates applied for calculating the HSMR rates, described in detail elsewhere (Statistics Netherlands, 2024). In the following section, we therefore briefly describe the applied methods, while deviations from the HSMR methodology or other methods specific to the current project are described in more detail.

2.3 Target population and data set

2.3.1 Patient identifier

Statistics Netherlands linked the LBZ data to the Dutch national population register using a pseudonym of the national personal identification number, and the combination of date of birth, sex and postal code as linkage keys. With this linkage >99% of all admissions could be uniquely linked to a person in the population register; thus the loss of data was minimal (<1%). Through linkage, a unique pseudonymised person ID could be added to the LBZ dataset. Using this identifier not only allows identification of transfers to other hospitals, it also eliminates bias due to administrative errors in hospital-specific patient numbers.

2.3.2 Admissions – general criteria

We consider both the population of hospitals and the population of admissions. Our population of (re)admissions consists of “all hospital stays (inpatient admissions) of Dutch residents in Dutch short-stay hospitals within the study period”. In the LBZ, the date of discharge, and not the day of admission, determines the LBZ year a record is assigned to. The following admissions were excluded:

• day cases and prolonged observations. Subsequent readmissions might be elective in these cases, e.g. for prolonged treatment.
• incomplete admissions without a registered main diagnosis. These are rare, however, as hospitals have to register inpatient admissions completely.
• admissions that do not meet the billing criteria of the Dutch Healthcare Authority. This primarily, but not exclusively, concerns one-day inpatient admissions where the patient returned home after discharge. Based on an algorithm using LBZ data DHD has added a variable to the LBZ dataset (from 2019 onwards) that indicates whether the admission meets the billing criteria or not. This variable was used to exclude the admissions not meeting the billing criteria.
• admissions of foreigners. Readmissions of foreigners might take place in a hospital in their residential country, foreigners cannot be linked to the Dutch population register and the number of admissions of foreigners is relatively small.
• admissions of healthy persons. These are for example admissions of healthy newborns, a healthy parent accompanying a sick child, or other healthy boarders. These are identified based on the main diagnosis of the admission (ICD-10 code Z76.2-Z76.4) or based on procedure codes. Admissions were excluded based on procedure codes if for each bed-day of the admission a procedure code for a stay of a healthy person has been registered (Dutch procedure codes 190032, 190033 (‘Zorgactiviteiten’ codes), 339911 or 339912 (‘CBV’ codes)).
• duplicate admissions with identical values for date and time of admission and of discharge in combination with identical values for either (1) hospital ID and hospital-specific patient ID or (2) the pseudonymised person ID. In case of duplicate admissions, the admission with the lowest LBZ registration number was removed and the one with the highest number was kept, since we assumed that the latter admission might have been registered as a corrected version of the first. Duplicate admissions rarely occur in the LBZ.

2.3.3 Hospitals

Hospitals report admission data (hospital stay data) in the LBZ. However, not all hospitals participate in the LBZ. In principle, the hospital readmission risk model includes all general hospitals, all university hospitals and short-stay specialised hospitals with inpatient admissions participating in the LBZ in the study period. However, two of the three short-stay specialised hospitals participating in the LBZ were excluded from the hospital readmission model as these two hospitals treat patients with oncological diseases, for which admissions are excluded from the data (see paragraph 2.3.5).

The readmission ratio is calculated using LBZ data on admissions, using the pseudonymised personal ID as the unique key for identifying (re)admissions. The combination of the person ID (for identifying patients) and the hospital ID number (for identifying the same hospital) was used for linking admissions. In case of merging hospitals, the hospital ID number that the hospital used in the LBZ registration year, was used for the associated study period in the models. For example, two hospitals that had merged in study period twere analysed as separate units for study period t-1 and as a single unit for study period t. Otherwise, if the merged hospital (C) ID was also used for study period t-1, the year in which the unmerged hospitals (A and B) were still operating separately, an admission in hospital A followed by an admission in hospital B, could then potentially be identified as an index admission - readmission combination in hospital C. This would result in the identification of readmissions that in reality were admissions in another hospital.

2.3.4 Study periods

For the calculation of the current model, LBZ data of 2022 and 2023 was used. Previously we have shown that to identify the highest percentage of readmissions ending in year t, using index admissions with a discharge date from November 1^st of year t-1 up to October 31^st of year t (study period) is optimal (Van der Laan et al. 2017). Thus, for study period 2022 (‘year’=2022 in the model) we selected index admissions with a discharge date from November 1^st 2021 up to October 31^st 2022 and for study period 2023 we selected index admissions with a discharge date from November 1^st 2022 up to October 31^st 2023 (‘year’= 2023 in the model), see figure 2.3.4.1. The occurrence of readmissions was analysed in the period between November 1^st 2021 up to December 31^st 2023. The study periods are displayed in figure 2.3.4.1. If hospitals had merged in study period t, the hospital ID of the merged hospital was also used for the data of November and December of year t-1.

The study period of the 2023 model is similar to the 2018 model but differs from the study period applied for the 2021 model. In the 2021 model, index admissions from February 1^st up to May 31^st 2020 were excluded because of the first wave of the COVID-19 pandemic in the Netherlands which severely disrupted hospital care.

A study period t-1 using hospital IDs from year t-1 for all admissions   
B study period t using hospital IDs from year t  for all admissions

Figure 2.3.4.1 shows which LBZ data is included in both study periods. To process index admissions, transfers and readmissions, two separate datasets (A and B in figure 2.3.4.1) for each of one of the study periods were constructed, instead of a single dataset containing data from both study periods. This is necessary to link admissions in a specific hospital to the correct hospital ID in case of a merger of hospitals. For example, if hospitals X and Y in year t-1 merge into hospital Z in year t, admissions in these hospitals are registered under hospital IDs X and Y in year t-1, and under hospital IDs Z in year t. This is a problem, as November and December of year t-1 are not only part of study period t-1, but also of study period t. Admissions registered under hospital IDs X and Y in year t-1 cannot be linked to hospital Z in year t if they are not registered under the same hospital ID. To avoid this issue, index admissions, transfers and potential readmissions were identified separately in each study period (see figure 2.3.4.1), after which all index admissions (with an added target variable whether a readmission had followed or not, see paragraph 2.4) of both study periods were combined into a single dataset that was entered into the model.

A disadvantage of separating the two datasets is that theoretically the following issue can occur: since the period of November and December of year t-1 is part of both datasets, some admissions that are identified as readmissions to index admissions in study period t-1 can also be labelled as readmissions to index admissions in study period t, and thus be linked to two separate index admissions. However, it was estimated that this will only occur in a few cases and that its effect will be negligible.

2.3.5 Criteria for index admissions

Expected readmission risk was only calculated for those inpatient admissions (meeting the general criteria for admissions, see paragraph 2.3.2) for which readmission was possible (i.e. patient did not die during the index admission), excluding some specific diagnosis groups. These admissions are referred to as index admissions. Thus, in summary, index admissions had to meet the following criteria:

• The patient did not die during the admission.
• The main diagnosis of the admission was not related to oncology (CCS groups 11-45) or psychiatry (CCS groups 65-75) since hospital care for these diagnoses is usually complex and follow-up care might be required. In addition the main diagnosis was not related to obstetrics (CCS groups 176-196; 218), since most deliveries do not take place during inpatient admissions and therefore an inpatient admission for this purpose may not be a ‘true’ index admission.
• The admission was not an admission of a healthy person. Admissions of healthy persons either have ICD-10 code Z76.2-Z76.4 as main diagnosis or each bed-day of the admission is registered with a procedure code for a stay of a healthy newborn or healthy mother (Dutch procedure codes 190032, 190033 (‘Zorgactiviteiten’ codes), 339911 or 339912 (‘CBV’ codes).
• The date of discharge was from November 1^st 2021 up to October 31^st 2022 (‘year t-1’= 2022), or from November 1^st 2022 up to October 31^st 2023 (‘year t’= 2023).

2.3.6 Criteria for potential readmissions

Inpatient admissions are only qualified as potential readmissions (meeting the general criteria for admissions, see paragraph 2.3.2) if the following criteria were matched:

• The main diagnosis of the admission was not related to oncology (CCS groups 11-45) or psychiatry (CCS groups 65-75) since hospital care for these diagnoses is usually complex and follow-up care might be required. In addition the main diagnosis was not related to obstetrics (CCS groups 176-196; 218), since most deliveries do not take place during inpatient admissions and therefore an inpatient admission for this purpose may not be a ‘true’ readmission.
• The admission was not an admission of a healthy person. Admissions of healthy persons either have ICD-10 code Z76.2-Z76.4 as main diagnosis or each bed-day of the admission is registered with a procedure code for a stay of a healthy newborn or healthy mother (Dutch procedure codes 190032, 190033 (‘Zorgactiviteiten’ codes), 339911 or 339912 (‘CBV’ codes).
• The main diagnosis of the admission was not related to social, socio-economic or psychosocial circumstances or administrative purposes (ICD10: Z55-Z65), other circumstances (ICD10: Z70-Z76) or screening, follow-up care or rehabilitation (CCS groups 254-258), since admissions for these purposes are usually planned.
• The discharge date of the admission was from November 1^st 2021 (year t-2) up to December 31^st 2023 (year t).
• The maximal time lapse between the admission date of the readmission and the discharge date of the index admission is 30 days (29 days interval at maximum). For example, when an index admission has a discharge date of January 1^st, a subsequent admission on January 30^th is classified as a readmission, while a subsequent admission on January 31^st is not.
• If a readmission in the same hospital started on the same day as the discharge date of the index admission, the minimal time lapse between both admissions is one hour. If the hour of discharge of the index admission or the hour of admission of the subsequent admission is unknown in this specific situation, the subsequent admission is not identified as a readmission. When the admission date of the subsequent admission in the same hospital precedes the discharge date of the index admission (overlapping admissions), the subsequent admission is not identified as a readmission either.

Note that the main diagnosis of the readmission does not have to be related to the main diagnosis of the index admission.

2.3.7 Transfers

Transfers were not labelled as readmissions. Transfers are defined as admissions with a date of admission that was identical to the date of discharge of the previous admission in another hospital. In case of ‘overlapping admissions’ in two different hospitals (i.e. the start date of the second admission preceded the date of discharge in the first hospital) the second admission was also labelled as a transfer. Transfers affect the identification of readmissions in two ways:

First, when index admissions are followed by a transfer, these index admissions (by definition) cannot have a readmission. Although index admissions that are followed by a transfer cannot have readmissions, these index admissions are not removed from the model.

Second, transfers cannot be readmissions. In case of ‘to and fro’ transfers from hospital A to hospital B and back to hospital A, the latter admission in hospital A is not a readmission of the first admission in hospital A. In fact, an admission in hospital A that is a transfer from hospital B can (by definition) never be a readmission of any other previous admission.

The general criteria for admissions, the additional criteria for index admissions and readmissions and the role of transfers are summarised in table 2.3.7.1.

2.4 Target variable

The target variable for the regression analysis of the model is the occurrence of a readmission within 30 days of the discharge date of the preceding index admission.

The pseudonymised person ID (resulting after linkage of the LBZ to the national population register) was used as the unique key for identifying admissions of the same patient in a single hospital and for the identification of transfers to other hospitals.

The dataset was composed based on the criteria presented in section 2.3. According to the criteria for index admissions and readmissions, two variables were added to the dataset to indicate whether admissions are potential index admissions, readmissions or both. Readmissions can also count as index admissions in case they are followed by another readmission.

After that, the dataset was processed to allocate readmissions to index admissions: index admissions and potential readmissions of the same patient (person ID) are identified within the same hospital only. As was explained in section 2.3.4, this allocation is done for each year separately. Within the set of admissions per patient, for each index admission the presence of a readmission within 30 days is determined. Each index admission can only be followed by a single subsequent readmission and a single readmission can only be allocated to a single index admission. If an index admission is followed by multiple potential readmissions within 30 days, only the first occurring readmission is marked as such. Based on this algorithm, for each index admission the occurrence of a readmission is indicated.

In this example a patient is admitted five times to two different hospitals within a period of 30 days. All admissions are index admissions, and admissions B1, A2 and B2 are consecutive admissions (date of admission of A2 is equal to date of discharge of B1; and date of admission of B2 is equal to date of discharge of A2). According to the criteria for readmissions, the occurrence of a readmission is determined in step 1. The presence of transfers is determined in step 2. Finally, the information of steps 1 and 2 is combined into step 3: the occurrence of readmissions is corrected for transfers, where we apply the rules ‘an index admission followed by a transfer cannot have a readmission’ and ‘a transfer cannot be a readmission’. For example, A2 is a possible readmission to A1, but since A2 is a transfer, it cannot be a readmission. As a result, A1 is not followed by a readmission. In addition, A3 is not a readmission of A2, since A2 is followed by a transfer (B2) and thus cannot have a readmission.

Transfers are identified according to the method presented in section 2.3.7. After that, the previously described rules are applied (‘an admission followed by a transfer cannot have a readmission’ and ‘a transfer cannot be a readmission’), with the result that some of the admissions are no longer regarded as readmissions. The index admissions associated with those readmissions were initially marked as having a readmission, but since these readmissions are no longer categorized as such after applying the transfer rules, the presence of a readmission is cleared from the respective index admissions.

Finally, all index admissions and corresponding covariates are selected, plus the target variable (whether the primary admission was followed by a readmission or not) and entered into the model.

To illustrate the implementation of excluding transfers from the model, an example is given in table 2.4.1.

2.5 Stratification

Instead of performing one logistic regression for all admissions, we performed separate logistic regressions for each main diagnosis group. Such stratification may improve the precision of the estimated readmission probabilities as these sub-populations of index admissions are more homogeneous than the entire population. As a result of the stratification, covariates are allowed to have different regression coefficients across diagnosis groups. Due to the exclusions of specific CCS groups for the index admissions, 35 of the 158 diagnosis groups (as used for the HSMR) are fully excluded. Therefore, the model included 123 separate logistic regressions, one for each diagnosis group selected (see Appendix II for the diagnosis groups included).

2.6 Covariates (explanatory variables or predictors of readmission risk)

The hospital readmission risk is adjusted for patient and index admission characteristics by including these as covariates in the model. We selected the same covariates that are used in the (H)SMR model estimations, which are variables (available in the LBZ) known to be associated with in-hospital mortality. During the development of the readmission model, it was demonstrated that these covariates indeed contributed to the predictive value of the model (Van der Laan et al. 2017).

The LBZ variables that are included in the model as covariates are age, sex, socio-economic status, severity of main diagnosis (based on mortality risk categories), urgency of admission, Charlson comorbidities, source of admission, month of admission and year. These variables are described below. Detailed information on these variables and their content is available in the HSMR methodology report (Statistics Netherlands, 2024). The detailed classifications of the variables socio-economic status, severity of the main diagnosis and source of admission are presented in the file ‘Classification of variables’, published together with the methodology report of the HSMR (Statistics Netherlands, 2024).

The following changes were made to the covariates in the model compared to previous readmission and HSMR-models:

• In line with the 2023 HSMR-model, socio-economic status (SES) scores were now based on 2021 data of Statistics Netherlands.
• The variable ‘year’ is different from the variable used for the HSMR model, since it reflects the study period the index admission belongs to, rather than year of discharge. The specific (modified) definitions of ‘year’ for the readmission model are described in 2.3.4.
• All index admissions in the full two-year period were included. For all models, including the COVID-19 model, the variable ‘year’ consists of two categories (2022 and 2023). In the previous 2021 readmission model, certain admissions in periods of 2020 were excluded because of the disruption of care due to the COVID-19 pandemic (all index admissions during the first COVID-19 wave were excluded and COVID-19 index admissions were excluded for the whole year of 2020).
• The two-month-categories of the variable ‘month of admission’ were used in all models, including the COVID-19-model. In the 2021 COVID-19 model a more detailed ‘month of admission’ variable was used, with one-month-categories and a ‘before year t’ category. Such detailed monitoring of the effects of COVID-19-waves was not deemed necessary anymore. Preliminary and exploratory analyses showed that changing the model to using the two-month-admission-variable was unlikely to significantly affect the predictive validity of the COVID-19 model.

For the regressions, all categorical covariates are transformed into dummy variables (indicator variables), having scores 0 and 1. A patient scores 1 on a dummy variable if he/she belongs to the corresponding category, and 0 otherwise. As the dummy variables for a covariate are linearly dependent, one dummy variable is left out for each categorical covariate. The corresponding category is the so-called reference category. We used the first category of each covariate as the reference category.

Covariates:

Age at admission (in years):

• 0, 1-4, 5-9, 10-14, …, 90-94, 95+.

Sex of the patient:

• male, female.

SES (socioeconomic status) of the postal area of patient’s home address:

• lowest, below average, average, above average, highest, unknown.

Severity of the main diagnosis groups:

• [0-0.01), [0.01-0.02), [0.02-0.05), [0.05-0.1), [0.1-0.2), [0.2-0.3), [0.3-0.4), [0.4-1], Other.

ICD-10 subdiagnosis (for COVID-19 instead of ‘Severity of the main diagnosis’):

• U07.1, U07.2, U10.9.

Urgency of the admission

• elective, acute.

Comorbidity 1 – Comorbidity 17.

• All these 17 covariates are dummy variables, having categories: 0 (no) and 1 (yes).

Source of admission:

• home, nursing home or other institution, hospital.

Month of admission:

• Six 2-month periods: January/February, …, November/December.

Year:

Year of the study period (generally for index admissions year t is defined by a discharge date from November 1^st of year t-1 up to October 31^st of year t):

• 2022, 2023.

2.7 Estimation of the model

Logistic regression models were estimated for each of the 123 diagnosis groups using the characteristics of the index admissions as covariates as described in the previous paragraph and a dichotomous variable indicating whether an admission was followed by a readmission as the target variable. Computations were performed using the glm function in R (R Core Team, 2015). Categories, including the reference category, are collapsed if the number of index admissions is smaller than 50 or when there are no readmissions in the category. For more information on this see the aforementioned methodology report for the HSMR.

The results of the model are described in chapter 3.

3.1 Dataset

Table 3.1.1 shows the number of hospitals that were included in the model. All general and university hospitals could be included in both study periods (2022 and 2023). Two university hospitals merged in 2022 and data of a specialised hospital that started registering data in the LBZ in 2022 were included. Specialised hospitals where patients are mostly treated for oncological disease (two in 2022 and 2023) were excluded. 

The number of index admissions included in the model, the total number of identified readmissions and the unadjusted readmission rate for both study periods are listed in Table 3.1.2. The number of admissions per year is comparable to the number of admissions in 2021 (1,039,097), but higher than in 2020 (742,733) (Van der Laan et al. 2023). The number of admissions did not return to the number before the COVID-19 pandemic (1,223,296 in 2018; Van der Laan et al. 2020). The unadjusted readmission rates, however, are quite stable: 9.0% in 2017, 8.9% in 2018, 8.8% in 2020, 8.6% in 2021, 8.7% in 2022 and 8.8% in 2022.

3.2 Impact of the covariates on readmission rate

Appendix I shows which covariates have a statistically significant (95 percent confidence) impact on the readmission rate for each of the 123 regression models (one for each diagnosis group, including the diagnosis group COVID-19). Tables 3.2.1 and 3.2.2 show the total number of significant covariates and the total Wald statistics for the 123 regression models. The tables are sorted in descending order (most important variables at the top). The first table shows the number of diagnosis groups in which a variable is significant in the model. The effect of variables on the predicted probabilities, and, therefore, the importance of the variables for the case mix correction performed by the models, is better measured with the Wald-statistics (shown in the second table).

The order of the variables differs somewhat in both tables, but in both tables age, Charlson comorbidity 13 (Renal disease), urgency and severity are in the top 5 of the most important variables for model estimation. Except for Charlson comorbity 13, this is also the case for the HSMR 2023 model (Statistics Netherlands, 2024), indicating that these variables are relevant for both predicting readmissions and in-hospital mortality.

For the 2023 readmission model, sex and Charlson comorbidity 10 (Diabetes) are more important than for the 2023 HSMR model. Vice versa, the Charlson comorbidities 16 (metastatic cancer) and 2 (congestive heart failure) are more important for the 2023 HSMR model than for the 2023 readmission model. Apparently congestive heart failure has a stronger influence on estimating in-hospital mortality, while diabetes has a stronger influence on estimating readmissions. The difference in importance of Charlson group 16 (metastatic cancer) in both models can be explained by the fact that cancer-related main diagnoses are excluded from the readmission model, since planned readmissions for those diagnoses are frequent.

Compared to the 2021 readmission model (Van der Laan et al. 2023), the current model was based on more data. As a result, the number of significant covariates and Wald statistics was slightly higher in the current model than in the 2021 model. However, as the number of admissions did not return to pre-2020 values, the number of significant covariates and Wald statistics is lower than in the 2018 model. The order of the variables remained largely comparable.

3.3 Model evaluation for the 123 regression analyses

Appendix II shows the Areas Under the Curve (AUCs; also known as C-statistics) for each of the 123 regression models. From these AUCs it can be concluded that most models have weak predictive power. This is comparable with earlier models. Of the 123 diagnosis groups, only 20 have an AUC of 0.70 or above. This is one more than for the 2021 model. The diagnosis groups with a AUC of 0.70 or above are largely (14/20) the same as in 2021. The average AUC is 0.649, which is nearly the same as for 2021 (0.651). However, compared to 2021 there are a number of models for which the AUC has changed substantially: both in positive and negative direction. There are 22 models for which the AUC has changed more than 0.02 and 9 models for which the AUC has changed more than 0.04. See table 3.3.1 for an overview.

It is not always clear why the predictive power of the models has changed. The COVID-19 pandemic has had a significant impact on admissions in hospitals. For example, there were substantially less influenza admissions in the 2021 model compared to 2018. However, the number in the 2023 model has returned to the level in the 2018 model. The AUC of influenza is also back to the value it had in 2018. However, in general there was no clear relationship between (changes in) the number of admissions and the AUC of a diagnosis group.

3.4 Regression coefficients

Appendix III contains the estimated regression coefficients (columns ‘Estimate’) for each of the 123 logistic regressions as well as their standard errors (columns ‘Std. Err.’). For the sake of clarity, the reference categories are given in the first row of the corresponding covariates, and by definition have the coefficient zero for each regression. In many cases categories are collapsed, which results in equal coefficients for the collapsed categories. If all categories were collapsed into one category for a certain variable and for a certain diagnosis group (i.e. if there was only one category with ≥50 admissions and ≥1 readmission), the variable was dropped from the model and all associated coefficients were set to zero. The significance of each of the coefficients is shown in Appendix I.

3.5 Limitations

The readmission indicator has largely the same limitations as the HSMR. Some issues that are specific to the readmission indicator are addressed below.

• In principle all readmissions are included in the model: planned and unplanned; related and not related to the index admission. Ideally, only unplanned readmissions should be included, but those are not registered as such in the LBZ. Although the LBZ contains the variable urgency (acute versus not acute; an admission is registered ‘acute’ if care is needed within 24 hours), this variable does not accurately reflect the difference between planned and unplanned readmissions. To avoid the inclusion of planned readmissions, some diagnosis groups where planned readmissions are likely (for example the various groups concerning cancer) are excluded as index and readmissions. Diagnoses that are likely planned readmissions (for example follow-up care and rehabilitation) are also excluded as potential readmissions. Furthermore, in the present model (planned) transfers are excluded as readmissions. However, some planned readmissions will likely remain in the dataset.
• Unlike with the HSMR, Statistics Netherlands does not provide readmission ratios for 2023, based on the model of 2023. DHD will use the estimated models to calculate the ratios using hospital data from 2024. This means that the models are applied to a different year than that on which they were estimated. As was shown for the readmission model 2015 (Van der Laan et al. 2017), this results in a bias and extra variance.
• It is difficult to predict readmissions using the variables present in the models: the models explain only a small part of the observed variation. This makes it more likely that there are unobserved population differences that are not corrected for, that influence the readmission probability. This means that some of the differences in the current readmission ratio can be caused by unobserved population differences.
• The model identifies intra-hospital readmissions only and readmissions that occur in another hospital are not identified. As a result, the readmission ratio for hospitals where patients are often readmitted in another hospital could be an underestimation, and vice versa.

In general the quality of the 2023 models is similar to that of previous versions of the hospital readmission models. However, there were some shifts in quality and estimated parameters for the models of some of the diagnosis groups. In part, this was likely caused by the disruption of hospital care by the COVID-19 pandemic, which may have affected the 2021 readmission model. The number of admissions increased in the 2023 model, compared to the 2021 model, but did not return to the values in the 2018 model.

Like in the previous models, ‘to and fro’ transfers are excluded as readmissions. This removes some of the noise from the model, as these planned transfers are not of interest when the readmission ratio is used as an indicator of quality of care. Additionally, several diagnosis groups consisting of diseases that require treatment during multiple, consecutive admissions have been excluded from the model. However, it is possible that the data still contains planned readmissions, resulting in a less reliable prediction. Although the predictive power of the model is generally low, the case mix correction performed by the model does remove some of the differences between the hospitals caused by population differences. However, because of the weak predictive power of the models, it is likely that there are still population differences remaining for which the model does not correct. Nevertheless, applying the model for calculating readmission ratios for individual hospitals is preferable to calculating crude rates.

Hekkert K., R.B. Kool, E. Rake, S. Cihangir, I Borghans, F. Atsma and G.P. Westert (2018). To what degree can variations in readmission rates be explained on the level of the hospital? a multilevel study using a large Dutch database. BMC Health Services Research 18(999). DOI: 10.1186/s12913-018-3761-y.

Hekkert K., I. Borghans, S. Cihangir, G.P. Westert and R.B. Kool (2019). What is the impact on the readmission ratio of taking into account readmissions to other hospitals? A cross-sectional study. BMJ Open;9:e025740. DOI:10.1136/bmjopen-2018-025740.

HSCIC (2016). Indicator Specification: Summary Hospital-level Mortality Indicator. Version 1.22, 24 February 2016. Health & Social Care Information Centre.

Kansagara D., H. Englander, A. Salanitro, D. Kagen, C. Theobald, M. Freeman and S. Kripalani (2011). Risk prediction models for hospital readmission: a systematic review. The Journal of the American Medical Association 306(15), pp. 1688-98. DOI: 10.1001/jama.2011.1515.

R Core Team (2015). R: A language and environment for statistical computing, R Foundation for Statistical Computing, Vienna, Austria.

Van der Laan J., C. Penning and A. de Bruin (2017). Hospital Readmission Ratio. Methodological report of 2015 model. Statistics Netherlands, The Hague.

Van der Laan J., C. Penning and A. de Bruin (2018). Hospital Readmission Ratio. Methodological report of 2016 models. Statistics Netherlands, The Hague.

Van der Laan, J., C. Penning and A. de Bruin (2020). Hospital Readmission Ratio. Methodological report of the 2018 model. Statistics Netherlands, The Hague.

Van der Laan, J., C. Penning and A. de Bruin (2023). Hospital Readmission Ratio. Methodological report of the 2021 model. Statistics Netherlands, The Hague.

Statistics Netherlands (2024). HSMR 2021: Methodological report, Statistics Netherlands, The Hague.

The Excel file ‘Appendix I: Results of the logistic regressions’ presents the statistical significance (95% confidence) of the covariates for the 123 logistic regressions.

The Excel file ‘Appendix II: AUC’ presents the AUC or C-statistic for the logistic regressions of the 123 main diagnosis groups.

The Excel file 'Appendix III: Coefficients'  contains the estimated regression coefficients and their standard errors for each of the 123 logistic regressions."