2. Methods

2.1 Changes compared to the previous intra-hospital model

In the present ‘2023 model’ we largely used the same methods as in the previous intra-hospital model (‘2021 model’), which excludes transfers as readmissions (Van der Laan et al. 2023).

• All admissions that met general criteria for index admissions were included for the full two-year period (in the previous model index admissions in the first wave of the COVID-19 pandemic in 2020 were excluded as hospital care was disrupted during this period, and all COVID-191 index admissions in 2020 were excluded).
• The covariate ‘month of admission’ of the COVID-19 model was the same as for the models of other diagnosis groups, consisting of two-month categories. It was no longer necessary to use the more detailed ‘month of admission’ variable that was used in the previous COVID-19 model.
• In line with the 2023 HSMR-model, socio-economic status (SES) scores were based on 2021 data of Statistics Netherlands.

2.2 Readmission ratio

The (hospital) readmission ratio is calculated using the observed readmission as the numerator and the expected (hospital) readmission risk as the denominator. The expected readmission risk is predicted for each individual admission within a given period, adjusted for patient and admission characteristics of that admission as covariates. Readmission risk was predicted for all (index) admissions that could potentially be followed by a readmission, excluding admissions for diagnoses with complex care paths where planned readmissions are often involved. Readmissions are defined as those admissions that occurred within 30 days of the discharge date of the preceding index admission. More information on the criteria of index admissions and readmissions is given in paragraphs 2.3.5 and 2.3.6 respectively.

The expected readmission risk is determined separately for each of the included diagnosis groups, based on the CCS (Clinical Classifications Software). The CCS clusters ICD codes of the main diagnoses of the admissions into 259 clinically meaningful categories (see https://www.cbs.nl/-/media/_excel/2024/43/classification-of-variables-hsmr-2023.xlsx), which were clustered further into 158 diagnosis groups. This clustering is in accordance with the HSMR and similar to the SHMI (Summary Hospital-level Mortality Indicator) in the UK (HSCIC, 2016). To determine the readmission risk we used logistic regression models, with an observed readmission as the target (dependent) variable and various patient and admission characteristics available in the LBZ as covariates.

The methodology for estimating the expected readmission risk is very similar to that used for estimating expected mortality rates applied for calculating the HSMR rates, described in detail elsewhere (Statistics Netherlands, 2024). In the following section, we therefore briefly describe the applied methods, while deviations from the HSMR methodology or other methods specific to the current project are described in more detail.

2.3 Target population and data set

2.3.1 Patient identifier

Statistics Netherlands linked the LBZ data to the Dutch national population register using a pseudonym of the national personal identification number, and the combination of date of birth, sex and postal code as linkage keys. With this linkage >99% of all admissions could be uniquely linked to a person in the population register; thus the loss of data was minimal (<1%). Through linkage, a unique pseudonymised person ID could be added to the LBZ dataset. Using this identifier not only allows identification of transfers to other hospitals, it also eliminates bias due to administrative errors in hospital-specific patient numbers.

2.3.2 Admissions – general criteria

We consider both the population of hospitals and the population of admissions. Our population of (re)admissions consists of “all hospital stays (inpatient admissions) of Dutch residents in Dutch short-stay hospitals within the study period”. In the LBZ, the date of discharge, and not the day of admission, determines the LBZ year a record is assigned to. The following admissions were excluded:

• day cases and prolonged observations. Subsequent readmissions might be elective in these cases, e.g. for prolonged treatment.
• incomplete admissions without a registered main diagnosis. These are rare, however, as hospitals have to register inpatient admissions completely.
• admissions that do not meet the billing criteria of the Dutch Healthcare Authority. This primarily, but not exclusively, concerns one-day inpatient admissions where the patient returned home after discharge. Based on an algorithm using LBZ data DHD has added a variable to the LBZ dataset (from 2019 onwards) that indicates whether the admission meets the billing criteria or not. This variable was used to exclude the admissions not meeting the billing criteria.
• admissions of foreigners. Readmissions of foreigners might take place in a hospital in their residential country, foreigners cannot be linked to the Dutch population register and the number of admissions of foreigners is relatively small.
• admissions of healthy persons. These are for example admissions of healthy newborns, a healthy parent accompanying a sick child, or other healthy boarders. These are identified based on the main diagnosis of the admission (ICD-10 code Z76.2-Z76.4) or based on procedure codes. Admissions were excluded based on procedure codes if for each bed-day of the admission a procedure code for a stay of a healthy person has been registered (Dutch procedure codes 190032, 190033 (‘Zorgactiviteiten’ codes), 339911 or 339912 (‘CBV’ codes)).
• duplicate admissions with identical values for date and time of admission and of discharge in combination with identical values for either (1) hospital ID and hospital-specific patient ID or (2) the pseudonymised person ID. In case of duplicate admissions, the admission with the lowest LBZ registration number was removed and the one with the highest number was kept, since we assumed that the latter admission might have been registered as a corrected version of the first. Duplicate admissions rarely occur in the LBZ.

2.3.3 Hospitals

Hospitals report admission data (hospital stay data) in the LBZ. However, not all hospitals participate in the LBZ. In principle, the hospital readmission risk model includes all general hospitals, all university hospitals and short-stay specialised hospitals with inpatient admissions participating in the LBZ in the study period. However, two of the three short-stay specialised hospitals participating in the LBZ were excluded from the hospital readmission model as these two hospitals treat patients with oncological diseases, for which admissions are excluded from the data (see paragraph 2.3.5).

The readmission ratio is calculated using LBZ data on admissions, using the pseudonymised personal ID as the unique key for identifying (re)admissions. The combination of the person ID (for identifying patients) and the hospital ID number (for identifying the same hospital) was used for linking admissions. In case of merging hospitals, the hospital ID number that the hospital used in the LBZ registration year, was used for the associated study period in the models. For example, two hospitals that had merged in study period twere analysed as separate units for study period t-1 and as a single unit for study period t. Otherwise, if the merged hospital (C) ID was also used for study period t-1, the year in which the unmerged hospitals (A and B) were still operating separately, an admission in hospital A followed by an admission in hospital B, could then potentially be identified as an index admission - readmission combination in hospital C. This would result in the identification of readmissions that in reality were admissions in another hospital.

2.3.4 Study periods

For the calculation of the current model, LBZ data of 2022 and 2023 was used. Previously we have shown that to identify the highest percentage of readmissions ending in year t, using index admissions with a discharge date from November 1^st of year t-1 up to October 31^st of year t (study period) is optimal (Van der Laan et al. 2017). Thus, for study period 2022 (‘year’=2022 in the model) we selected index admissions with a discharge date from November 1^st 2021 up to October 31^st 2022 and for study period 2023 we selected index admissions with a discharge date from November 1^st 2022 up to October 31^st 2023 (‘year’= 2023 in the model), see figure 2.3.4.1. The occurrence of readmissions was analysed in the period between November 1^st 2021 up to December 31^st 2023. The study periods are displayed in figure 2.3.4.1. If hospitals had merged in study period t, the hospital ID of the merged hospital was also used for the data of November and December of year t-1.

The study period of the 2023 model is similar to the 2018 model but differs from the study period applied for the 2021 model. In the 2021 model, index admissions from February 1^st up to May 31^st 2020 were excluded because of the first wave of the COVID-19 pandemic in the Netherlands which severely disrupted hospital care.

A study period t-1 using hospital IDs from year t-1 for all admissions   
B study period t using hospital IDs from year t  for all admissions

Figure 2.3.4.1 shows which LBZ data is included in both study periods. To process index admissions, transfers and readmissions, two separate datasets (A and B in figure 2.3.4.1) for each of one of the study periods were constructed, instead of a single dataset containing data from both study periods. This is necessary to link admissions in a specific hospital to the correct hospital ID in case of a merger of hospitals. For example, if hospitals X and Y in year t-1 merge into hospital Z in year t, admissions in these hospitals are registered under hospital IDs X and Y in year t-1, and under hospital IDs Z in year t. This is a problem, as November and December of year t-1 are not only part of study period t-1, but also of study period t. Admissions registered under hospital IDs X and Y in year t-1 cannot be linked to hospital Z in year t if they are not registered under the same hospital ID. To avoid this issue, index admissions, transfers and potential readmissions were identified separately in each study period (see figure 2.3.4.1), after which all index admissions (with an added target variable whether a readmission had followed or not, see paragraph 2.4) of both study periods were combined into a single dataset that was entered into the model.

A disadvantage of separating the two datasets is that theoretically the following issue can occur: since the period of November and December of year t-1 is part of both datasets, some admissions that are identified as readmissions to index admissions in study period t-1 can also be labelled as readmissions to index admissions in study period t, and thus be linked to two separate index admissions. However, it was estimated that this will only occur in a few cases and that its effect will be negligible.

2.3.5 Criteria for index admissions

Expected readmission risk was only calculated for those inpatient admissions (meeting the general criteria for admissions, see paragraph 2.3.2) for which readmission was possible (i.e. patient did not die during the index admission), excluding some specific diagnosis groups. These admissions are referred to as index admissions. Thus, in summary, index admissions had to meet the following criteria:

• The patient did not die during the admission.
• The main diagnosis of the admission was not related to oncology (CCS groups 11-45) or psychiatry (CCS groups 65-75) since hospital care for these diagnoses is usually complex and follow-up care might be required. In addition the main diagnosis was not related to obstetrics (CCS groups 176-196; 218), since most deliveries do not take place during inpatient admissions and therefore an inpatient admission for this purpose may not be a ‘true’ index admission.
• The admission was not an admission of a healthy person. Admissions of healthy persons either have ICD-10 code Z76.2-Z76.4 as main diagnosis or each bed-day of the admission is registered with a procedure code for a stay of a healthy newborn or healthy mother (Dutch procedure codes 190032, 190033 (‘Zorgactiviteiten’ codes), 339911 or 339912 (‘CBV’ codes).
• The date of discharge was from November 1^st 2021 up to October 31^st 2022 (‘year t-1’= 2022), or from November 1^st 2022 up to October 31^st 2023 (‘year t’= 2023).

2.3.6 Criteria for potential readmissions

Inpatient admissions are only qualified as potential readmissions (meeting the general criteria for admissions, see paragraph 2.3.2) if the following criteria were matched:

• The main diagnosis of the admission was not related to oncology (CCS groups 11-45) or psychiatry (CCS groups 65-75) since hospital care for these diagnoses is usually complex and follow-up care might be required. In addition the main diagnosis was not related to obstetrics (CCS groups 176-196; 218), since most deliveries do not take place during inpatient admissions and therefore an inpatient admission for this purpose may not be a ‘true’ readmission.
• The admission was not an admission of a healthy person. Admissions of healthy persons either have ICD-10 code Z76.2-Z76.4 as main diagnosis or each bed-day of the admission is registered with a procedure code for a stay of a healthy newborn or healthy mother (Dutch procedure codes 190032, 190033 (‘Zorgactiviteiten’ codes), 339911 or 339912 (‘CBV’ codes).
• The main diagnosis of the admission was not related to social, socio-economic or psychosocial circumstances or administrative purposes (ICD10: Z55-Z65), other circumstances (ICD10: Z70-Z76) or screening, follow-up care or rehabilitation (CCS groups 254-258), since admissions for these purposes are usually planned.
• The discharge date of the admission was from November 1^st 2021 (year t-2) up to December 31^st 2023 (year t).
• The maximal time lapse between the admission date of the readmission and the discharge date of the index admission is 30 days (29 days interval at maximum). For example, when an index admission has a discharge date of January 1^st, a subsequent admission on January 30^th is classified as a readmission, while a subsequent admission on January 31^st is not.
• If a readmission in the same hospital started on the same day as the discharge date of the index admission, the minimal time lapse between both admissions is one hour. If the hour of discharge of the index admission or the hour of admission of the subsequent admission is unknown in this specific situation, the subsequent admission is not identified as a readmission. When the admission date of the subsequent admission in the same hospital precedes the discharge date of the index admission (overlapping admissions), the subsequent admission is not identified as a readmission either.

Note that the main diagnosis of the readmission does not have to be related to the main diagnosis of the index admission.

2.3.7 Transfers

Transfers were not labelled as readmissions. Transfers are defined as admissions with a date of admission that was identical to the date of discharge of the previous admission in another hospital. In case of ‘overlapping admissions’ in two different hospitals (i.e. the start date of the second admission preceded the date of discharge in the first hospital) the second admission was also labelled as a transfer. Transfers affect the identification of readmissions in two ways:

First, when index admissions are followed by a transfer, these index admissions (by definition) cannot have a readmission. Although index admissions that are followed by a transfer cannot have readmissions, these index admissions are not removed from the model.

Second, transfers cannot be readmissions. In case of ‘to and fro’ transfers from hospital A to hospital B and back to hospital A, the latter admission in hospital A is not a readmission of the first admission in hospital A. In fact, an admission in hospital A that is a transfer from hospital B can (by definition) never be a readmission of any other previous admission.

The general criteria for admissions, the additional criteria for index admissions and readmissions and the role of transfers are summarised in table 2.3.7.1.

2.4 Target variable

The target variable for the regression analysis of the model is the occurrence of a readmission within 30 days of the discharge date of the preceding index admission.

The pseudonymised person ID (resulting after linkage of the LBZ to the national population register) was used as the unique key for identifying admissions of the same patient in a single hospital and for the identification of transfers to other hospitals.

The dataset was composed based on the criteria presented in section 2.3. According to the criteria for index admissions and readmissions, two variables were added to the dataset to indicate whether admissions are potential index admissions, readmissions or both. Readmissions can also count as index admissions in case they are followed by another readmission.

After that, the dataset was processed to allocate readmissions to index admissions: index admissions and potential readmissions of the same patient (person ID) are identified within the same hospital only. As was explained in section 2.3.4, this allocation is done for each year separately. Within the set of admissions per patient, for each index admission the presence of a readmission within 30 days is determined. Each index admission can only be followed by a single subsequent readmission and a single readmission can only be allocated to a single index admission. If an index admission is followed by multiple potential readmissions within 30 days, only the first occurring readmission is marked as such. Based on this algorithm, for each index admission the occurrence of a readmission is indicated.

In this example a patient is admitted five times to two different hospitals within a period of 30 days. All admissions are index admissions, and admissions B1, A2 and B2 are consecutive admissions (date of admission of A2 is equal to date of discharge of B1; and date of admission of B2 is equal to date of discharge of A2). According to the criteria for readmissions, the occurrence of a readmission is determined in step 1. The presence of transfers is determined in step 2. Finally, the information of steps 1 and 2 is combined into step 3: the occurrence of readmissions is corrected for transfers, where we apply the rules ‘an index admission followed by a transfer cannot have a readmission’ and ‘a transfer cannot be a readmission’. For example, A2 is a possible readmission to A1, but since A2 is a transfer, it cannot be a readmission. As a result, A1 is not followed by a readmission. In addition, A3 is not a readmission of A2, since A2 is followed by a transfer (B2) and thus cannot have a readmission.

Transfers are identified according to the method presented in section 2.3.7. After that, the previously described rules are applied (‘an admission followed by a transfer cannot have a readmission’ and ‘a transfer cannot be a readmission’), with the result that some of the admissions are no longer regarded as readmissions. The index admissions associated with those readmissions were initially marked as having a readmission, but since these readmissions are no longer categorized as such after applying the transfer rules, the presence of a readmission is cleared from the respective index admissions.

Finally, all index admissions and corresponding covariates are selected, plus the target variable (whether the primary admission was followed by a readmission or not) and entered into the model.

To illustrate the implementation of excluding transfers from the model, an example is given in table 2.4.1.

2.5 Stratification

Instead of performing one logistic regression for all admissions, we performed separate logistic regressions for each main diagnosis group. Such stratification may improve the precision of the estimated readmission probabilities as these sub-populations of index admissions are more homogeneous than the entire population. As a result of the stratification, covariates are allowed to have different regression coefficients across diagnosis groups. Due to the exclusions of specific CCS groups for the index admissions, 35 of the 158 diagnosis groups (as used for the HSMR) are fully excluded. Therefore, the model included 123 separate logistic regressions, one for each diagnosis group selected (see Appendix II for the diagnosis groups included).

2.6 Covariates (explanatory variables or predictors of readmission risk)

The hospital readmission risk is adjusted for patient and index admission characteristics by including these as covariates in the model. We selected the same covariates that are used in the (H)SMR model estimations, which are variables (available in the LBZ) known to be associated with in-hospital mortality. During the development of the readmission model, it was demonstrated that these covariates indeed contributed to the predictive value of the model (Van der Laan et al. 2017).

The LBZ variables that are included in the model as covariates are age, sex, socio-economic status, severity of main diagnosis (based on mortality risk categories), urgency of admission, Charlson comorbidities, source of admission, month of admission and year. These variables are described below. Detailed information on these variables and their content is available in the HSMR methodology report (Statistics Netherlands, 2024). The detailed classifications of the variables socio-economic status, severity of the main diagnosis and source of admission are presented in the file ‘Classification of variables’, published together with the methodology report of the HSMR (Statistics Netherlands, 2024).

The following changes were made to the covariates in the model compared to previous readmission and HSMR-models:

• In line with the 2023 HSMR-model, socio-economic status (SES) scores were now based on 2021 data of Statistics Netherlands.
• The variable ‘year’ is different from the variable used for the HSMR model, since it reflects the study period the index admission belongs to, rather than year of discharge. The specific (modified) definitions of ‘year’ for the readmission model are described in 2.3.4.
• All index admissions in the full two-year period were included. For all models, including the COVID-19 model, the variable ‘year’ consists of two categories (2022 and 2023). In the previous 2021 readmission model, certain admissions in periods of 2020 were excluded because of the disruption of care due to the COVID-19 pandemic (all index admissions during the first COVID-19 wave were excluded and COVID-19 index admissions were excluded for the whole year of 2020).
• The two-month-categories of the variable ‘month of admission’ were used in all models, including the COVID-19-model. In the 2021 COVID-19 model a more detailed ‘month of admission’ variable was used, with one-month-categories and a ‘before year t’ category. Such detailed monitoring of the effects of COVID-19-waves was not deemed necessary anymore. Preliminary and exploratory analyses showed that changing the model to using the two-month-admission-variable was unlikely to significantly affect the predictive validity of the COVID-19 model.

For the regressions, all categorical covariates are transformed into dummy variables (indicator variables), having scores 0 and 1. A patient scores 1 on a dummy variable if he/she belongs to the corresponding category, and 0 otherwise. As the dummy variables for a covariate are linearly dependent, one dummy variable is left out for each categorical covariate. The corresponding category is the so-called reference category. We used the first category of each covariate as the reference category.

Covariates:

Age at admission (in years):

• 0, 1-4, 5-9, 10-14, …, 90-94, 95+.

Sex of the patient:

• male, female.

SES (socioeconomic status) of the postal area of patient’s home address:

• lowest, below average, average, above average, highest, unknown.

Severity of the main diagnosis groups:

• [0-0.01), [0.01-0.02), [0.02-0.05), [0.05-0.1), [0.1-0.2), [0.2-0.3), [0.3-0.4), [0.4-1], Other.

ICD-10 subdiagnosis (for COVID-19 instead of ‘Severity of the main diagnosis’):

• U07.1, U07.2, U10.9.

Urgency of the admission

• elective, acute.

Comorbidity 1 – Comorbidity 17.

• All these 17 covariates are dummy variables, having categories: 0 (no) and 1 (yes).

Source of admission:

• home, nursing home or other institution, hospital.

Month of admission:

• Six 2-month periods: January/February, …, November/December.

Year:

Year of the study period (generally for index admissions year t is defined by a discharge date from November 1^st of year t-1 up to October 31^st of year t):

• 2022, 2023.

2.7 Estimation of the model

Logistic regression models were estimated for each of the 123 diagnosis groups using the characteristics of the index admissions as covariates as described in the previous paragraph and a dichotomous variable indicating whether an admission was followed by a readmission as the target variable. Computations were performed using the glm function in R (R Core Team, 2015). Categories, including the reference category, are collapsed if the number of index admissions is smaller than 50 or when there are no readmissions in the category. For more information on this see the aforementioned methodology report for the HSMR.

The results of the model are described in chapter 3.