Logistic versus hierarchical modeling: an analysis of a statewide inpatient sample

Background
Although logistic regression is traditionally used to calculate hospital standardized mortality ratio (HSMR), it ignores the hierarchical structure of the data that can exist within a given database. Hierarchical models allow examination of the effect of data clustering on outcomes.

Study Design
Traditional logistic regression and random intercepts fixed slopes hierarchical models were fitted to a dataset of patients hospitalized between 2005 and 2007 in Massachusetts. We compared the observed to expected (O/E) in-hospital death ratios between the 2 modeling techniques, a restricted HSMR using only those diagnosis models that converged in both methods and a full hybrid HSMR using a combination of the hierarchical diagnosis models when they converge, plus the remaining diagnoses using standard logistic regression models.

Results
We restricted the analysis to the 36 diagnoses accounting for 80% of in-hospital deaths nationally, based on 1,043,813 admissions (59 hospitals). A failure of the hierarchical models to converge in 15 of 36 diagnosis groups hindered full HSMR comparisons. A restricted HSMR, derived from a dataset based on the 21 diagnosis groups that converged (552,933 admissions) showed very high correlation (Pearson r = 0.99). Both traditional logistic regression and hierarchical model identified 12 statistical outliers in common, 7 with high O/E values and 5 with low O/E values. In addition, the multilevel analysis identified 5 additional unique high outliers and 1 additional unique low outlier, and the conventional model identified 2 additional unique low outliers.

Conclusions
Similar results were obtained from the 2 modeling techniques in terms of O/E ratios. However, because a hierarchical model is associated with convergence problems, traditional logistic regression remains our recommended procedure for computing HSMRs.