Objective
The purpose of this study was to determine whether hospital differences in the frequency of adverse obstetric outcomes are related to differences in care.
Study Design
The Assessment of Perinatal EXcellence cohort comprises 115,502 women and their neonates who were born in 25 hospitals in the United States between March 2008 and February 2011. Hierarchical logistic regression was used to quantify the amount of variation in postpartum hemorrhage, peripartum infection, severe perineal laceration, and a composite adverse neonatal outcome among hospitals that is explained by differences in patient characteristics, hospital characteristics, and obstetric care provided.
Results
The study included 115,502 women. For most outcomes, 20-40% of hospital differences in outcomes were related to differences in patient populations. After adjusting for patient-, provider-, and hospital-level factors, multiple care processes were associated with the predefined adverse outcomes; however, these care processes did not explain significant variation in the frequency of adverse outcomes among hospitals. Ultimately, 50-100% of the interhospital variation in outcomes was unexplained.
Conclusion
Hospital differences in the frequency of adverse obstetric outcomes could not be explained by differences in frequency of types of care provided.
See related editorial, page 85
Obstetric admissions are a leading cause of hospitalization in the United States. Accordingly, there has been an increasing demand for quality measurement from multiple stakeholders. Quality measures typically take 2 forms: (1) outcome measures, such as frequency of peripartum infection, which reflect the actual outcomes, and (2) process measures, such as frequency of episiotomy, which reflect adherence to or avoidance of a given type of care.
However, several uncertainties remain about obstetric outcome and process measures and their ability to represent quality care. There is controversy whether and to what extent hospital differences in outcomes are actually due to differences in the characteristics of their patient population; correspondingly, case-mix adjustment has been used inconsistently. Also, there is often an implicit assumption that those hospitals that perform best on process measures will have the best outcomes. Yet, this assumption has not been proved in obstetrics.
In fact, there are several potential contributors to the frequency of adverse outcomes that include patient characteristics (such as maternal age), hospital characteristics (such as the types of obstetric providers or continual availability of interventional radiology), and the types of care that are provided (such as the frequency of cesarean delivery). Although poorly understood, the extent to which each of these categories explains hospital differences in outcomes is important in determining the adequacy of quality measures. For example, if all variation in an outcome were due to differences in patient populations, it would make little sense to use that outcome to represent a hospital’s quality. On the other hand, if much of the variation in an outcome were not due to differences in patient populations but were due to differences in a particular process of care, the use of both specific outcome and process measures would be better supported.
The specific aim of the present study was to assess whether and to what extent hospital differences in the frequency of adverse obstetric outcomes are related to patient and hospital characteristics and to types of care provided.
Methods
Study design
The Assessment of Perinatal EXcellence (APEX) study is an observational study that was designed to assist in the development of quality measures for intrapartum obstetrics care. This study was approved by the institutional review board at each participating institution under a waiver of informed consent. Full details of the study design have been published previously.
In summary, patients who were eligible for data collection were those who delivered on randomly selected days between March 2008 and February 2011 at any of the 25 hospitals in the Eunice Kennedy Shriver National Institute of Child Health and Human Development Maternal-Fetal Medicine Units Network, who were at least 23 weeks of gestation, and who had arrived at the hospital with a live fetus. Days were chosen by computer-generated random selection, with enrollment from larger hospitals limited to avoid overrepresentation of patients from these hospitals. The medical records of all eligible women and their neonates were abstracted by trained and certified research personnel at the clinical centers. Patient data that were reported in the chart included demographic characteristics (including race and ethnicity as reported in the chart), details of the medical and obstetric history, types of intrapartum and postpartum care, and obstetric outcomes. In addition, characteristics of the providers who cared for the patients and the hospitals in which they delivered were collected. Maternal data were collected until discharge, and neonatal data were collected until discharge or until 120 days of age, whichever came first.
Outcomes
The 5 a priori primary outcomes were (1) venous thromboembolism, (2) postpartum hemorrhage (PPH), (3) peripartum infection, (4) severe perineal laceration, restricted to women with vaginal singleton deliveries with no shoulder dystocia and stratified by spontaneous, forceps-assisted vaginal delivery, and vacuum-assisted vaginal delivery, and (5) a composite neonatal adverse outcome, restricted to term (≥37 weeks of gestation), nonanomalous singleton infants. Additional details regarding the definitions of these outcomes are detailed elsewhere.
Statistical analysis
Sample size for the APEX cohort was based on thromboembolism in cesarean deliveries, which was expected to have the lowest frequency (0.175% overall and 0.550% in cesarean deliveries) of the 5 a priori primary outcomes, with techniques that consider the cluster design. The following assumptions were included: 2-sided type I error = 0.01 and the proportion of deliveries without an associated process measure = 25%. The sample size estimate was based on 30,000 cesarean deliveries. Conservatively, assuming a cesarean frequency of 25%, a total sample size of 120,000 would enable the detection of an odds ratio of 2.75 for the association between a process measure and outcome with at least 80% power for the outcome of thromboembolism. Assuming an odds ratio of 1.5 and assuming event frequencies that ranged from 2.4–8.0% for the remaining 4 outcomes (PPH, peripartum infection, severe perineal laceration in vaginal deliveries, and the composite neonatal adverse outcome in term nonanomalous singletons), power was estimated to range from 83–99%; power was >99% for these 4 outcomes when we assumed an odds ratio of 2.0. Because of fewer than expected thromboembolism events (0.03% overall), this outcome was not further evaluated.
For each of the adverse obstetric outcomes, hierarchical logistic regression with hospital random effects was used to quantify the amount of variation in outcomes among hospitals that was due to (1) patient characteristics, (2) provider and hospital characteristics, and (3) the types of care provided (process measures). The initial regression equation included only the hospitals as random-effect terms. In each successive stage of the model, another level of variables (ie, the patient characteristics, hospital characteristics, or care characteristics) was added as fixed effects. Per the methods used by Synnes et al, each equation contained a random effects term, and it is the standard deviation of this term that serves to quantify the overall variation in outcome frequency across the hospitals. The difference in the value of standard deviation as each set of characteristics is added to the model then quantifies the amount of variation between hospitals explained by the additional characteristics. Odds ratios and 99% confidence intervals (CIs) for each hospital, with the use of the hospital with the median observed outcome frequency as the referent, were also obtained from these hierarchical models.
Patient, provider, hospital, and care characteristics that were eligible for multivariable models were selected a priori for each outcome, based on a plausible association with the outcome (ie, face validity). Details regarding the methods and results for selection of the patient characteristics have been reported previously. The provider and hospital characteristics that were eligible for multivariable models included the specialty of the attending provider, years since the attending provider graduated from medical/midwifery school, nurse-to-patient ratio during the shift that delivery occurred, a hospital’s annual delivery volume (expressed in quartiles), the existence of a prenatal electronic medical record, the occurrence of a structured review of laboring patients attended by both nursing staff and attending providers, and the availability of a 24-hour anesthesia service dedicated to the labor and delivery unit. The presence of a 24-hour in-house attending obstetric provider, a 24-hour in-house neonatologist or pediatrician, and a 24-hour in-house interventional radiology service also were evaluated. For each outcome, after the patient characteristics that were selected previously for risk-adjustment were forced into the model, a backwards selection method was used with a probability value of < .05 to determine which provider and hospital characteristics were to remain in the regression for each outcome.
After a model that included patient, provider, and hospital characteristics was established, we examined which types of care (ie, process measures) provided, selected a priori, were associated with each outcome. Eligible process measures included elective delivery <39 weeks of gestation without documented lung maturity, cervical dilation at admission among women in spontaneous labor, labor induction, proportion of labor with oxytocin augmentation, maximum dose of oxytocin, duration (minutes) of active stage (5 cm to either 10 cm or to cesarean delivery), vaginal examinations per hour in the first stage of labor, duration (minutes) from complete dilation (10 cm) to start of pushing, duration (minutes) from start of pushing to delivery, vaginal delivery, episiotomy, and type of anesthesia (epidural/regional or general). The process measures were added individually to patient and hospital characteristic-adjusted models that were restricted to women who were eligible for the type of care being assessed (eg, labor induction was not assessed among women with a placenta previa, because women with this diagnosis would not be eligible to receive induction). To facilitate interpretation, process measures that were explored initially as continuous variables were dichotomized for use in the final regression model based on clinical relevance and assessment of plots with the use of a locally weighted scatterplot smoothing technique. Process measures that were associated significantly with a greater frequency of an adverse obstetric outcome were identified and used to derive a composite process measure “exposure score” that was calculated according to the methods by Peterson et al as the proportion of the care processes that a patient was eligible to receive that actually were received by the patient. Thus, if a patient received 3 of the 4 care processes that were associated significantly with the outcome of interest, her composite exposure was 75%.
SAS software (SAS Institute, Cary, NC) was used for the analyses. All tests were 2-tailed. A probability value of < .01 was used to define statistical significance; 99% CIs were estimated when directly testing a hypothesis (ie, examining the association between the process measures and outcomes) and to identify hospital outliers. A probability value of < .05 and 95% CIs were estimated for model building and other descriptive analyses.
Results
During the study period, data were collected on 115,502 women and their neonates and on 1797 different delivery-attending providers at 25 hospitals. Characteristics of these patients and their providers and hospitals are provided in Tables 1 and 2 . As shown, women were delivered by a variety of types of providers, and these providers had a range of experience. Hospital characteristics, including availability of medical services (eg, obstetric anesthesia), the presence of electronic medical records, and the attendance of providers at structured obstetric patient review, varied as well.
| Characteristic | n (%) |
|---|---|
| MATERNAL | |
| Age, y | |
| <20 | 10,187 (8.8) |
| 20-24.9 | 24,299 (21.0) |
| 25-29.9 | 31,101 (26.9) |
| 30-34.9 | 30,570 (26.5) |
| ≥35 | 19,345 (16.8) |
| Race/ethnicity a | |
| Non-Hispanic white | 52,040 (45.1) |
| Non-Hispanic black | 23,878 (20.7) |
| Non-Hispanic Asian | 5999 (5.2) |
| Hispanic | 27,291 (23.6) |
| Other | 5083 (4.4) |
| Not documented | 1211 (1.1) |
| Body mass index at delivery, b kg/m 2 | |
| <25 | 14,242 (12.6) |
| 25-29.9 | 41,268 (36.5) |
| 30-34.9 | 32,088 (28.4) |
| 35-39.9 | 15,088 (13.3) |
| ≥40 | 10,481 (9.3) |
| Cigarette use during pregnancy | 11,370 (9.9) |
| Cocaine or methamphetamine use during pregnancy | 830 (0.7) |
| Insurance status | |
| Uninsured/self-pay | 11,989 (10.5) |
| Government-assisted | 45,125 (39.4) |
| Private | 57,462 (50.2) |
| Prenatal care b | 107,510 (97.9) |
| Obstetric history | |
| Nulliparous | 46,773 (40.5) |
| Previous vaginal delivery only | 49,865 (43.2) |
| Previous cesarean delivery only | 8872 (7.7) |
| Previous cesarean and vaginal deliveries | 9963 (8.6) |
| Any hypertension | 13,272 (11.5) |
| Diabetes mellitus | |
| None | 106,706 (92.4) |
| Gestational | 6999 (6.1) |
| Pregestational | 1734 (1.5) |
| Anticoagulant use during pregnancy | 920 (0.8) |
| Multiple gestation | 2815 (2.4) |
| Polyhydramnios | 940 (0.8) |
| Oligohydramnios | 4700 (4.1) |
| Placenta previa | 467 (0.4) |
| Placenta accreta | 158 (0.1) |
| Placental abruption | 930 (0.8) |
| Premature rupture of membranes/preterm premature rupture of membranes b | 6004 (5.3) |
| Group B Streptococcus status | |
| Negative | 68,918 (59.7) |
| Positive | 24,390 (21.1) |
| Unknown | 22,194 (19.2) |
| NEONATAL | |
| Presentation at delivery | |
| Vertex | 111,174 (94.1) |
| Breech | 6010 (5.1) |
| Nonbreech malpresentation | 931 (0.8) |
| Gestational age at delivery, wk | |
| 23 0 -27 6 | 1256 (1.1) |
| 28 0 -33 6 | 4282 (3.6) |
| 34 0 -36 6 | 10,024 (8.5) |
| 37 0 -37 6 | 10,914 (9.2) |
| 38 0 -38 6 | 20,723 (17.5) |
| 39 0 -39 6 | 37,695 (31.8) |
| 40 0 -40 6 | 23,876 (20.2) |
| 41 0 -41 6 | 8998 (7.6) |
| ≥42 0 | 654 (0.6) |
| Birthweight, g | |
| <2500 | 12,498 (10.6) |
| 2500-3999 | 96,708 (81.7) |
| ≥4000 | 9186 (7.8) |
| Size for gestational age | |
| Small | 11,530 (9.7) |
| Appropriate | 97,774 (82.6) |
| Large | 9088 (7.7) |
a Race/ethnicity was reported in the chart
b n = 113,167 with body mass index data; n = 109,773 with prenatal care visit data; n = 113,446 with premature rupture of membranes/preterm premature rupture of membranes data.
| Characteristics | n (%) |
|---|---|
| Specialty of attending at delivery | |
| General obstetrics and gynecology | 84,057 (72.8) |
| Midwife | 7808 (6.8) |
| Family medicine | 3728 (3.2) |
| Maternal-fetal medicine | 18,954 (16.4) |
| No attending at delivery | 859 (0.7) |
| Years since attending at delivery graduated medical or midwifery school | |
| 0-9.9 (includes no attending at delivery) | 26,717 (23.4) |
| 10-14.9 | 21,793 (19.1) |
| 15-20.9 | 19,880 (17.4) |
| 20-24.9 | 16,248 (14.2) |
| ≥25 | 29,428 (25.8) |
| Nurse-to-patient ratio at delivery a | |
| <1 | 31,781 (27.6) |
| 1-1.9 | 58,263 (50.7) |
| 2-2.9 | 15,804 (13.7) |
| ≥3 | 9160 (8.0) |
| Patient delivered at hospital where prenatal electronic medical record available | |
| No | 47,727 (41.3) |
| Sometimes | 35,083 (30.4) |
| Yes | 32,692 (28.3) |
| Patient delivered at hospital with 24-hour in-house obstetric anesthesia service | |
| No | 13,150 (11.4) |
| Yes | 102,352 (88.6) |
| Patient delivered at hospital with 24-hour in-house attending obstetric provider | |
| No | 13,823 (12.0) |
| Yes | 101,679 (88.0) |
| Patient delivered at hospital with attending providers and/or nurses present for structured obstetric patient review b | |
| No obstetricians present at review | 21,106 (18.3) |
| Obstetricians but no nurses present at review | 38,052 (32.9) |
| Both obstetricians and nurses present at review | 56,344 (48.8) |
| Patient delivered at hospital with 24-hour in-house interventional radiology available | |
| No | 79,452 (68.8) |
| Yes | 36,050 (31.2) |
| Patient delivered at hospital with 24-hour in-house attending neonatologist or pediatrician | |
| No neonatologist, no pediatrician | 12,532 (10.9) |
| Pediatrician, no neonatologist | 4363 (3.8) |
| Neonatologist | 98,314 (85.3) |
a Total number of nursing hours worked in Labor and Delivery during the 8-hour shift divided by 8, divided by the number of patient admissions during the 8-hour shift
b Official board sign-out at shift change or other structured patient review.
The frequencies of the selected outcomes were as follows: PPH, 2.29% (95% CI, 2.20–2.38%), peripartum infection, 5.06% (95% CI, 4.93–5.19%), severe perineal laceration at spontaneous vaginal delivery, 2.16% (95% CI, 2.06–2.27%), severe perineal laceration at forceps-assisted vaginal delivery, 27.56% (95% CI, 25.54–29.57%), severe perineal laceration at vacuum-assisted vaginal delivery, 14.51% (95% CI, 13.34–15.67%), and composite neonatal adverse outcome, 2.73% (95% CI, 2.63–2.84%). As previously reported, the frequency of the selected adverse outcomes varied widely and differed significantly among hospitals ( P < .001 for all). The type of care experienced by patients at different hospitals varied widely as well ( Table 3 ).The frequency of labor induction among women who were eligible for such an intervention, for example, ranged among hospitals from 21-37%. Oxytocin at rates >20 mU/min rarely was administered to laboring women at some hospitals; however, this practice occurred in nearly 50% of women who received oxytocin at other hospitals. There was a >20-fold difference in the frequency of delayed pushing among women who reached the second stage and a difference in the frequency of vaginal delivery that ranged from 61-80%. Delivery practices varied as well, with a 50-fold difference in the frequency of episiotomy among women who had a vaginal delivery and >10-fold difference in the use of general anesthesia at cesarean delivery.
| Variable | Lowest % | Median % | Highest % |
|---|---|---|---|
| Labor induction a | 20.8 | 28.2 | 37.1 |
| Dilation ≤2 cm at admission b | 6.6 | 13.6 | 25.9 |
| Maximum oxytocin ≥20 mU/min c | 8.7 | 17.6 | 46.3 |
| ≥80% of labor augmented with oxytocin d | 1.0 | 10.1 | 22.6 |
| ≥1 hr between complete dilation and initiation of pushing e | 0.8 | 10.9 | 21.2 |
| ≥2 hr between initiation of pushing to delivery e | 4.4 | 9.1 | 19.2 |
| ≥8 hr active phase f | 2.9 | 8.3 | 19.2 |
| <1 vaginal examination per every 3 hr in first stage g | 2.9 | 21.0 | 43.7 |
| Vaginal delivery h | 60.6 | 70.1 | 79.5 |
| Episiotomy i | 0.7 | 7.0 | 35.4 |
| Epidural/regional anesthesia j | 45.3 | 77.7 | 89.7 |
| General anesthesia k | 1.1 | 6.5 | 14.8 |
| Elective delivery at <39 weeks’ gestation without documented fetal lung maturity l | 0.2 | 0.5 | 12.2 |
a In patients with no previa and no history of classic, T, or J cesarean delivery (n = 113,049)
b In patients at term with intact membranes and spontaneous intended labor with no previa and cervical dilation measured within 1 hour before or after Labor and Delivery admission (n = 46,068)
c In patients who received oxytocin in labor (n = 58,228)
d In patients with spontaneous intended labor who were admitted to Labor and Delivery before delivery (n = 61,157)
e In patients who reached complete after intended labor (n = 60,290)
f In patients with intended labor who reached active stage (5 cm) with a term nonanomalous singleton pregnancy (n = 71,571)
g In patients with intended labor who were treated in hospital for >1 hour during first stage (n = 81,826)
h In all patients (n = 115,502)
i In patients with a vaginal delivery and no shoulder dystocia (n = 77,071)
j In patients with nonoperative vaginal delivery of a singleton and no shoulder dystocia and who reached complete after intended labor (n = 70,362)
k In patients with a cesarean delivery (n = 36,201)
l In patients with a term nonanomalous singleton pregnancy (n = 98,509).
The associations of processes measures (individual and composite exposure score) with the studied outcomes are given in Table 4 . Even after adjusting for patient, provider, and hospital characteristics, particular types of obstetric care remained associated with the outcomes of interest.
| Process measure | Postpartum hemorrhage a | Peripartum infection b | Severe perineal laceration at spontaneous vaginal delivery c,d | Severe perineal laceration at forceps-assisted vaginal delivery c,e | Severe perineal laceration at vacuum-assisted vaginal delivery c,d | Composite neonatal adverse outcome f,g |
|---|---|---|---|---|---|---|
| n | 105,987 | 110,205 | 68,144 | 1898 | 3515 | 89,279 |
| Labor induction | 1.20 (1.04–1.37) | 1.22 (1.13–1.33) | 1.04 (0.90–1.21) | 1.05 (0.78–1.42) | 0.92 (0.70–1.21) | 1.18 (1.05–1.34) |
| Dilation ≤2 cm at admission | 1.58 (1.37–1.82) | |||||
| Maximum oxytocin ≥20 mU/min | 1.61 (1.33–1.95) | 1.30 (1.16–1.44) | ||||
| ≥80% of labor augmented with oxytocin | 1.08 (0.78–1.50) | 1.63 (1.42 –1.87) | ||||
| ≥1 hr between complete dilation and initiation of pushing | 1.67 (1.22–2.28) | 1.29 (1.04–1.59) | 1.10 (0.74–1.64) | 0.94 (0.65–1.34) | 1.13 (0.89–1.45) | |
| ≥2 hrs between initiation of pushing to delivery | 4.02 (3.10–5.23) | 1.88 (1.51– 2.34) | 1.21 (0.87–1.69) | 1.55 (1.15–2.09) | 1.83 (1.46–2.28) | |
| ≥8 hrs active stage | 1.32 (1.08–1.62) | |||||
| <1 vaginal examination per every 3 hours in first stage | 1.18 (1.07–1.30) | 1.18 (1.01–1.38) | ||||
| Vaginal delivery | 0.19 (0.16–0.22) | 0.52 (0.47–0.56) | 0.72 (0.63–0.83) | |||
| Episiotomy | 1.22 (1.04–1.43) | 2.47 (2.08–2.93) | 1.24 (0.87–1.79) | 1.99 (1.51–2.62) | ||
| Epidural/regional anesthesia | 0.88 (0.73–1.06) | (small no. precludes analysis) | 0.90 (0.57–1.45) | |||
| General anesthesia | 3.61 (2.98–4.37) | |||||
| Elective delivery at <39 wk gestation without documented fetal lung maturity | 1.39 (0.67–2.89) | |||||
| Composite process measure exposure score (Percentage of care received that was associated with fewer adverse outcomes; referent received 100% of care eligible) | 0-67%: 4.69 (3.89–5.64) | 0-57%: 1.88 (1.68–2.11) | 0-67%: 2.18 (1.88–2.54) | N/A | 0-50%: 2.64 (1.96–3.55) | 0-67%: 1.65 (1.43–1.91) |
| 75-83%: 2.25 (1.79–2.83) | 60-86%: 1.89 (1.70–2.09) | 75-83%: 1.34 (1.16–1.56) |
Stay updated, free articles. Join our Telegram channel
Full access? Get Clinical Tree
