Objective
The purpose of this study was to assess the Adverse Outcome Index perinatal quality indicator system that was derived from administrative data.
Study Design
Adverse events were identified for 10 component measures; the Adverse Outcome Index was calculated by the National Perinatal Information Center from 42 months of administrative data. After retrospective chart review, we estimated positive predictive value for 10 measures that were obtained by corrected calculations of Adverse Outcome Index.
Results
Positive predictive values were 86-100% in 7 indicators, with lower values in 3 indicators: neonatal death, 0/2 fetuses; inborn birth trauma, 22/33 infants (67%); and maternal return to the operating room, 16/33 women (48.5%). In term admission to the neonatal intensive care unit, 107 false negatives were identified, with a negative predictive value of 45%.
Conclusion
Indicator positive predictive value was variable. Performance can be strengthened by methods to identify both false-positive and false-negative adverse events that would include chart review and some measure specification revisions to improve alignment with original indicator intent. Interhospital comparison application requires further study.
In the United States, pregnancy and childbirth is the second most common reason for hospital admission after heart disease. In 2006, there were an estimated 4,127,000 hospital-based deliveries. Patient safety in obstetric care has received less attention than other health conditions. National agreement on a set of quality perinatal core measures has only recently been proposed by the National Quality Forum.
The Washington State Perinatal Regional Networks identified a need for regional benchmarking and meaningful standardized quality indicators to inform its activities. In our search for quality measurement systems, we identified the Adverse Outcome Index (AOI) measurement system offered by the National Perinatal Information Center (NPIC) as a potential tool. The AOI and 2 other associated composite measures, the Weighted Adverse Outcome Score (WAOS) and the Severity Index (SI), are derived from 10 standardized perinatal indicators that were developed by a national panel of experts. A severity weight for each type of event is assigned. More serious adverse events (such as maternal or neonatal death) receive higher weight points (750 and 400 points, respectively), and less serious or more common events (such as third- or fourth-degree laceration and maternal blood transfusion) receive lower weight points (5 and 20 points, respectively; Appendix ). Weight points for each type of adverse event were guided by a consensus process that was carried out at a meeting of the American College of Obstetricians and Gynecologists Quality Improvement and Patient Safety committee. These measures initially were based on prospectively gathered data and were used first for measuring the impact of team training in a multicenter labor and delivery study from 2002-2004 that was sponsored by the Department of Defense and the Harvard Risk Management Strategies Foundation. A less resource-intensive option was developed under this study by NPIC that was based on hospital administrative data-based algorithms for calculating the 10 measures and was judged to compare favorably to the prospectively gathered data-based AOI. A subsequent study at Yale-New Haven Hospital used a prospectively gathered data-based AOI score that was applied to deliveries from 2004-2007 for measurement of the impact of comprehensive patient safety strategies in obstetrics.
Adverse event title (weight points) | Inclusion data | Exclusion data | DRG and ICD-9-CM diagnosis and procedure code descriptions |
---|---|---|---|
Maternal death (750) | DRG 370-375 Discharge disposition = death | Death after discharge or transfer out Transfer in with delivery at other location | DRG 370: cesarean delivery with complications or comorbidities DRG 371: cesarean delivery without complications or comorbidities DRG 372: vaginal delivery with complications or comorbidities DRG 373: vaginal delivery without complications or comorbidities DRG 374: vaginal delivery with sterilization or curettage DRG 375: vaginal delivery with operating room procedure except sterilization and/or curettage |
Intrapartum neonatal death (400) | Birthweight ≥2500 g Inborn only Death within 7 d of birth | Congenital anomalies (diagnosis codes 740-759.9) Fetal hydrops (diagnosis code 778.0) Death at >7 d Birthweight <2500 g | 740-759.9: comprehensive range of all ICD-9-CM congenital anomaly codes available 778.0: idiopathic fetal hydrops (not because of isoimmunization) |
Uterine rupture (100) | DRG 370-375 Diagnosis code 665.1 in primary, first, or second diagnosis code position only | Diagnosis code 665.1 in ≥3rd diagnosis code position | 665.1: rupture of uterus during labor DRGs 370-375 (as above) |
Unplanned maternal admission to intensive care unit (65) | DRG 370-375 Intensive care unit day or charge at anytime during stay (before or after delivery) Diagnosis code 640.xx – 677.xx with 5th digit = 2 (delivered with mention of postpartum condition) | Intensive care unit stay or charges without diagnosis code 640.xx –677.xx and 5th digit = 2 | 640.xx-677.xx: includes complications and comorbidities that include hemorrhage, hypertension, infections of genitourinary tract, other infections, anesthetic complications, other conditions in mother that complicate the postpartum condition that includes diabetes mellitus, thyroid dysfunction, anemia, drug dependence, mental disorders, congenital cardiovascular disorders DRGs 370-375 (as above) |
Birth trauma (60) | Inborn only (all gestational ages included) Diagnosis codes 767.0, 767.11, 767.3, 767.4, 767.5, 767.6, 767.7 | Outborn transferred in | 767.0: subdural and cerebral hemorrhage 767.11: epicranial subaponeurotic hemorrhage-massive 767.3: other injuries to skeleton 767.4: injury to spine and spinal cord 767.5: facial nerve injury 767.6: injury to brachial plexus 767.7: other cranial and peripheral nerve injuries |
Maternal return or operating room/labor and delivery (40) | DRG 370-375 Any of following procedure codes in first or second position: 75.92, 69.02, 54.61, 38.86, 39.98, 69.52 | Any other procedures and same inclusion list of procedure codes but in ≥3rd position | 75.92: evacuation of other hematoma of vulva or vagina 69.02: curettage after delivery 54.61: reclosure of postoperative disruption of abdominal wall 38.86: other surgical occlusion of abdominal vessels 39.98: control of hemorrhage 69.52: aspiration curettage after delivery DRGs 370-375 (as above) |
Admission to neonatal intensive care unit (35) | Inborns only ≥2500 g ≥37 wk gestation Intensive care unit charge or length of stay >1 d Within 1 d of birth | Congenital anomalies: diagnosis codes 740-759.9 Fetal hydrops: diagnosis code 778.0 | 740-759.9: comprehensive range of all ICD-9-CM congenital anomaly codes available 778.0: idiopathic fetal hydrops (not because of isoimmunization) |
Apgar score of <7 at 5 min (25) | Inborns only and birthweight ≥2500 g | Congenital anomalies: diagnosis codes 740-759.9 Fetal hydrops: diagnosis code 778.0 | 740-759.9: comprehensive range of all ICD-9-CM congenital anomaly codes available 778.0: idiopathic fetal hydrops (not because of isoimmunization) |
Maternal blood transfusion (20) | DRG 370-375 Procedure code 99.0 or Blood Transfusion indicator = 1 | Transfers into hospital after delivery | 99.0: transfusion of blood and blood components DRGs 370-375 (as above) |
Third- or fourth-degree perineal laceration (5) | DRG 370-375 Diagnosis code 664.2 or 664.3 | Transfers into hospital after delivery | 664.2: third-degree perineal laceration 664.3: fourth-degree perineal laceration |
The potential use of these standardized adverse outcome indicators that were based on hospital coded data was an attractive quality benchmarking possibility for our statewide efforts. Use of quality measures from administrative coding is less resource intensive, compared with quality measures that are based on prospectively collected or chart abstracted data. However, reliability of administrative data for the identification of clinically accurate quality issues has been mixed. In a comprehensive study that involved 52 California hospitals, Yasmeen et al found wide variation in positive predictive values for obstetric diagnoses and procedures in hospital discharge data that ranged from 28-100%. Before recommending the use of the AOI for our statewide efforts, we chose to investigate the application of the AOI system to maternal and neonatal data at the University of Washington Medical Center.
Materials and Methods
Our quaternary center is 1 of 2 major obstetric referral centers for western Washington State. Annual deliveries are just >2000, with 20% low birthweight neonates and 3.6% of the neonates <1000 g. Maternal demographics for the time period for primary payer were 52% Medicaid and 44% commercial insurance. Fifty-three percent of the women were 18-30 years old; 44% of the women were 31-50 years old, and 2.5% of the women were <18 years old. Our study proposal to examine and validate the AOI indicators and composite measures by medical record review was submitted to the University of Washington Human Subjects Division and received Institutional Review Board approval.
The 10 adverse events, their definitions and respective weights that comprise NPIC’s AOI measurement system are included in the Table . The 3 composite scores that were derived from these adverse event cases are calculated in the following manner: AOI, percent of all deliveries with ≥1 adverse events; WAOS, the sum of each adverse event multiplied by its severity weight then divided by the total number of deliveries; SI, the sum of each adverse event multiplied by its severity weight then divided by the number of deliveries with an adverse event.
Adverse outcome indicator (weight points) | Adverse events identified by algorithm | Contribution to initial Weighted Adverse Outcome Score, % | Records reviewed | Confirmation by record review | Positive predictive value, % | 95% CI |
---|---|---|---|---|---|---|
Intrapartum neonatal death ≥2500 g (400) | 2 | 4.6 | 2 | 0 | 0 | 0–84.2 a |
Apgar 5 <7 for bodyweight ≥2500 g (25) | 153 | 22 | 33 b | 31 | 93.90 | 79.8–99.3 |
Inborn birth trauma (60) | 33 | 11.4 | 33 | 22 | 66.70 | 48.2–82.0 |
Admission to neonatal intensive care unit >1 day, bodyweight ≥2500 g and gestational age ≥37 wks (35) | 75 | 15.1 | 75 | 74 | 98.7 | 92.8–99.97 |
Maternal death (750) | 1 | 4.3 | 1 | 1 | 100 | 2.5–100 a |
Uterine rupture (100) | 7 | 4 | 7 | 6 | 85.7 | 42.1–99.6 |
Unplanned maternal admission to intensive care unit (65) | 18 | 6.7 | 18 | 18 | 100 | 81.5–100 a |
Maternal blood transfusions (20) | 164 | 18.9 | 33 b | 33 | 100 | 89.4–100 a |
Unplanned maternal return to operating room or labor and delivery (40) | 33 | 7.6 | 33 | 16 | 48.5 | 30.8–66.5 |
Third- and fourth-degree perineal laceration (5) | 181 | 5.2 | 44 b | 44 | 100 | 91.96–100 a |