Objective
To determine the impact on referral patterns of using a Multivariate Index Assay, CA125, modified-American College of Obstetricians and Gynecologists referral guidelines, and clinical assessment among patients undergoing surgery for an adnexal mass after initial evaluation by nongynecologic oncologists.
Study Design
Overall, 770 patients were enrolled by nongynecologic oncologists from 2 related, multiinstitutional, prospective trials and analyzed retrospectively. All patients had preoperative imaging and biomarker analysis. The subset of patients enrolled by nongynecologic oncologists was analyzed to determine the projected referral patterns and sensitivity for malignancy based on multivariate index assay (MIA), CA125, modified-American College of Obstetricians and Gynecologists (ACOG) guidelines, and clinical assessment compared with actual practice.
Results
The prevalence of malignancy was 21.3% (n = 164). In clinical practice, 462/770 patients (60.0%) were referred to a gynecologic oncologist for surgery. Triage based on CA125 predicted referral of 157/770 patients (20.4%) with sensitivity of 68.3% (95% confidence interval [CI], 60.8–74.9). Triage based on modified-ACOG guidelines would have resulted in referral of 256/770 patients (33.2%) with a sensitivity of 79.3% (95% CI, 72.4–84.8). Clinical assessment predicted referral of 184/763 patients (24.1%) with a sensitivity of 73.2% (95% CI, 65.9–79.4). Risk stratification using multivariate index assay would have resulted in referral of 429/770 (55.7%) patients, with sensitivity of 90.2% (95% CI, 84.7–93.9). MIA demonstrated statistically significant higher sensitivity ( P < .0001) and lower specificity ( P < .0001) for detecting malignancy compared with clinical assessment, CA125, and modified-ACOG guidelines.
Conclusion
In this study population, use of MIA as a risk stratification test was associated with referral patterns by nongynecologic oncologists comparable to actual clinical practice and higher sensitivity for malignancy than other adnexal mass triage algorithms.
The American Cancer Society has estimated that 22,240 new cases of ovarian cancer will be diagnosed in the United States in 2013. With an estimated 14,030 women dying from disease, ovarian cancer accounts for as many deaths as all other gynecologic cancers combined. The number of women diagnosed with an adnexal mass far exceeds the number of ovarian cancer cases, making accurate identification of the subgroup of patients most likely to benefit from consultation with a gynecologic oncologist a clinical challenge. To date, no single prediction model or set of referral guidelines for the evaluation of an adnexal mass has received widespread acceptance. Recently, novel biomarker testing decision algorithms have been developed to aid in the preoperative evaluation process. These triage tools are not screening tests, which are designed to detect disease in asymptomatic patients. The multivariate index assay (MIA, OVA1) is a multiple biomarker test that was cleared for use in clinical practice by the Food and Drug Administration (FDA) in 2009 based on a high sensitivity and negative predictive value for identifying ovarian malignancy in a clinical utility study reported by Ueland et al. The high sensitivity and negative predictive value for ovarian malignancy were confirmed in an independent but related intended-use provider population clinical validation study.
Optimizing the sensitivity of a diagnostic triage test necessarily comes at the expense of a reduced level of specificity or low positive predictive value (PPV). As such, concerns have been raised about the potential for overreferral to gynecologic oncology specialists of women with an adnexal mass. Although multiple population-based studies have documented that fewer than 50% of ovarian cancer patients in the United States are referred to gynecologic oncologists for surgery, there is surprisingly little data examining the preceding triage phase of the clinical care continuum–referral patterns of women diagnosed with an adnexal mass without a known diagnosis of ovarian cancer. The objective of the current study was to determine the projected impact on referral patterns of patients undergoing surgery for an adnexal mass after initial evaluation by a nongynecologic oncologist using the MIA, CA125, modified-American College of Obstetricians and Gynecologists (ACOG) referral guidelines, and clinical assessment using the combined datasets of 2 previously reported clinical trials of the MIA.
Materials and Methods
The data from 2 independent but related national clinical trials (clinical utility study and intended-use validation study) conducted between 2007 and 2012 on the use of the MIA for triaging patients with an adnexal mass were merged and analyzed retrospectively. A total of 1110 subjects were prospectively enrolled at 44 sites across the United States, including primary care women’s health clinics, general obstetrics and gynecology group practices, gynecologic oncology practices, community and university-based hospitals, and health maintenance organizations. Institutional review board approval was obtained from each site. From the combined clinical trial populations, the subset of patients enrolled by nongynecologic oncologist providers was selected for further study. All clinicians initially enrolling patients in the current analysis were from nongynecologic oncology specialty practices, although patients may ultimately have had consultation with or undergone surgery by a gynecologic oncologist.
Both trials had identical inclusion and exclusion criteria: consented females aged ≥18 years, agreeable to phlebotomy, with an adnexal mass documented by imaging (computed tomography, ultrasonography, or magnetic resonance imaging) and planned for surgery within 3 months of imaging. Exclusion criteria included a diagnosed malignancy within the past 5 years, with the exception of nonmelanoma skin cancer, declined phlebotomy or did not conduct the surgery within 3 months as planned. Menopause was defined as the absence of menses for ≥12 months, or age ≥50 years. All data were collected on standardized case report forms.
The methods for blood collection and specimen handling have been previously reported. Biomarker measurements were performed according to the MIA Instructions for Use at Quest Diagnostics, Inc (Chantilly, VA) or the Division of Clinical Chemistry, Department of Pathology, Johns Hopkins Medical Institutions. The MIA is a FDA-cleared, assay that incorporates CA125 (CA125-II), transferrin, transthyretin (prealbumin), apolipoprotein A1, and beta-2-microglobulin. These biomarker results were transformed by the OvaCalc software (Vermillion, Inc., Austin, TX) using a proprietary multivariate algorithm to generate an ovarian malignancy risk score as described previously. The algorithm renders a single risk score from 0.0 to 10.0, and subjects were stratified as high risk with MIA scores ≥5.0 (premenopausal) or ≥4.4 (postmenopausal). For CA125 measurement, the same value used for MIA calculation was used for individual analysis, and compared with clinical cutoff values in accordance with published ACOG referral criteria of ≥200 units/mL for premenopausal women or >35 units/mL for postmenopausal women.
To establish a benchmark for clinical accuracy in predicting ovarian malignancy before surgery, clinicians were required to document their assessment according to previously reported methods. Clinical assessment methods always included physical examination, family history, imaging, and laboratory tests (including CA125 results, if used) but not MIA results. Clinical prediction of malignancy was recorded, as was the specialty of the surgeon who performed surgery (nongynecologic oncologist or gynecologic oncologist). The postoperative pathology diagnosis was recorded at each enrolling site, and independently reviewed.
The Dearking modified-ACOG guidelines for consultation with a gynecologic oncologist were used for patients meeting 1 or more of the following criteria :
Premenopausal women
- 1.
Very elevated CA125 (>67 units/mL)
- 2.
Ascites
- 3.
Evidence of abdominal or distant metastasis
Postmenopausal women
- 1.
Elevated CA125 (>35 units/mL)
- 2.
Nodular or fixed pelvic mass
- 3.
Ascites
- 4.
Evidence of abdominal or distant metastasis
Case report forms, biomarker values and MIA scores were sent to Applied Clinical Intelligence for statistical analysis. Results were statistically stratified based on specialty of clinician who performed the surgery, the subject’s menopausal status, stage of malignancy, and surgical pathology. Clinical diagnostic performance criteria (sensitivity, specificity, PPV, and negative predictive value) were calculated for clinical assessment, CA125, modified ACOG guidelines, and MIA. Ninety-five percent confidence intervals were constructed where appropriate, and differences in sensitivity and specificity were tested for statistical significance using McNemar’s test. Statistical analysis was performed with SAS 9.2 (SAS Institute Inc., Cary, NC). Descriptive statistics were compiled for the expected and actual gynecologic oncologist referral rates for clinical assessment, CA125, and modified-ACOG guidelines based on surgeon of record.
Results
A total of 770 patients (clinical utility study, n = 276; intended-use validation study, n = 494) were enrolled by nongynecologic oncologist providers and were evaluable for CA125, MIA, and modified-ACOG guidelines, and 763 patients were evaluable for clinical assessment of the likelihood of ovarian cancer ( Table 1 ). The overall prevalence of malignancy was 21.3%. A primary invasive malignancy arising in the ovary or ovaries was present in 14.9% of cases (n = 115), whereas, invasive epithelial ovarian cancer was the pathological diagnosis in 13.5% of cases. Of the invasive ovarian malignancies, 52.1% of cases had International Federation of Gynecology and Obstetrics (FIGO) stage I or stage II disease.
Demographic | All subjects n = 770 (%) | Premenopausal women n = 424 (%) | Postmenopausal women n = 346 (%) |
---|---|---|---|
Age, y | |||
Mean (standard deviation) | 49.0 (13.97) | 39.9 (8.74) | 60.2 (10.72) |
Median | 48 | 42 | 60 |
Range (minimum–maximum) | 18–90 | 18–60 | 33–90 |
Ethnicity | |||
Asian | 17 (2.2) | 11 (2.6) | 6 (1.7) |
Black or African American | 116 (15.1) | 81 (19.1) | 35 (10.1) |
Native Hawaiian or Other Pacific Islander | 1 (0.1) | 1 (0.2) | 0 (0.0) |
White | 558 (72.5) | 271 (63.9) | 287 (82.9) |
Other | 6 (0.8) | 5 (1.2) | 1 (0.3) |
Hispanic or Latino | 72 (9.4) | 55 (13.0) | 17 (4.9) |
Number of pregnancies | |||
None | 129 (16.8) | 90 (21.2) | 39 (11.3) |
1 | 127 (16.5) | 78 (18.4) | 49 (14.2) |
2 | 196 (25.5) | 106 (25.0) | 90 (26.0) |
3 | 152 (19.7) | 80 (18.9) | 72 (20.8) |
≥4 | 164 (21.3) | 70 (16.5) | 94 (27.2) |
Not specified | 2 (0.3) | 0 (0.0) | 2 (0.6) |
Enrolling physician | |||
Nongynecologic oncologist | 770 (100.0) | 424 (100.0) | 346 (100.0) |
Surgeon | |||
Nongynecologic oncologist | 308 (40.0) | 218 (51.4) | 90 (26.0) |
Gynecologic oncologist | 462 (60.0) | 206 (48.6) | 256 (74.0) |
Malignancy | |||
Malignant | 164 (21.3) | 57 (13.4) | 107 (30.9) |
Benign | 606 (78.7) | 367 (86.6) | 239 (69.1) |
Pathological diagnosis | |||
Benign ovarian conditions | 606 (78.7) | 367 (86.6) | 239 (69.1) |
Epithelial ovarian cancer | 104 (13.5) | 32 (7.5) | 72 (20.8) |
Other primary ovarian malignancies | 11 (1.4) | 8 (1.9) | 3 (0.9) |
Low malignant potential (borderline) | 29 (3.8) | 7 (1.7) | 22 (6.4) |
Nonprimary ovarian malignancy with involvement of the ovaries | 11 (1.4) | 6 (1.4) | 5 (1.4) |
Nonprimary ovarian malignancy without involvement of ovaries | 9 (1.2) | 4 (0.9) | 5 (1.4) |
If malignant ovarian tumor: predominant histology (primary ovarian cancer), n | 115 | 40 | 75 |
Epithelial | |||
Serous | 52 (45.2) | 16 (40.0) | 36 (48.0) |
Mucinous | 13 (11.3) | 3 (7.5) | 10 (13.3) |
Endometroid | 18 (15.7) | 7 (17.5) | 11 (14.7) |
Clear cell | 7 (6.1) | 2 (5.0) | 5 (6.7) |
Carcinosarcoma | 3 (2.6) | 1 (2.5) | 2 (2.7) |
Mixed | 2 (1.7) | 1 (2.5) | 1 (1.3) |
Undifferentiated | 2 (1.7) | 0 (0.0) | 2 (2.7) |
Other | 12 (10.4) | 5 (12.5) | 7 (9.3) |
Nonepithelial, other | 6 (5.2) | 5 (12.5) | 1 (1.3) |
Stage (primary ovarian cancer) | |||
Stage I | 42 (36.5) | 15 (37.5) | 27 (36.0) |
Stage II | 18 (15.7) | 8 (20.0) | 10 (13.3) |
Stage III | 50 (43.5) | 15 (37.5) | 35 (46.7) |
Stage IV | 5 (4.3) | 2 (5.0) | 3 (4.0) |
Grade (primary ovarian cancer) | |||
Grade 1 | 14 (12.2) | 3 (7.5) | 11 (14.7) |
Grade 2 | 24 (20.9) | 11 (27.5) | 13 (17.3) |
Grade 3 | 69 (60.0) | 23 (57.5) | 46 (61.3) |
Grade 4 | 5 (4.3) | 3 (7.5) | 2 (2.7) |
Not given | 3 (2.6) | 0 (0.0) | 3 (4.0) |
Test performance for detection of malignancy is shown in Table 2 . MIA demonstrated statistically significantly higher sensitivity (90.2%, 95% confidence interval [CI], 84.7–93.9) compared with clinical assessment (73.2%, 95% CI, 65.9–79.4), CA125 (68.3%, 95% CI, 60.8–74.9), and modified ACOG guidelines (79.3%, 95% CI, 72.4–84.8) ( P < .0001). MIA showed statistically significant deterioration in specificity compared with clinical assessment, CA125, and modified-ACOG criteria ( P < .0001). Test performance for detection of malignancy stratified by menopausal status is shown in Tables 3 and 4 .
Variable | Clinical assessment | CA125 a | Modified ACOG guidelines | MIA |
---|---|---|---|---|
(n = 763) | (n = 770) | (n = 770) | (n = 770) | |
Sensitivity | 73.2 | 68.4 | 79.3 | 90.2 |
n/N | 120/164 | 112/164 | 130/164 | 148/164 |
95% CI | 65.9–79.4 | 60.8–74.9 | 72.4–84.8 | 84.7–93.9 |
Specificity | 89.3 | 92.6 | 79.2 | 53.6 |
n/N | 535/599 | 561/606 | 480/606 | 325/606 |
95% CI | 86.6–91.5 | 90.2–94.4 | 75.8–82.3 | 49.6–57.6 |
PPV | 65.2 | 71.3 | 50.8 | 34.5 |
n/N | 120/184 | 112/157 | 130/256 | 148/429 |
95% CI | 58.1–71.7 | 63.8–77.8 | 44.7–56.8 | 30.2–39.1 |
NPV | 92.4 | 91.5 | 93.4 | 95.3 |
n/N | 535/579 | 561/613 | 480/514 | 325/341 |
95% CI | 90.0–94.3 | 89.0–93.5 | 90.9–95.2 | 92.5–97.1 |
a High risk cutoff: premenopausal subjects CA125 >67U/mL; postmenopausal subjects CA125 >35U/mL.
Variable | Clinical assessment | CA125 a | Modified ACOG guidelines | MIA |
---|---|---|---|---|
(n = 421) | (n = 424) | (n = 424) | (n = 424) | |
Sensitivity | 70.2 | 52.6 | 82.5 | 87.7 |
n/N | 40/57 | 30/57 | 47/57 | 50/57 |
95% CI | 57.3–80.5 | 39.9–65.0 | 70.6–90.2 | 88.1–99.0 |
Specificity | 90.1 | 96.2 | 75.5 | 60.5 |
n/N | 328/364 | 353/367 | 277/367 | 222/367 |
95% CI | 86.6–92.8 | 93.7–97.7 | 70.8–79.6 | 55.4–65.4 |
PPV | 52.6 | 68.2 | 34.3 | 25.6 |
n/N | 40/76 | 30/44 | 47/137 | 50/195 |
95% CI | 41.6–63.5 | 53.4–80.0 | 26.9–42.6 | 20.0–32.2 |
NPV | 95.1 | 92.9 | 96.5 | 96.9 |
n/N | 328/345 | 353/380 | 277/287 | 222/229 |
95% CI | 92.3–96.9 | 89.9–95.1 | 93.7–98.1 | 93.8–98.5 |