Introduction
Ovarian cancer is a common and lethal disease for which early detection and treatment in high-volume centers and by specialized clinicians is known to improve survival. Hence, accurate methods to preoperatively characterize the nature of an ovarian tumor are pivotal. In 2008 the International Ovarian Tumor Analysis (IOTA) group described the Simple Rules . These are based on a set of 5 ultrasound features indicative of a benign tumor (B-features) and 5 ultrasound features indicative of a malignant tumor (M-features). When using the Simple Rules , tumors are classified as benign if only B-features are observed and as malignant if only M-features are observed. If no features are observed or if conflicting features are present, the Simple Rules cannot classify the tumor as benign or malignant (inconclusive results). Masses in which the Simple Rules yield an inconclusive result can be classified using subjective assessment by an experienced ultrasound operator or, given the high prevalence of malignancy in this group, they can all be classified as malignant to increase the sensitivity for ovarian cancer. On prospective validation both by the IOTA group (2 studies including 1938 and 2403 patients, respectively) and by other research teams (9 studies including a total of 2101 tumors), the Simple Rules were applicable in 77-94% of tumors (range between studies). The malignancy rate ranged from 1-9% in cases classified as benign, from 69-94% in cases classified as malignant, and from 13-53% in inconclusive cases. In a metaanalysis comparing the ability of 19 methods to discriminate between benign and malignant adnexal masses before surgery, the Simple Rules had a sensitivity of 93% and a specificity of 81% when classifying inconclusive tumors as malignant. In the metaanalysis the Simple Rules and the IOTA logistic regression model 2 were superior to all other methods. This suggests that evidence-based approaches to the preoperative characterization of adnexal masses should incorporate the use of Simple Rules or the logistic regression model 2. Logistic regression model 2 is a mathematical risk prediction model based on age and 5 ultrasound variables (presence of blood flow in a papillary structure, irregular cyst walls, ascites, acoustic shadows, and maximum diameter of the largest solid component).
The Simple Rules have been well received by clinicians, and the Royal College of Obstetricians and Gynecologists in the United Kingdom has included the Simple Rules in their Green Top guideline on the assessment and management of ovarian masses in premenopausal women.
Despite a combination of simplicity and excellent performance, important limitations of the Simple Rules are the inconclusive results in a proportion of cases and the absence of an estimated risk of malignancy. The ability to provide accurate risk estimates is highly relevant for risk stratification and individualized patient management. The objective of this study was to develop and validate a model to calculate the risk of malignancy in adnexal masses based on the 10 ultrasound features in the Simple Rules .
Materials and Methods
Study design and setting
This international multicenter cross-sectional cohort study involves patients from 22 centers (oncology centers and other hospitals) ( Table 1 ) with at least 1 adnexal (ovarian, paraovarian, or tubal) tumor selected for surgery by the managing clinician. Exclusion criteria were: (1) pregnancy at the time of examination, (2) refusal of transvaginal ultrasonography, (3) declining participation, and (4) surgical intervention >120 days after the ultrasound examination. Data collection was carried out within the framework of the IOTA collaboration. The primary aim of the IOTA studies is to develop and validate methods for making a correct diagnosis in adnexal tumors prior to surgery. This aim is pursued by prospectively examining a large number of patients with ultrasound using a standardized examination technique and standardized terms and definitions to describe ultrasound findings. Through consecutive phases, data were collected from 24 centers in 10 countries. In phase 1 data were collected from 1999 through 2002, in phase 1b from 2002 through 2005, in phase 2 from 2005 through 2007, and in phase 3 from 2009 through 2012. Data from phase 1 were used to develop the Simple Rules and were therefore not used in the present study. The research protocols were approved by the ethics committees in each contributing center.
| Center | Data set | Patients | Malignant, N (%) | Classification using SR | ||
|---|---|---|---|---|---|---|
| SR benign, N (%mal) | SR inconclusive, N (%mal) | SR malignant, N (%mal) | ||||
| All oncology centers | 3263 | 1402 (43) | 1436 (5) | 788 (49) | 1039 (90) | |
| Leuven, Belgium | D,V | 668 | 242 (36) | 306 (4) | 153 (35) | 209 (85) |
| Rome, Italy | D,V | 661 | 365 (55) | 224 (7) | 163 (59) | 274 (92) |
| Monza, Italy | D,V | 356 | 76 (22) | 247 (4) | 69 (42) | 40 (95) |
| Prague, Czech Republic | D,V | 354 | 234 (66) | 102 (13) | 109 (77) | 143 (96) |
| Milan, Italy | D,V | 312 | 177 (57) | 112 (7) | 45 (56) | 155 (93) |
| Lublin, Poland | D,V | 285 | 102 (36) | 132 (5) | 86 (45) | 67 (85) |
| Bologna, Italy a | V | 213 | 65 (31) | 126 (3) | 52 (58) | 35 (89) |
| Stockholm, Sweden | V | 120 | 53 (44) | 38 (0) | 33 (27) | 49 (90) |
| Lund, Sweden | D,V | 77 | 20 (26) | 36 (0) | 20 (10) | 21 (86) |
| Beijing, China | D | 73 | 16 (22) | 36 (0) | 20 (15) | 17 (76) |
| London, United Kingdom | D | 65 | 25 (38) | 32 (6) | 18 (50) | 15 (93) |
| Udine, Italy | D,V | 64 | 19 (30) | 36 (3) | 16 (44) | 12 (92) |
| Naples 2, Italy | D,V | 15 | 8 (53) | 9 (22) | 4 (100) | 2 (100) |
| All other centers | 1585 | 263 (17) | 1021 (1) | 327 (23) | 237 (76) | |
| Malmö, Sweden | D,V | 462 | 100 (22) | 205 (0) | 146 (12) | 111 (74) |
| Genk, Belgium | D,V | 428 | 61 (14) | 301 (1) | 67 (21) | 60 (73) |
| Cagliari, Italy | D,V | 261 | 37 (14) | 200 (2) | 36 (33) | 25 (88) |
| Milan 2, Italy | D,V | 136 | 20 (15) | 99 (0) | 25 (40) | 12 (83) |
| Bologna, Italy a | D | 135 | 11 (8) | 110 (0) | 15 (27) | 10 (70) |
| Naples, Italy | D,V | 72 | 18 (25) | 42 (2) | 17 (35) | 13 (85) |
| Barcelona, Spain | V | 37 | 11 (30) | 21 (10) | 11 (55) | 5 (60) |
| Milan 3, Italy | D | 21 | 4 (19) | 13 (0) | 7 (43) | 1 (100) |
| Milan 4, Italy | V | 21 | 0 (0) | 20 (0) | 1 (0) | 0 (–) |
| Hamilton, Ontario, Canada | D | 12 | 1 (8) | 10 (0) | 2 (50) | 0 (–) |
a Bologna Center in Italy changed from other hospital to oncology center during course of International Ovarian Tumor Analysis study and is therefore listed in both categories (different patients in 2 types of centers).
Data collection
Oral and/or written informed consent was obtained in accordance with the requirements of the local ethics committee. A standardized history was taken from each patient to collect clinical information. All patients underwent a standardized transvaginal ultrasound examination by a principal investigator, who was a gynecologist or radiologist with extensive experience in gynecological ultrasound and with a special interest in adnexal masses. Transabdominal sonography was added in women with large masses that could not be visualized completely by the transvaginal approach. For women with multiple masses, the dominant mass was selected for statistical analysis. To apply the Simple Rules , information on the following variables is required: the diameters of the lesion (millimeters), the diameters of the largest solid component (millimeters), type of tumor (unilocular, unilocular-solid, multilocular, multilocular-solid, solid), presence of wall irregularity, ascites, acoustic shadows, number of papillary structures, and the color score, the latter reflecting vascularization on Doppler ultrasound (1, no flow; 2, minimal flow; 3, moderate flow; 4, very strong flow). Detailed information can be found in previous reports. The 5 B-features and the 5 M-features were not directly recorded, but were derived from the variables described above.
Reference standard
The reference standard denotes whether the tumor is benign or malignant based on the histopathologic diagnosis of the tumor following surgical removal. Surgery was performed through laparoscopy or laparotomy, as considered appropriate by the surgeon. Excised tumor tissues were histologically examined at the local center. Histological classification was performed without knowledge of the ultrasound results. Borderline tumors were classified as malignant.
Statistical analysis
Using the IOTA data from phases 1b and 2, we estimated the risk of malignancy by quantifying the predictive value of each of the 10 features of the Simple Rules and of the type of center in which the patients underwent an ultrasound examination (oncology center vs other hospital; the definition of oncology center being tertiary referral center with a specific gynecological oncology unit). The predictive values for malignancy were determined by the regression coefficients estimated by multivariable logistic regression. Interaction terms were not considered. The analysis included a random intercept to account for variability between centers.
The risk estimates were externally validated on IOTA phase 3 data. The area under the receiver operating characteristic curve (AUC), sensitivity, specificity, and predictive values were calculated through a metaanalysis of center-specific results, similar to a previous validation study using phase 3 data. Positive likelihood ratio (LR+) and negative likelihood ratio (LR–) were derived from these results. The risk cutoffs considered to classify a mass as malignant were 1%, 3%, 5%, 10%, 15%, 20%, 25%, and 30%. Calibration plots were constructed to assess the relationship between calculated risks and observed proportions.
After external validation, the risk calculation was updated using the same procedure but now using all available data (phases 1b, 2, and 3) to fully exploit all available information.
Materials and Methods
Study design and setting
This international multicenter cross-sectional cohort study involves patients from 22 centers (oncology centers and other hospitals) ( Table 1 ) with at least 1 adnexal (ovarian, paraovarian, or tubal) tumor selected for surgery by the managing clinician. Exclusion criteria were: (1) pregnancy at the time of examination, (2) refusal of transvaginal ultrasonography, (3) declining participation, and (4) surgical intervention >120 days after the ultrasound examination. Data collection was carried out within the framework of the IOTA collaboration. The primary aim of the IOTA studies is to develop and validate methods for making a correct diagnosis in adnexal tumors prior to surgery. This aim is pursued by prospectively examining a large number of patients with ultrasound using a standardized examination technique and standardized terms and definitions to describe ultrasound findings. Through consecutive phases, data were collected from 24 centers in 10 countries. In phase 1 data were collected from 1999 through 2002, in phase 1b from 2002 through 2005, in phase 2 from 2005 through 2007, and in phase 3 from 2009 through 2012. Data from phase 1 were used to develop the Simple Rules and were therefore not used in the present study. The research protocols were approved by the ethics committees in each contributing center.
| Center | Data set | Patients | Malignant, N (%) | Classification using SR | ||
|---|---|---|---|---|---|---|
| SR benign, N (%mal) | SR inconclusive, N (%mal) | SR malignant, N (%mal) | ||||
| All oncology centers | 3263 | 1402 (43) | 1436 (5) | 788 (49) | 1039 (90) | |
| Leuven, Belgium | D,V | 668 | 242 (36) | 306 (4) | 153 (35) | 209 (85) |
| Rome, Italy | D,V | 661 | 365 (55) | 224 (7) | 163 (59) | 274 (92) |
| Monza, Italy | D,V | 356 | 76 (22) | 247 (4) | 69 (42) | 40 (95) |
| Prague, Czech Republic | D,V | 354 | 234 (66) | 102 (13) | 109 (77) | 143 (96) |
| Milan, Italy | D,V | 312 | 177 (57) | 112 (7) | 45 (56) | 155 (93) |
| Lublin, Poland | D,V | 285 | 102 (36) | 132 (5) | 86 (45) | 67 (85) |
| Bologna, Italy a | V | 213 | 65 (31) | 126 (3) | 52 (58) | 35 (89) |
| Stockholm, Sweden | V | 120 | 53 (44) | 38 (0) | 33 (27) | 49 (90) |
| Lund, Sweden | D,V | 77 | 20 (26) | 36 (0) | 20 (10) | 21 (86) |
| Beijing, China | D | 73 | 16 (22) | 36 (0) | 20 (15) | 17 (76) |
| London, United Kingdom | D | 65 | 25 (38) | 32 (6) | 18 (50) | 15 (93) |
| Udine, Italy | D,V | 64 | 19 (30) | 36 (3) | 16 (44) | 12 (92) |
| Naples 2, Italy | D,V | 15 | 8 (53) | 9 (22) | 4 (100) | 2 (100) |
| All other centers | 1585 | 263 (17) | 1021 (1) | 327 (23) | 237 (76) | |
| Malmö, Sweden | D,V | 462 | 100 (22) | 205 (0) | 146 (12) | 111 (74) |
| Genk, Belgium | D,V | 428 | 61 (14) | 301 (1) | 67 (21) | 60 (73) |
| Cagliari, Italy | D,V | 261 | 37 (14) | 200 (2) | 36 (33) | 25 (88) |
| Milan 2, Italy | D,V | 136 | 20 (15) | 99 (0) | 25 (40) | 12 (83) |
| Bologna, Italy a | D | 135 | 11 (8) | 110 (0) | 15 (27) | 10 (70) |
| Naples, Italy | D,V | 72 | 18 (25) | 42 (2) | 17 (35) | 13 (85) |
| Barcelona, Spain | V | 37 | 11 (30) | 21 (10) | 11 (55) | 5 (60) |
| Milan 3, Italy | D | 21 | 4 (19) | 13 (0) | 7 (43) | 1 (100) |
| Milan 4, Italy | V | 21 | 0 (0) | 20 (0) | 1 (0) | 0 (–) |
| Hamilton, Ontario, Canada | D | 12 | 1 (8) | 10 (0) | 2 (50) | 0 (–) |
a Bologna Center in Italy changed from other hospital to oncology center during course of International Ovarian Tumor Analysis study and is therefore listed in both categories (different patients in 2 types of centers).
Data collection
Oral and/or written informed consent was obtained in accordance with the requirements of the local ethics committee. A standardized history was taken from each patient to collect clinical information. All patients underwent a standardized transvaginal ultrasound examination by a principal investigator, who was a gynecologist or radiologist with extensive experience in gynecological ultrasound and with a special interest in adnexal masses. Transabdominal sonography was added in women with large masses that could not be visualized completely by the transvaginal approach. For women with multiple masses, the dominant mass was selected for statistical analysis. To apply the Simple Rules , information on the following variables is required: the diameters of the lesion (millimeters), the diameters of the largest solid component (millimeters), type of tumor (unilocular, unilocular-solid, multilocular, multilocular-solid, solid), presence of wall irregularity, ascites, acoustic shadows, number of papillary structures, and the color score, the latter reflecting vascularization on Doppler ultrasound (1, no flow; 2, minimal flow; 3, moderate flow; 4, very strong flow). Detailed information can be found in previous reports. The 5 B-features and the 5 M-features were not directly recorded, but were derived from the variables described above.
Reference standard
The reference standard denotes whether the tumor is benign or malignant based on the histopathologic diagnosis of the tumor following surgical removal. Surgery was performed through laparoscopy or laparotomy, as considered appropriate by the surgeon. Excised tumor tissues were histologically examined at the local center. Histological classification was performed without knowledge of the ultrasound results. Borderline tumors were classified as malignant.
Statistical analysis
Using the IOTA data from phases 1b and 2, we estimated the risk of malignancy by quantifying the predictive value of each of the 10 features of the Simple Rules and of the type of center in which the patients underwent an ultrasound examination (oncology center vs other hospital; the definition of oncology center being tertiary referral center with a specific gynecological oncology unit). The predictive values for malignancy were determined by the regression coefficients estimated by multivariable logistic regression. Interaction terms were not considered. The analysis included a random intercept to account for variability between centers.
The risk estimates were externally validated on IOTA phase 3 data. The area under the receiver operating characteristic curve (AUC), sensitivity, specificity, and predictive values were calculated through a metaanalysis of center-specific results, similar to a previous validation study using phase 3 data. Positive likelihood ratio (LR+) and negative likelihood ratio (LR–) were derived from these results. The risk cutoffs considered to classify a mass as malignant were 1%, 3%, 5%, 10%, 15%, 20%, 25%, and 30%. Calibration plots were constructed to assess the relationship between calculated risks and observed proportions.
After external validation, the risk calculation was updated using the same procedure but now using all available data (phases 1b, 2, and 3) to fully exploit all available information.
Results
During IOTA phases 1b, 2, and 3, data on 5020 patients were recorded at 22 centers (2 centers from IOTA phase 1 did not take part in later phases). Data on 172 patients were excluded because the patients fulfilled an exclusion criterion (n = 124; 43 women were pregnant and 81 women were operated on >120 days after the ultrasound examination), data errors or uncertain/missing final histology (n = 47), or protocol violation (n = 1). This leaves data on 4848 patients ( Tables 1 , 2 , and 3 ). The development set (phases 1b and 2) contains data on 2445 patients recruited at 11 oncology centers (n = 1548) and 8 other centers (n = 897). The temporal validation set (phase 3) contains data on 2403 patients recruited at 11 oncology centers (n = 1715) and 7 other centers (n = 688).
| Ultrasound feature | Development, n = 2445 | Validation, n = 2403 | ||
|---|---|---|---|---|
| Benign, n = 1760 | Malignant, n = 685 | Benign, n = 1423 | Malignant, n = 980 | |
| Maximum lesion diameter, mm | 61 (43–85) | 89 (58–136) | 64 (47–90) | 86 (55.5–126) |
| Solid components | ||||
| Presence of solid components | 541 (30.7%) | 638 (93.1%) | 474 (33.3%) | 916 (93.5%) |
| Maximum diameter if present, mm | 25 (13–47) | 54 (35–82) | 28 (13–54) | 59 (36.5–87) |
| No. of papillations | ||||
| None | 1538 (87.4%) | 427 (62.3%) | 1243 (87.4%) | 777 (79.3%) |
| 1 | 137 (7.8%) | 84 (12.3%) | 96 (6.8%) | 52 (5.3%) |
| 2 | 35 (2.0%) | 23 (3.4%) | 31 (2.2%) | 31 (3.2%) |
| 3 | 22 (1.3%) | 30 (4.4%) | 26 (1.8%) | 29 (3.0%) |
| >3 | 27 (1.5%) | 121 (17.7%) | 27 (1.9%) | 91 (9.3%) |
| Color score | ||||
| 1, No flow | 769 (43.7%) | 29 (4.2%) | 574 (40.3%) | 32 (3.3%) |
| 2, Minimal flow | 621 (35.3%) | 170 (24.8%) | 563 (40.0%) | 199 (20.3%) |
| 3, Moderate flow | 331 (18.8%) | 298 (43.5%) | 239 (16.8%) | 442 (45.1%) |
| 4, Very strong flow | 39 (2.2%) | 188 (27.5%) | 47 (3.3%) | 307 (31.3%) |
| Type of tumor | ||||
| Unilocular | 825 (47.0%) | 10 (1.5%) | 595 (41.8%) | 5 (0.5%) |
| Unilocular-solid | 187 (10.7%) | 112 (16.5%) | 141 (9.9%) | 117 (11.9%) |
| Multilocular | 390 (22.2%) | 37 (5.4%) | 354 (24.9%) | 59 (6.0%) |
| Multilocular-solid | 196 (11.2%) | 268 (39.1%) | 179 (12.6%) | 326 (33.3%) |
| Solid | 158 (9.0%) | 257 (37.5%) | 154 (10.8%) | 473 (48.3%) |
| Irregular cyst walls | 484 (27.5%) | 457 (66.7%) | 385 (27.1%) | 572 (58.4%) |
| Ultrasound features of Simple Rules | ||||
| B1, unilocular cyst | 825 (46.9%) | 10 (1.5%) | 595 (41.8%) | 5 (0.5%) |
| B2, solid components present, but <7 mm | 44 (2.5%) | 5 (0.7%) | 40 (2.8%) | 2 (0.2%) |
| B3, acoustic shadows | 307 (17.4%) | 29 (4.2%) | 265 (18.6%) | 34 (3.5%) |
| B4, smooth multilocular tumor, largest diameter <100 mm | 233 (13.2%) | 3 (0.4%) | 224 (15.7%) | 13 (1.3%) |
| B5, no blood flow; color score 1 | 769 (43.7%) | 29 (4.2%) | 574 (40.3%) | 32 (3.3%) |
| M1, irregular solid tumor | 12 (0.7%) | 97 (14.2%) | 16 (1.1%) | 189 (19.3%) |
| M2, ascites | 23 (1.3%) | 222 (32.4%) | 18 (1.3%) | 322 (32.9%) |
| M3, at least 4 papillary structures | 27 (1.5%) | 121 (17.7%) | 27 (1.9%) | 91 (9.3%) |
| M4, irregular multilocular-solid tumor, largest diameter ≥100 mm | 45 (2.6%) | 144 (21.0%) | 40 (2.8%) | 153 (15.6%) |
| M5, very strong flow; color score 4 | 39 (2.2%) | 188 (27.5%) | 47 (3.3%) | 307 (31.3%) |
Stay updated, free articles. Join our Telegram channel
Full access? Get Clinical Tree
