Development of the digital ECAS application

The ECAS application is a cross-platform digital application coded in HyperText Markup Language, Cascading Style Sheets, and JavaScript ( Fig. 1 ). It can be accessed through a desktop or mobile browser using the following link: https://medanalysis.app/uterine/study . We extracted the parameters required to determine the stage of endometrial cancer into five categories: histological type (12 parameters), tumor involvement (19 parameters), lymphovascular space invasion (3 parameters), lymph node status (10 parameters), and molecular status (4 parameters). The algorithm for stage determination in the application conformed to the standard provided in the original publication [ ]. The criteria for every stage was further confirmed by members of our study, which included three gynecologic oncologists, one obstetrician-gynecologist, and one pathologist specializing in gynecologic cancer.

The digital application, Endometrial Cancer Staging (ECAS), demonstrating the result of stage IA3 endometrial cancer with relevant selected parameters. It can be accessed via a desktop or mobile browser using the following link: https://medanalysis.app/uterine/study .

Inclusion and exclusion of evaluator

The study was conducted between February and May 2024 and included clinical practitioners from the Obstetrics and Gynecology Department of Taipei Veterans General Hospital. This study was approved by the Institutional Review Board (IRB) of Taipei Veterans General Hospital (IRB number 2024-01-026CC).

Evaluators were included to test the accuracy and efficiency of the digital applications. The evaluators included doctors with different levels of clinical experience, ranging from medical students (clerks and interns) to postgraduate year residents (PGY), gynecologic residents, fellows, and attending physicians. Evaluators with incomplete data, those who discontinued the test for any reason, and those who chose not to share their results were excluded. For the subgroup analysis, we stratified the evaluators into two groups: those without gynecologic specialties (the non-GYN group, which included medical students and PGY) and those with gynecologic specialties (the GYN group, which included gynecologic residents, fellows, and attending physicians).

Design of the evaluation test

To evaluate the accuracy and time required to determine the stage, we created an evaluation test consisting of two separate parts (parts I and II), each with ten pieces of pathology reports. The evaluator attempted to determine the stage of endometrial cancer according to pathology reports by utilizing the ECAS application in one part and a conventional lookup in the other part.

Pathology reports were selected from patients with endometrial cancer who underwent complete staging surgery at our hospital after June 2023 (publication of the updated staging system). We selected reports with sufficient information, including tumor involvement description, degree of LVSI, lymph node status, and staining conditions, as required by the 2023 system. Each part of the test consists of four cases of stage I disease, two cases of stage II disease, and four cases of stage III disease. Of all the parameters considered in the 2023 FIGO staging system, only the DNA polymerase epsilon ( POLE ) status required genetic testing. As a result, we have added a description at the end of the pathology report addressing the POLE mutation status to facilitate stage determination. The name, medical record number, date of surgery, requested serial number, and personal information were removed from the selected pathology report. Controversial conditions, such as concurrent POLE mutation and mismatch repair (MMR) deficiency, and conditions that were not clearly described in the original document, were not included. An example is provided in Supplementary Data 1 .

In each part of the test, three reports were selected to specifically assess the consideration of complex conditions: two for molecular classification parameters and one for the stage IA3 criteria. The molecular classification condition was defined as a stage modified by the identification of relevant molecular results, namely, stage I to II disease with POLE mutations or abnormal p53 expression.

Validity and reliability of the evaluation test

Content validity [ ] was used to evaluate the test. Pathology reports were reviewed by all members of this study, including two gynecologic oncologists, one pathologist specializing in gynecologic oncology, and two obstetrician-gynecologists, to establish a standard stage interpretation. The pathology report was based on the standard report format of the endometrial cancer protocol from the College of American Pathologists (v5.0.0.0, December 2023) [ ]. The level of difficulty between Part I and Part II of the test was managed to be as equal as possible. Each part of the test consisted of four stage I, two stage II, and four stage III diseases. Parallel form reliability [ ] was tested using Pearson’s correlation [ ] to evaluate the difference in difficulty between the two parts of the tool. We recruited another 12 evaluators from each of the six levels of clinical experience to test the difficulty of the evaluation test. The Pearson correlation coefficient reached 0.86, suggesting that the difficulty levels of parts I and II of the test were fairly equal.

Study design

This was a self-controlled study with a paired design. The evaluators were asked to complete two parts of the evaluation test consecutively using different tools: one with the ECAS application and the other with the conventional lookup method ( Fig. 2 ). The answers and time required for each question were automatically collected using a computer questionnaire program (AutoProctor, via https://www.autoproctor.co/ ). One research member supervised the entire testing process without providing any further instructions.

Counterbalancing study design: different order of tools to minimize the maturation effect that could lead to higher accuracy and reduced time in the latter test.

When using the ECAS application, the evaluators were provided with a web version of the application. They were briefed on the purpose of the application and asked to select the desired parameters. No further introduction to the updated endometrial cancer system or guidance on how to use the application was provided to the evaluators. When using the conventional lookup method, they were provided with an extracted version of the original 2023 FIGO staging article, which included two tables regarding the staging system, all footnotes, and a paragraph regarding the molecular classification. Subsequently, they were verbally instructed that all conditions mentioned in the pathology reports were included in the provided material.

The maturation effect may occur during the testing process, resulting in a higher accuracy in the second part of the test. To address this issue, we randomly assigned tools to address this bias. This was achieved using a random number table generated using the Excel RAND function. The evaluators were assigned to either utilize the ECAS application first or the conventional lookup method first. In each part of the evaluation test, ten pathology reports were displayed in a randomized order.

We evaluated two outcomes: the accuracy and time required to complete the stage determination. Accuracy was defined as the number of correct stage determinations divided by the total number of questions (which is ten for each part of the test). The standard answer for stage interpretation was determined by all members involved in our study. The time was recorded in seconds and presented as the total or mean of the results.

Statistical analysis

One-way ANOVA was used to compare the time and accuracy between the two tools (ECAS application and the conventional lookup method). Continuous variables are presented as mean, standard deviation (SD), and range. Categorical variables were analyzed using the chi-square test and are presented as numbers (n) and percentages. The level of significance was set at p < 0.05. All statistical analyses were conducted using SPSS software (version 23.0; IBM Corp., Somers, NY, USA). Data visualization was performed using the yED Graph Editor version 3.23.2 (yWorks GmbH, Tübingen, Germany) and GraphPad Prism (version 9.0; GraphPad Software, San Diego, California, USA). Descriptive statistical analyses were then performed.