Background
Evaluation of trainee’s ability in obstetrical ultrasound is a time-consuming process, which requires involving patients as volunteers. With the use of obstetrical ultrasound simulators, virtual reality could help in assessing competency and evaluating trainees in this field.
Objective
The objective of the study was to test the validity of an obstetrical ultrasound simulator as a tool for evaluating trainees following structured training by comparing scores obtained on obstetrical ultrasound simulator with those obtained on volunteers and by assessing correlations between scores of images and of dexterity given by 2 blinded examiners.
Study Design
Trainees, taking the 2013 French national examination for the practice of obstetrical ultrasound were asked to obtain standardized ultrasound planes both on volunteer pregnant women and on an obstetrical ultrasound simulator. These planes included measurements of biparietal diameter, abdominal circumference, and femur length as well as reference planes for cardiac 4-chamber and outflow tracts, kidneys, stomach/diaphragm, spine, and face. Images were stored and evaluated subsequently by 2 national examiners who scored each picture according to previously established quality criteria. Dexterity was also evaluated and subjectively scored between 0 and 10. The Raghunathan’s modification of Pearson, Filon’s z, Spearman’s rank correlation, and analysis of variance tests were used to assess correlations between the scores by the 2 examiners and scores of dexterity and also to compare the final scores between the 2 different methods.
Results
We evaluated 29 trainees. The mean dexterity scores in simulation (6.5 ± 2.0) and real examination (5.9 ± 2.3) were comparable ( P = .31). Scores with an obstetrical ultrasound simulator were significantly higher than those obtained on volunteers ( P = .027). Nevertheless, there was a good correlation between the scores of the 2 examiners judging on simulation ( R = 0.888) and on volunteers ( R = 0.873) ( P = .81).
Conclusion
An obstetrical ultrasound simulator is as good a method as volunteer-based examination for evaluating practical skills in trainees following structured training in obstetrical ultrasound. The threshold for success/failure should, however, be adapted as candidates obtain higher scores on the simulator. Advantages of the obstetrical ultrasound simulator include the absence of location and time constraints without the need to involve volunteers or to interfere with the running of ultrasound clinics. However, an obstetrical ultrasound simulator still lacks the ability to evaluate the trainees’ ability to interact with patients.
Ultrasonography is the most used imaging tool in prenatal assessment. Although it offers an efficient screening strategy, it still has limitations in detecting congenital anomalies. It is dependent on the operator’s technical skills, its practice is standardized, and certification of competency is often required. Evaluation of trainee’s ability is a time-consuming process, which requires involving patients as volunteers.
Simulation has spread out from education in aviation and military industry to health care over the last 40 years. It is being used as a method to train but also to assess learners. Multiple factors have contributed to this expansion including limited patient availability, increasing concern for patients’ safety and the need to minimize medical errors. This development has also been imposed by technological advances in diagnosis and treatment, which require new skills and increased training hours.
With the advent of obstetrical ultrasound simulators (OUS), training and assessing competency in this field may also evolve rapidly. Virtual reality could help solve some of the problems aforementioned by providing a virtual patient and a virtual setting in which the ultrasound examination could take place without interfering with the normal activities and settings of medical units.
The aim of our study was to assess the potential for an obstetrical ultrasound simulator to be used as a substitute for the use of volunteers to evaluate trainees’ ability to perform obstetrical ultrasound (US).
Materials and Methods
Candidate sonologists to the Annual French National Examination of Ultrasound in Obstetrics and Gynecology were recruited at Paris Descartes University in September 2013.
In France, obstetrical sonologists are obstetricians, radiologists, or midwives. They are required to attend more than 100 hours of theoretical teaching and a minimum of 40 3 hour sessions under supervision within a clinical setting of ultrasound in obstetrics and gynecology over 1 year. They are then subjected to a national written examination, and those who score at least 50% are then required to sit for a practical evaluation.
The latter consists of performing a second-trimester ultrasound examination including the following: identification of fetal presentation, placental position, and amniotic fluid index in addition to completing a minimum of 6 of the 9 planes and measurements requested by the national committee for prenatal ultrasound as part of the second-trimester ultrasound examination. ( Figure 1 ).
Approximately 150 trainees are evaluated at Paris Descartes University every year, which requires canceling a whole day in the ultrasound department and occupying 6 machines and 12 doctors and midwives to run the evaluation process. In addition, voluntary participation of 50 and 60 pregnant women who have previously undergone a detailed ultrasound examination that was found to be normal and who consented to participate to this pedagogic endeavor are required.
The study consisted of randomly selecting 1 of 5 candidates prior to performing the formal examination on a volunteer as per previous years. The 6 original 2-dimensional ultrasound biometric and morphological images assigned and obtained live by each candidate were printed and stored for later evaluation. In the absence of a standard method for evaluating dexterity, the examiner was asked to subjectively score each student’s dexterity on a scale from 1 to 10.
In addition, these candidates were also asked to perform a similar session on the obstetrical ultrasound simulator Vimedix (CAE Healthcare, Sarasota, FL) ( Figure 2 ).
This device features a 20-week fetus and aims at mimicking fetal and maternal anatomy. It consists of a pregnant mannequin, a computer with a keyboard, mouse, and monitor as well as a transducer ( Figure 3 ). It provides a realistic reconstruction of the fetal anatomy. It can present 3-dimensional anatomical structures next to the generated 2-dimensional ultrasound images using a feature called augmented reality. The fetus and its placenta can be given various positions. The system allows the trainee to adjust the gain, magnification, and brightness as well as to save and store the images obtained during each session. It is also possible to measure fetal biometrics but the device lacks Doppler simulation.
During the virtual examination, the trainees were given 10 minutes to familiarize themselves with the device, and they were then asked to conduct a detailed examination while another examiner evaluated them. The augmented reality option was turned off. They were asked to obtain the same six US planes as those obtained during the live examination. These images were also digitally stored for subsequent evaluation. At the end of the virtual examination and in the absence of a standard protocol for evaluating dexterity, the examiner scored the candidate on a scale of 1 to 10 on his dexterity while performing the virtual US scan.
We used the same method for scoring the images obtained by the candidates during the 2 types of examinations. Each image was assigned a score according to the objective quality criteria already established and validated. One point was given for each criterion met, for a total score of 0 to 4, 5, or 6, depending on the number of criteria required for each plane ( Tables 1 and 2 ). Then, with the purpose of uniformity and simplicity, each score was converted to a score of between 0 and 10. The sum of scores for the 6 images for each type of examination gave the final score greater than 60.
| Criteria | 4 chambers | Outflow tracts | Kidneys | Stomach/diaphragm | Spine | Face |
|---|---|---|---|---|---|---|
| 1 | 4 chambers visible | Pulmonary artery bifurcation visible | Circular view of the first kidney | Heart visible | Dorsal spine visible | Upper lip visible |
| 2 | Apex of the heart visible | Ascending aorta visible | Circular view of the second kidney | Stomach visible | Sacrum visible | Two nostrils visible |
| 3 | Heart crux visible | Right ventricle visible | Posterior kidney clear from the spine acoustic shadow | Spine nonvisible | Alignment of the vertebrae visible from the dorsal level to the sacrum | Two lip angles visible |
| 4 | One pulmonary vein visible | Pulmonary artery curling up the aorta | Corticomedullary differentiation of the pyelic cavity visible | Diaphragmatic interface visible from back to front | Amniotic fluid visible beyond the skin | — |
| 5 | Descending thoracic aorta visible | — | — | Thigh and neck visible | — | — |
| 6 | ROI occupying more than half of the total image size | ROI occupying more than half of the total image size | ROI occupying more than half of the total image size | ROI occupying more than half of the total image size | ROI occupying more than half of the total image size | ROI occupying more than half of the total image size |
| Criteria | Type of image | ||
|---|---|---|---|
| Cephalic | Abdominal | Femoral | |
| 1 | Symmetrical plane | Symmetrical plane | Both ends of the bone clearly visible |
| 2 | Plane showing the thalami | Plane showing the stomach bubble | < 45° angle to the horizontal |
| 3 | Plane showing the cavum septi pellucidi | Plane showing the portal sinus | Femoral plane occupying more than half of the image size |
| 4 | Cerebellum not visible | Kidneys not visible | Calipers placed correctly |
| 5 | Head plane occupying more than half of the image size | Abdominal plane occupying more than half of the image size | — |
| 6 | Calipers and dotted ellipse placed correctly | Calipers and dotted ellipse placed correctly | — |
Stay updated, free articles. Join our Telegram channel
Full access? Get Clinical Tree
