Objective Structured Assessment of Technical Skills (OSATS) in Obstetrics

Figure 32.1

Different stages of the RCOG Specialty Training Programme in obstetrics and gynaecology.


Reproduced with permission from the RCOG.


Basic training incorporates the first two years of specialty training, and requires the trainee to develop and demonstrate, among other targets of training, technical competence in a number of surgical procedures, before being able to progress to the next stage of training. Trainees will usually spend a year in two different training units during basic training, within a defined region of training, and once competency is achieved for these key surgical procedures, would be expected to demonstrate continued competency alongside development of more complex technical and clinical skills as their training progresses across other units in the region. OSATS are one of a number of workplace-based assessment (WPBA) formats used to evaluate progression throughout specialty training, aiming to link teaching, learning and assessment in a structured way.




How do OSATS Fit into the RCOG Training Curriculum?


There are three types of WPBA used in O&G training:




1. OSATS (objective structured assessment of technical skills);



2. Mini-CEX (mini-clinical evaluation exercise); and



3. CBD (case-based discussion).


Within the core curriculum, OSATS are expected to be carried out on a number of surgical procedures felt to be fundamental to the practice of O&G:




  • diagnostic laparoscopy;



  • diagnostic hysteroscopy;



  • fetal blood sampling;



  • manual removal of placenta;



  • opening and closing the abdomen;



  • operative laparoscopy;



  • operative vaginal delivery;



  • caesarean section;



  • perineal repair; and



  • uterine evacuation.


For all procedures, trainees are expected to gain competence during their first two years of basic training, and then to continue providing evidence of competence and progression onto more complex cases as they progress through their intermediate and advanced training years. A Matrix of Educational Progression provides guidance to trainees about how many of each procedure are expected to be completed each year [6] to demonstrate development and acquisition of competence and evidence of progression for their trainers and training programme leads. Although it is recognized that within a competency-based curriculum trainees will progress at different rates, the competency levels are the minimum that trainees must achieve before moving to the next stage of training.


In addition to the set requirements outlined above, a generic OSATS form exists to enable trainees to collate evidence of experience and competence for other surgical procedures which they undertake during their training, e.g. cervical cerclage, caesarean hysterectomy.



OSATS Requirements


In order for the OSATS process to be formalized further within the curriculum, several key requirements have been determined by the Specialty Education and Assessment Committee of the RCOG to guide trainees undertaking specialty training, and their trainers:




  • The trainee must declare in advance whether an OSATS is summative or formative and there must always be a distinction between the two.



  • At least two different assessors must be used for each type of OSATS procedure. The same assessor must not be used for all OSATS and a consultant must do at least one of the assessments for any one procedure.



  • The trainee must retain all OSATS forms in their e-Portfolio, whether they were completed satisfactorily or not in order to enable their educational supervisor to review progress effectively.



  • Before a competence can be signed off as fully competent for independent practice in the logbook, each OSATS must have been successfully completed (i.e. every box ticked for independent practice) on at least three separate occasions.



  • Once the trainee is signed off as fully competent for independent practice for a particular skill, they should undergo an annual OSATS assessment (one per procedure) to demonstrate continued competence until they achieve their Certification of Completion of Training (CCT).



  • Trainees must also keep count of the number of each procedure they perform annually until they achieve CCT.



Undertaking an OSATS


When a trainee feels ready to undertake an OSATS, they will be expected to discuss and agree being assessed with a clinical supervisor, who will:




  • assess the procedure; and



  • complete the OSATS form on the e-Portfolio.


The trainee will need to declare whether they wish the assessment to be a formative or summative process before the start of the procedure in order that the clinical supervisor is able to determine their expectations for performance. Clinical supervisors can be any health professional who is able to carry out the procedure independently themselves. Typically, clinical supervisors will be consultants or trainees at a more senior level than the trainee being assessed, but can include midwives (for example, for assessment of perineal repair competence) and clinical nurse specialists (for some gynaecological procedures). Most trainees will have a wide number of clinical supervisors taking part in their assessment of competence during their training, with the overall assessment of competence during the year being overseen by their dedicated educational supervisor. On completion of the required number of summative OSATS needed to be signed off for that competence in the logbook, the trainee can request that a clinical supervisor, or their educational supervisor, records the date that each OSATS is signed off.



Formative vs. Summative Assessment


OSATS can be carried out as either a formative or summative form of assessment:




  • Formative (assessments for learning) are used as an opportunity to practise a skill and to provide feedback for further development of skills.



  • Summative (assessments of learning) are used to enable the trainee to demonstrate competence in a given clinical situation.


Trainees are required to undertake both formative and summative OSATS in order to provide evidence of progression of skills acquisition. This model was reinforced in the RCOG curriculum update of August 2014 with the introduction of specific and separate formative and summative completion forms. Trainees are advised to take as many formative OSATS as they feel they need in order to feel sufficiently competent in a procedure before requesting a summative OSATS. Before competences can be signed off in the trainee logbook, each OSATS must be completed as a summative process, in full, on at least three separate occasions.



The Formative Assessment


Before starting the procedure, the trainee and assessor determine what the procedure is, establish the clinical details of the case and the level of complexity which is expected to exist. The assessor then observes the trainee undertaking the procedure, considering a number of technical and non-technical skills (not an exhaustive list) during the observation:




  • checking of equipment/environment;



  • communication with patients and/or relatives;



  • perioperative planning, e.g. positioning;



  • use of assistants;



  • technical ability;



  • communication with staff;



  • selection of instruments and equipment;



  • forward planning;



  • economy of movement;



  • dealing with problems and/or difficulties;



  • tissue handling;



  • documentation;



  • completion of task as appropriate; and



  • safety considerations.


The assessor then provides specific, constructive verbal and written feedback to the trainee around their performance, focusing upon:




  • what went well;



  • what could have gone better; and



  • a learning plan for the future.


The trainee is expected to complete a reflection about their learning from the event in order to reinforce a process of reflective learning for future development. For formative assessments, there is no overall judgement on competence this is assessed summatively once the trainee feels ready to be assessed for competence acquisition.


Written feedback is entered by the assessor into the trainee’s e-Portfolio, the online training portfolio onto which all training and assessment evidence is collated for trainees registered in a Specialty Training Programme. The trainee, their educational supervisor, college tutor and the regional training programme directors, will be able to view individual assessment forms and collation of evidence gathered on the e-Portfolio. This is used to assess the trainee against the Matrix of Training as he or she progresses through each year of training. For doctors who are not part of the Specialty Training Programme within a region, it is still recommended to register with the RCOG for online e-Portfolio access to enable coordinated collation of competence progression and acquisition for appraisal and revalidation purposes. For doctors who are not registered on e-Portfolio, the OSATS forms may be downloaded and inserted into a hard-copy portfolio.


The discussion between assessor and trainee about the formative OSATS procedure should take place as quickly as possible after the event, with written feedback being carried out as soon as is feasible, in order that the contemporaneous nature of the discussion is captured effectively.



The Summative Assessment


The summative OSATS is an assessment of performance (AoP) and is a mandatory, summative tool designed to:




1. enable judgement of surgical competency in the procedure being assessed; and



2. provide specific, constructive feedback to the trainee about their performance.


There is a judgement to be made in each assessment relating to the overall performance observed: competent or working towards competence. Trainees require a minimum of three procedures deemed competent per core procedure, by more than one assessor, including a consultant or post-CCT holder. For an assessor, the judgement of competence which they make as part of a summative OSATS is specific to that assessment only.


As for formative assessments, the trainee should agree with the assessor beforehand that they are willing and able to carry out an assessment for the procedure, and should specify that the assessment is to be summative in nature. The clinical details and complexity of the procedure need to be detailed in the assessment form, and the assessor needs to indicate whether the procedure falls into the basic, intermediate or advanced section of the curriculum.


The summative OSATS template provides a number of anchor statements to guide assessors in their determination of competence for a particular procedure [7]:




  • For the trainee considered competent in the observed procedure it would generally be expected that the trainee was able to perform all aspects of the procedure safely and competently with no or minimal need for help, or in the context of an unexpectedly difficult case, may have needed more assistance for the more difficult aspects of the procedure.



  • For the trainee considered to be working towards competence it would generally be expected that the trainee required significant help throughout or with the majority of steps, or that the trainee was unable to perform any of the necessary procedures to be safe and competent at this stage.


The assessor is directed to consider technical and non-technical skills in determining their assessment of the trainee (using the same list as for formative OSATS). The assessor is then asked to specify which of the following statements apply in their assessment of the procedure completed:




  • This trainee performed this observed procedure competently;


or




  • This trainee is working towards competence in this procedure.


As with the formative OSATS, the assessor is expected to provide verbal and subsequent written feedback to the trainee about what went well and what could have gone better, and a formulated learning plan. Even for the trainee who is determined to be competent at the procedure, or who is reaching the end of training, the concept of lifelong learning still applies, such that there should always be a learning plan for the future.

Only gold members can continue reading. Log In or Register to continue

Stay updated, free articles. Join our Telegram channel

Jan 31, 2017 | Posted by in OBSTETRICS | Comments Off on Objective Structured Assessment of Technical Skills (OSATS) in Obstetrics

Full access? Get Clinical Tree

Get Clinical Tree app for offline access