How do OSATS Fit into the RCOG Training Curriculum?

There are three types of WPBA used in O&G training:

Within the core curriculum, OSATS are expected to be carried out on a number of surgical procedures felt to be fundamental to the practice of O&G:

For all procedures, trainees are expected to gain competence during their first two years of basic training, and then to continue providing evidence of competence and progression onto more complex cases as they progress through their intermediate and advanced training years. A Matrix of Educational Progression provides guidance to trainees about how many of each procedure are expected to be completed each year [6] to demonstrate development and acquisition of competence and evidence of progression for their trainers and training programme leads. Although it is recognized that within a competency-based curriculum trainees will progress at different rates, the competency levels are the minimum that trainees must achieve before moving to the next stage of training.

In addition to the set requirements outlined above, a generic OSATS form exists to enable trainees to collate evidence of experience and competence for other surgical procedures which they undertake during their training, e.g. cervical cerclage, caesarean hysterectomy.

OSATS Requirements

In order for the OSATS process to be formalized further within the curriculum, several key requirements have been determined by the Specialty Education and Assessment Committee of the RCOG to guide trainees undertaking specialty training, and their trainers:

Undertaking an OSATS

When a trainee feels ready to undertake an OSATS, they will be expected to discuss and agree being assessed with a clinical supervisor, who will:

The trainee will need to declare whether they wish the assessment to be a formative or summative process before the start of the procedure in order that the clinical supervisor is able to determine their expectations for performance. Clinical supervisors can be any health professional who is able to carry out the procedure independently themselves. Typically, clinical supervisors will be consultants or trainees at a more senior level than the trainee being assessed, but can include midwives (for example, for assessment of perineal repair competence) and clinical nurse specialists (for some gynaecological procedures). Most trainees will have a wide number of clinical supervisors taking part in their assessment of competence during their training, with the overall assessment of competence during the year being overseen by their dedicated educational supervisor. On completion of the required number of summative OSATS needed to be ‘signed off’ for that competence in the logbook, the trainee can request that a clinical supervisor, or their educational supervisor, records the date that each OSATS is signed off.

Formative vs. Summative Assessment

OSATS can be carried out as either a formative or summative form of assessment:

Trainees are required to undertake both formative and summative OSATS in order to provide evidence of progression of skills acquisition. This model was reinforced in the RCOG curriculum update of August 2014 with the introduction of specific and separate formative and summative completion forms. Trainees are advised to take as many formative OSATS as they feel they need in order to feel sufficiently competent in a procedure before requesting a summative OSATS. Before competences can be signed off in the trainee logbook, each OSATS must be completed as a summative process, in full, on at least three separate occasions.

The Formative Assessment

Before starting the procedure, the trainee and assessor determine what the procedure is, establish the clinical details of the case and the level of complexity which is expected to exist. The assessor then observes the trainee undertaking the procedure, considering a number of technical and non-technical skills (not an exhaustive list) during the observation:

The assessor then provides specific, constructive verbal and written feedback to the trainee around their performance, focusing upon:

The trainee is expected to complete a reflection about their learning from the event in order to reinforce a process of reflective learning for future development. For formative assessments, there is no overall judgement on competence – this is assessed summatively once the trainee feels ready to be assessed for competence acquisition.

Written feedback is entered by the assessor into the trainee’s e-Portfolio, the online training portfolio onto which all training and assessment evidence is collated for trainees registered in a Specialty Training Programme. The trainee, their educational supervisor, college tutor and the regional training programme directors, will be able to view individual assessment forms and collation of evidence gathered on the e-Portfolio. This is used to assess the trainee against the Matrix of Training as he or she progresses through each year of training. For doctors who are not part of the Specialty Training Programme within a region, it is still recommended to register with the RCOG for online e-Portfolio access to enable coordinated collation of competence progression and acquisition for appraisal and revalidation purposes. For doctors who are not registered on e-Portfolio, the OSATS forms may be downloaded and inserted into a hard-copy portfolio.

The discussion between assessor and trainee about the formative OSATS procedure should take place as quickly as possible after the event, with written feedback being carried out as soon as is feasible, in order that the contemporaneous nature of the discussion is captured effectively.

The Summative Assessment

The summative OSATS is an assessment of performance (AoP) and is a mandatory, summative tool designed to:

There is a judgement to be made in each assessment relating to the overall performance observed: competent or working towards competence. Trainees require a minimum of three procedures deemed competent per core procedure, by more than one assessor, including a consultant or post-CCT holder. For an assessor, the judgement of competence which they make as part of a summative OSATS is specific to that assessment only.

As for formative assessments, the trainee should agree with the assessor beforehand that they are willing and able to carry out an assessment for the procedure, and should specify that the assessment is to be summative in nature. The clinical details and complexity of the procedure need to be detailed in the assessment form, and the assessor needs to indicate whether the procedure falls into the basic, intermediate or advanced section of the curriculum.

The summative OSATS template provides a number of anchor statements to guide assessors in their determination of competence for a particular procedure [7]:

The assessor is directed to consider technical and non-technical skills in determining their assessment of the trainee (using the same list as for formative OSATS). The assessor is then asked to specify which of the following statements apply in their assessment of the procedure completed:

or

As with the formative OSATS, the assessor is expected to provide verbal and subsequent written feedback to the trainee about what went well and what could have gone better, and a formulated learning plan. Even for the trainee who is determined to be competent at the procedure, or who is reaching the end of training, the concept of lifelong learning still applies, such that there should always be a learning plan for the future.