Posters

The validity status of summative and formative assessment tools for technical and nontechnical skills in ophthalmic surgery – a systematic review

Poster Details

First Author: T.Wood UK

Co Author(s):    S. Maqsood   M. Nanavaty   S. Rajak              

Abstract Details

Purpose:

The skills that ophthalmic surgery trainees must develop are intricate and challenging; the microsurgical techniques required for core procedures usually involve steep learning curves. Applicable, valid and reliable assessment tools are therefore required in order for ophthalmic surgery trainees to demonstrate their technical and nontechnical skills. The objective of this systematic review is to outline, evaluate and recommend the assessment tools specific to ophthalmic surgery, with reference to their validity and reliability statuses.

Setting:

This systematic review was undertaken at Sussex Eye Hospital (Brighton and Sussex University Hospitals NHS Trust, Brighton, East Sussex, UK).

Methods:

This systematic review was conducted in accordance with the guidelines outlined in the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement. A systematic and comprehensive literature search was performed on Pubmed, ScienceDirect and Cochrane Library between 20th December 2019 and 25th January 2020. Data extraction occurred in line with specific eligibility criteria, including the assessment tool used, simulated or live settings, participant numbers and training levels, and statistical methods. The validity and reliability outcomes for each tool was evaluated in accordance with pre-set definitions.

Results:

26 articles were included in this review. 18 assessment tools for technical skills and 4 assessment tools for nontechnical skills were identified. The ‘International Council of Ophthalmology’s Ophthalmology Surgical Competency Assessment Rubrics’ (ICO-OSCARs) and ‘Objective Structured Assessment of Cataract Surgical Skill’ (OSACSS) are the most extensively analysed assessment tools for technical skills. Observational Teamwork Assessment for Surgery (OTAS), Non-Technical Skills for Surgeons (NOTSS) and Anaesthetists Non-Technical Skills (ANTS) were the assessment tools for nontechnical skills which demonstrated content and concurrent validity, as well as internal consistency and interrater reliability in ophthalmic surgery settings.

Conclusions:

Multiple assessment tools for technical and nontechnical skills are valid for use in ophthalmic surgery, however their validity and reliability statuses have been analysed to different extents. Furthermore, there is a paucity of assessment tools for nontechnical skills compared to those for technical skills. Tools exist to satisfy the training requirements for multiple domains of ophthalmic surgery, however further research is required to validate them all to consistent levels. Ultimately, none of the assessment tools satisfied all measures of validity and reliability. Limitations included infrequent quantification of face and content validity, and inconsistency in terminology and statistical methods between studies.

Financial Disclosure:

None