Free Papers

Comparison of electronic vs conventional assessment methods in ophthalmology residents

Free Paper Details

First Author: H.Hasani IRAN

Co Author(s):    A. Khavandegar   S. Shamsoddinimotlagh                 

Abstract Details

Purpose:

To compare electronic and conventional methods to assess the ophthalmology residents in different levels of residency training program

Setting:

Rassoul Akram Hospital, Iran University of Medical Sciences

Methods:

We implemented conventional assessment methods consisting of eight different part 3 weeks prior to electronic examination; residents’ attendance, Log book, scholarship and research skills, journal club and outpatient department participation, MCQ, OSCE, 360 degrees and finally professionalism (embedded with 360-degree). For electronic implementation, training workshops, trial mandatory electronic and eventually, final electronic examination were held. Electronic assessment consisted of e-PMP and modified electronic OSCE.

Results:

Totally 24 ophthalmology residents consisting of 9 females and 15 males participated all stages of conventional exam and involved in the electronic assessment three weeks later. 33.3% of participants were first-year residents, 16.7% second year, 25% third-year and 25% fourth-year residents. In electronic part assessment, average e-PMP scores (48.01±12.40) was much lesser compared with me-OSCE (65.34±17.11). In conventional assessment method, the highest average score allocated to residents’ attendance (99.79±1.02), while the least acquired average score was in OSCE (52.79±12.04). Other assessment tools lied in between: professionalism/360 degree (91.70±4.14), Logbook (91.66±10.07), journal club and OPD participation (90.00±11.32), research skills (79.16±13.80) and finally MCQ (with average score of 60.10±13.57). Total average electronic score was 56.67±11.28 while total average conventional score was 80.74±5.99. Female and male residents’ average score in electronic and conventional was (59.15±12.32 versus 83.01±4.95) and (55.19±10.77 versus 79.38±6.29), respectively. Total average score in first, second, third and fourth-year residents was (50.91±8.94 versus 76.15± 2.83), (58.12± 5.34 versus 83.14± 1.86), (67.08± 6.81 versus 81.00±7.87) and (53.00± 14.46 versus 85.00± 5.44) in electronic and conventional assessment, respectively. Correlation between modified electronic OSCE and conventional methods was insignificant. Correlation between e-PMP and five conventional methods including professionalism/360-degree assessment tool was significant. Female residents’ average score was higher in each part of conventional, electronic and total scores. Finally, overall correlation between conventional and electronic methods was significant.

Conclusions:

Electronic and conventional methods are correlated significantly in evaluating the ophthalmology residents, implying the point that electronic assessment method could be a suitable complementary to currently used conventional methods. Combined electronic PMP and me-OSCE considered to be a suitable substitution for contemporary used gold-standard assessment methods.

Financial Disclosure:

-