Simulation-based training and assessment of mastoidectomy —perspectives on the outside, inside, and in-between conditions of practice

Mastoidectomy is a complex surgical procedure that involves drilling of the temporal bone of the skull to gain access to the middle ear and surrounding structures. Hands-on training through traditional cadaver dissection is increasingly difficult to provide given the diminishing number of donated human temporal bones and the high number of trainees. Modern surgical education also requires evidence-based approaches to skills training and competency assessment. Altogether, this has propelled the development of virtual reality (VR) simulation for surgical skills training including in temporal bone surgery.

It is well-established that VR temporal bone surgical simulation is highly useful for the training of novices. However, the performance of trainees seems to plateau early and at an insufficient level under self-directed training conditions. We therefore need to better understand what elements make self-directed simulation-based training work in order to design efficient training programs and create high-quality learning experiences. In this thesis, different learning conditions based on contemporary medical educational frameworks were studied because what we do outside, inside and in-between simulation matters.

The principle of mastery learning requires repeated practice until the level of proficiency. This motived us to develop a metrics-based performance assessment for automated assessment and to define pass/fail standards of proficiency using the expert performance framework. Nevertheless, the metrics-based score failed to capture key indicators of a safe performance compared with the established but manual and time-consuming final-product assessment. Next, generalizability theory was used to explore the reliability of the established final-product assessment under different training conditions: Contextual variables such as simulation modality and fidelity considerably affected reliability, cautioning the use of established assessment tools under conditions and contexts different from their original setting without specific considerations on reliability.

Distribution of practice over time is in the motor skills literature recognized to be superior to massed practice for skills acquisition and we have previously corroborated this for VR simulation training of temporal bone surgery. In this thesis, the effect of supplemental distributed VR simulation training on transfer of skills to the cadaveric dissection simulation training modality was therefore investigated: five training blocks of three simulated procedures improved subsequent cadaveric dissection performance by 25 %.

Cadaveric dissection represents a more complex learning environment and learning task than the VR simulation condition and further we found that the learning conditions of cadaveric dissection induces a significantly higher cognitive load in trainees. According to cognitive load theory, a cognitive load that exceeds the capacity of the learner can negatively affect learning. Therefore, the effects of repeated practice on cognitive load were explored and in contrast to massed practice, distributed practice significantly reduced cognitive load.

Altogether, these findings have the implication for the temporal bone surgical curriculum that training should be organized with structured and distributed VR simulation first to optimize the subsequent use of the limited and costly human temporal bones for dissection training after basic skills have been acquired.

VR simulation training allows distributed and repeated practice at the individual trainee’s convenience but without instructor presence, other learning supports for feedback are needed to ensure a successful learning experience. The concept of directed, self-regulated learning emphasizes the importance of providing the learner with direction and guidance to support and scaffold training.

Simulator-integrated tutoring by green-lighting is an innovative approach to dealing with this challenge. However, the use of simulator-integrated tutoring in VR temporal bone surgical simulation was previously found to lead to tutoring over-reliance, causing a poor performance once tutoring was discontinued. Distributed practice was found to have a moderately protective effect against this phenomenon and we therefore hypothesized that intermittent simulator-integrated tutoring would be a better strategy. Consequently, the effect of intermittent tutoring in a distributed training program was studied and it was found that tutoring increased performance while active, but resulted in an inferior performance in subsequent non-tutored sessions compared with a never-tutored reference cohort. This tutor over-reliance degrades motor skills learning and concurrent feedback through simulator-integrated tutoring should be reconsidered.

We next explored increasing fidelity of the VR simulation to better bridge the gap between simulation-based training and real-life conditions where an operating microscope is used to magnify the surgical field and to enable visualization of minute visual cues such as the vasculature of underlying anatomical structures. However, the improvement in resemblance and functional task alignment of introducing the eyepiece from a digital operating microscope did not benefit learners: compared with learners who were randomized for the conventional screen-based VR simulation condition, the learners in the “ultra-high fidelity” condition performed significantly poorer and their CL was higher. Consequently, improving instructional design and other learning supports should be considered over increasing realism of simulation.

Finally, we used an automated pipeline for segmentation of clinical CBCT imaging of the temporal bone to create patient-specific VR simulation that can be used for surgical rehearsal and planning ahead of actual surgery. Clinicians rated the patient-specific simulation highly, found that it contributed to a better understanding of the patient’s anatomy, and perceived it to be of benefit to training of surgeons at both the resident and fellow level. Nonetheless, a major limitation was the quality of the clinical scans, which were often limited by field-of-view and poor scan quality due to motion artefacts.

Altogether, the work presented in this thesis provides insights into the outside, inside and in-between conditions of VR simulation training in temporal bone surgery—with broader implications for simulation-based surgical skills training in general. The optimal VR simulation training program consists of structured and distributed practice, supporting directed, self-regulated learning. The role of addressing the cognitive process, motivating the trainee, and providing proper direction and feedback cannot be stressed enough.

Future research directions include developing an adaptive training program that tailors feedback and case difficulty based on valid and reliable automated assessment, better integration into the clinical training curriculum, and advancing simulation for training beyond the novice level so it becomes a useful tool even for patient-specific rehearsal and surgical planning and navigation.