Article Text
PDF
Abstract
Background The quality of ophthalmic surgical training is increasingly challenged by an untimely convergence of several factors. This article reviews the tools currently available for training and assessment in phacoemulsification surgery.
Methods Medline searches were performed to identify articles with combinations of the following words: phacoemulsification, training, curriculum, virtual reality and assessment. Further articles were obtained by manually searching the reference lists of identified papers.
Results Thus far phacoemulsification training outside the operating room include wet labs and micro-surgical skills courses. These methods have been criticised for being unrealistic, inaccurate and inconsistent. Virtual reality simulators have the ability to teach phacoemulsification psychomotor skills, as well as to carry out objective assessment. Other ophthalmic surgical skill assessment tools such as Objective Assessment of Skills in Intraocular Surgery (OASIS) and Global Rating Assessment of Skills in Intraocular Surgery (GRASIS) are emerging. Assessor bias is minimised by using video-based assessments, which have been shown to reduce subjectivity. Dexterity analysis technology such as the Imperial College Surgical Assessment Device (ICSAD) and virtual reality simulators can be used as objective assessment devices.
Conclusion Improvements in technology can be utilised in ophthalmology and will help to address the increasingly limited opportunities for training and assessment during training and throughout a subsequent career (re-training and re-validation). This will inevitably translate into enhanced patient care.
- Assessment
- lens and zonules
- ophthalmic surgical training
- skill acquisition
- virtual reality, wet lab
Statistics from Altmetric.com
Introduction
Cataract extraction by phacoemulsification was pioneered by Charles David Kelman1 in New York in 1967. Over the next 25 years it replaced the traditional inpatient cataract surgery with a much less intrusive and more curative procedure that could be performed on an outpatient basis. According to Hospital Episode Statistics, more than 280 000 phacoemulsifications were carried out in the financial year 2005–6 in the UK alone.2 It has become by far the key index operation for ophthalmologists, with the Royal College guidelines expecting trainees to complete 350 phacoemulsification cataract procedures by the end of Ophthalmic Specialist Training year 7.3
Challenges in phacoemulsification training
The quality of ophthalmic surgical training is increasingly challenged by an untimely convergence of several factors. The public's unprecedented high expectations and demands for safety are fuelled by readily accessible information on hospital performances such as the Electronic Cataract National Dataset.4 In addition, consultant-led care provided by Independent Treatment Centres and the further reduction of working hours (to 48 h a week by 2009), imposed by the European Working Time Directive, are leading to concerning reports5 6 about the reduced overall training opportunities.
The training of novice phacoemulsification surgeons, as with all postgraduate medical training, has altered substantially in the last decade.7 Although the Halstedian apprentice model of teaching has served ophthalmology education for more than 100 years, it fails to ensure that experience is gained in all the essential areas and is therefore inappropriate as well as stressful. The limitations8 of this approach are hard to ignore, and even more difficult to amend.
Increasingly, the demonstration of competency is gathering general consensus as the preferred method to drive ophthalmic trainees along the surgical learning curve. Benjamin9 reports that team-working, leadership, insight, dexterity, decision-making, prioritising and empathy are all important and should be assessed.
Moreover, the almost 10-year-old accepted standard for complication rates as determined by the 1997–8 National Cataract Surgery Survey is out-of-date. This reflects the great advances in the practice and provision of modern phacoemulsification surgery achieved since then.10 It also suggests that although training opportunities are being compromised, the required expected standards are higher.
The challenge for the current generation of ophthalmologists is to increase the quality of training and indeed maintain standards. It is necessary to utilise training methods outside the operating theatre (wet labs, simulation) while simultaneously demonstrating competence during formal assessments.
Phacoemulsification training outside the operating theatre
Wet labs use cadaveric human or animal models, or synthetic eyes (designed specifically for performing phacoemulsification) to rehearse the steps of cataract extraction. Laboratory practice allows surgeons to acquire skills in a controlled environment, free of the pressures of operating on real patients according to Piaget's and Vygotsky's pedagogical philosophy of ‘learning by doing’. This has been proven to be effective in training general surgeons.11 12
The mandatory introduction of micro-surgical skills courses in the UK was an attempt to teach basic micro-surgical skills within a structured curriculum. This is not an entirely new concept in ophthalmology. Kirby13 reported a resident course in basic ophthalmic surgery techniques that had been in use since 1966. The success of this form of training is now evident from the large number of micro-surgical skills courses available worldwide. Courses last 2–3 days and consist of didactic lectures with some hands-on wet lab simulator practice. Skills practice varies from performing single tasks such as knot-tying to learning complex procedures such as lid and corneal wound repair, capsulorrhexis and phacoemulsification.
However, these methods have been criticised for being unrealistic14 with inaccurate simulation of tissue consistency and anatomy8 and also lacking any form of objective assessment. These problems can be addressed through the development of computer simulators—similar technology has been used to train airline pilots for a number of years, providing realistic simulation with an accurate assessment of performance. It has already been emulated successfully in other branches of surgery.15–18 The possibility of an analogous situation in ophthalmology is becoming a reality.
Virtual reality systems in phacoemulsification
The term virtual reality refers to a ‘computer-generated representation of an environment that allows sensory interaction, thus giving the impression of actually being present’.19 Over the past two decades researchers have developed many ophthalmic virtual reality surgery simulators.
The training systems described in the published literature come from the USA, Japan, France, Sweden, Austria and Germany. One of the first projects to embrace this technology for ophthalmic surgical education was the Ophthalmic Retrobulbar Injection Simulator,20 while Sinclair et al21 were the first to develop haptic (tactile) feedback in an eye surgical simulator using four instruments (scalpel, forceps, scissors and phacoemulsifier). Webster et al utilised haptic feedback for simulating Gimbel and Neuhann's continuous curvilinear capsulorrhexis technique.22 They recently further described simulations of this technique using the EYESI.23 24
Phacoemulsification surgery prototypes have started to appear over the past 5 years, one of the earliest being developed in Sweden by Laurell et al25 Another novel method of teaching cataract surgery was developed by Prinz et al. The Ophthalmic Operation Vienna uses surgical videos accompanied by three-dimensional animated sequences of all surgical steps for cataract and glaucoma surgery.26 Moreover, Henderson et al described the Virtual Mentor27 to teach cognitive aspects of cataract surgery isolated from the physical tasks. The use of endoscopy has also been proposed as a tool to augment three-dimensional understanding of ocular structures and thereby assist in teaching cataract surgery to residents.28 Posterior segment simulators range from retinal photocoagulation29 to vitreoretinal surgery.30–33
At present, the only commercially available ophthalmic virtual reality surgical training system on the market is the EYESI, produced by VR magic (Mannheim, Germany) (figure 1). EYESI was originally designed as a vitreoretinal surgery simulator.34 It has since evolved to include a wealth of ophthalmic surgical procedures, including capsulorrhexis simulation and phacoemulsification modules. The simulator consists of a mannequin head prop with a separate vitreoretinal and cataract surgical interface. Movements of the instruments are relayed in real time onto a computer monitor, as well as onto a stereo microscope complete with foot pedals.
(A) The EYESI virtual reality ophthalmosurgical simulator. (B) Display screen. (C) Training history graph.
Using this simulator, trainees can practice standardised phacoemulsification techniques and abstract tasks repeatedly with instant objective feedback of performance. With graded exercises at different skill levels, they can be used as the basis for a structured training programme.
Learning curves on phacoemulsification trainers
A learning curve is defined as a line graph displaying opportunities across the x-axis and a measure of student performance along the y-axis. To our knowledge, there are no published data on the learning curve of ophthalmic surgical trainees during wet lab training or microsurgical skills courses. This is probably because validated eye surgical skills assessment tools are only just emerging. Prior to this, accurate assessment of the rate of proficiency acquisition using these methods was not possible.
Assessment using virtual reality systems is more readily available using the accompanying manufacture-provided default performance thresholds (although these too need to be validated).
Transfer of skills from phacoemulsification trainer to human patient
There are virtual reality to operating room (so-called ‘VR to OR’) trials published for laparoscopy17 18, bronchoscopy19 and endovascular20 surgery. The legitimacy of ophthalmologic surgery simulators through trials, however, is still uncertain. Studies the examine the transfer of simulated skills to real operations are essential before simulators can be incorporated into the ophthalmology training curriculum.
The studies that have so far established validity relate to vitreoretinal procedures. Peugnet et al35 tested a retinal photocoagulation simulator. Their results showed that residents training on the simulator required only 25.4 days of training compared with 42.25 days in the group that received conventional training. EYESI's vitreoretinal module was studied by Jonas and colleagues,31 who found that the group that trained on the simulator performed better than the non-trained group when evaluated in a wet lab setting performing vitrectomy on pig eyes.
One of the first phacoemulsification simulator trials was carried out by Folgar et al,36 who compared objective surgical outcomes in cataract surgeries performed by residents who have trained with the EYESI versus those who have trained utilising traditional didactic and wet lab methods. They suggest that ophthalmology residents who train with the EYESI simulator system may decrease surgical time, ultrasound time and energy dissipation, and rely less on attending interventions compared with residents who begin training directly with patients.
The promising trial results are an indication that virtual reality simulation might be what training programmes desperately need in these challenging times. However, it is necessary to obtain objective measurements of real performance to confirm this.
Assessment of technical skills in phacoemulsification surgery
Adequate assessment is necessary to demonstrate competence. For any method of skill assessment to be used with confidence it must be objective, reliable, valid and feasible. Reliability of an evaluation instrument relates to its ability to provide consistent results with minimal errors of measurement. Test/retest reliability and inter/intra-observer reliability are the most commonly used methods.37
Validity is defined as the extent to which an assessment instrument measures what it was designed to measure. An assessment should be able to demonstrate several forms of validity from the most basic face validity to content and construct or contrast (discriminative) validity to the most powerful predictive validity. Feasibility or practicality refers to the length and number of assessments, the number of personnel and the required time, space and costs.
Traditionally, however, assessment methods have been subjective, making them inherently susceptible to weaknesses. The opinion of the assessing trainer, for example, may be critically important. Friedlich et al38 demonstrated that a new examination for family medicine had low reliability when family physicians served as examiners, but moderate reliability when surgeons were the evaluators. Furthermore, the keeping of a personal surgical logbook is a reflection of one's experience as opposed to one's expertise. Complication rates, not only accumulation of numbers, need to be addressed when evaluating the competence of a surgeon.
Emerging tools in ophthalmic surgical skill assessment
In an attempt to design better ways of training ophthalmic surgeons, a number of different objective assessment tools have been described. Objective Assessment of Skills in Intraocular Surgery (OASIS)39 (table 1) is an entirely objective tool and thus has no inter-rater variability. It is a feasible one-page evaluation form with pre-, intra- and postoperative results components for phacoemulsification. The more subjective Global Rating Assessment of Skills in Intraocular Surgery (GRASIS) was developed later to be complementary to OASIS. GRASIS is a subjective, also one-page, evaluation form based on a five-point Likert Scale, with middle and extreme points anchored by explicit descriptors.40 A similar method has been used in assessing ophthalmic plastic surgical skills.41
Comparison of OASIS and GRASIS
Another method of assessment, originally developed to improve human performance and safety in high-risk industry, is the Human Reliability Analysis. The underlying principle is error analysis and has been used successfully in assessing laparoscopic cholecystectomies. Its use was modified to enable use in ophthalmology and it has had face and content validity established.42
Video-based assessment in phacoemulsification surgery
In order to reduce assessor bias, video-based assessments have been shown to reduce subjectivity in a manner similar to the Objective Structured Clinical Examination (OSCE) used in assessing the clinical skills of history-taking, physical examination and patient–doctor communication.43 Martin et al44 developed a similar approach to the assessment of operative skill, the Objective Structured Assessment of Technical Skill (OSATS), in general surgery. They used a 2 h six-station measurement of surgical performance of surgical trainees during simulated tasks, for example, excision of skin lesion, abdominal wall closure and bowel anastomoses. The OSATS tool demonstrated high reliability and construct validity. This study also concluded that bench model simulation gives equivalent results to use of live animals for this test format, suggesting that we can effectively measure residents' technical ability outside the operating room using bench model simulations.
Saleh et al45 adapted OSATS for Cataract Surgery (OSACSS). OSACSS has a six-item global rating system and a 14-item task-specific component consisting of a checklist particular to cataract surgery. Each of the components is rated on a five-point Likert scale. Thirty-eight surgical videotapes from 38 surgeons of four different levels of experience were evaluated. This study demonstrated construct validity of the OSACCS system with the total scores as well as the global and task-specific scores.
Binenbaum et al46 47 developed the Eye Surgical Skills Assessment Test (ESSAT). This is a three-station wet lab setting including skin suturing, muscle recession and phacoemulsification/wound construction, and suturing technique. Similar to OSACCS, the assessment methods include a station-specific checklist and a global rating scale performance for experts to complete while reviewing the resident's videotaped performance.
The disadvantages with all of the above rating scales are that they are complex and time-consuming. Furthermore, the scales are open to human error and not entirely without subjectivity. To achieve instant objective feedback of a surgeon's technical skills dexterity analysis and virtual reality may be more useful.
Dexterity analysis in ophthalmic surgery
The Imperial College Surgical Assessment Device (ICSAD) (figure 2) is an objective assessment of corneal suturing by using economy of movement measures. It is a system that has recently been validated to discern between trainees of different surgical experience in laparoscopic procedures, both for simple tasks48 and for real procedures such as laparoscopic cholecystectomy.49
Imperial College surgical assessment device (ICSAD). (A) Sensors and motion box. (B) Data analysis graphs.
The ICSAD has sensors placed on the back of a surgeon's hands50. A commercially available device (Isotrack II; Polhemus,Colchester, Vermont, USA) emits electromagnetic waves to track the position of the sensors in the x-, y- and z-axes 20 times per second. This device is able to run from a standard laptop computer and data are analysed in terms of time taken, distance travelled and total number of movements for each hand.
Saleh et al51evaluated motion tracking using the ICSAD to assess its role as a more objective assessment of ophthalmic surgical skill. The performance of three groups of differing levels of surgical skill was analysed while performing corneal sutures. Highly statistically significant differences were found between the three grades of surgeon experience for time taken (p<0.001), number of hand movements (p<0.001) and path length of the hand movements (p<0.002) to complete the given task.
Virtual reality simulators as assessment devices
Assessment of any task performed on a simulator or otherwise, together with meaningful feedback, is a vital part of the learning process.52 The importance of virtual reality simulators in assessing the training of ophthalmic surgeons is acknowledged in the new Work Based Assessments (WBAs) for Ophthalmic Specialist Training (OST) released by the Royal College of Ophthalmologists in June 2007. The practical skills WBAs, such as the Direct Observation of Procedural Skills (DOPS) and the OSATS, include simulator together with wet lab and patient as the optional method of assessment of the procedure performance.
The use of the EYESI as an assessment device was first demonstrated by Rossi et al33 using the vitreoretinal module. They showed a significant difference between students, residents and experienced surgeons in average completion time and number of surgical mistakes when performing the membrane-peeling module. Park et al53 later demonstrated that more experienced intraocular surgeons scored higher than more junior residents when performing five basic psychomotor tasks on the EYESI. When evaluated by group, higher course scores correlated with years of prior intraocular surgical experience. The more experienced intraocular surgeons reported that they felt the simulator to be an excellent supplemental teaching tool for intraocular surgery.
Comparison of assessment tools
Currently there is no consensus regarding the optimal assessment tool for phacoemulsification procedures. Video-based, virtual reality and dexterity systems should perhaps be used in conjunction. Other tools, such as eye-tracking devices, may also be used to highlight important areas of the operation.
Conclusion
Prior to incorporating virtual simulation into current training programmes there is a need to develop validated curricula for basic, intermediate and advanced levels of training. Learning curves then need to be determined that will allow the correct frequency of training to be calculated and whether tutors need to be present at all times. Competency levels should be defined using at least a national approach, enabling standardisation of training programmes. The organisation of these programmes must also be addressed. It is not necessary for every hospital to have a virtual reality simulation laboratory: training can be provided at regional skills centres. Unlike the current one-off microsurgical skills courses, with the establishment of regional centres it should prove possible to develop training programmes over periods of months and years.54
Such tools can then be utilised to address the increasingly limited opportunities for technical training and assessment that are offered to doctors, not only during training but throughout their careers (re-training and re-validation). Furthermore, for the first time, a proficiency-based curriculum can make the actual level of skill rather than a predetermined period of time the primary factor in physicians' progression up the training ladder. This risk-free training in technical skills will also ensure that patients are cared for by doctors with expertise in the procedures they perform. Financially, improved surgical training outside the theatre translates into greater cost-effectiveness in terms of anaesthesia and operating room time. Furthermore, costs will be reduced from managing patient morbidity following surgical complications.
Simulations, however, are not intended as a stand-alone teaching medium. They are more likely to be successful if systematically integrated with enriched curricular and educational environments. Previous attempts in other surgical specialities to rely solely on virtual reality have been unsuccessful55 and such a notion in ophthalmology training is likewise not recommended. Without doubt, virtual reality is more likely to be successful if systematically integrated into a carefully constructed education and training programme that objectively evaluates technical skills proximate to the learning experience.56 Training could comprise starting with wet labs and simulators to gain awareness, followed by structured or modular training in theatre with scored assessments and ICSAD type systems to show improvements in efficiency. This would comprise a good training package, and will lead to improved clinical reasoning and professionalism, inevitably translating into enhanced patient care.
References
Footnotes
Competing interests None.
Provenance and peer review Not commissioned; externally peer reviewed.
Linked Articles
- At a glance