You are here

Article Text

Article menu
Clinical science
Intra-operative decision making by ophthalmic surgeons
Free
  1. Keryn Pauley1,
  2. Rhona Flin1,
  3. Augusto Azuara-Blanco2
  1. 1Industrial Psychology Research Centre, University of Aberdeen, Aberdeen, UK
  2. 2Centre for Vision and Vascular Sciences, Queen's University Belfast, Belfast, UK
  1. Correspondence to Professor Augusto Azuara-Blanco, Centre for Vision and Vascular Sciences, Queen's University Belfast, Institute of Clinical Science, Block A, Royal Victoria Hospital, Grosvenor Road, Belfast BT12 6BA, UK; a.azuara-blanco{at}qub.ac.uk

Abstract

Purpose Research on surgical decision making and risk management usually focuses on peri-operative care, despite the magnitude and frequency of intra-operative risks. The aim of this study was to examine ophthalmic surgeons’ intra-operative decisions and risk management strategies in order to explore differences in cognitive processes.

Method Critical decision method interviews were conducted with 12 consultant ophthalmologists who recalled cases and selected important decisions during the operations. These decisions were then discussed in detail in relation to decision making style and risk management. Transcripts were coded according to decision making strategy (analytical, recognition primed decision, creative and rule-based) and risk management (threats, risk assessment and risk tolerance).

Results The key decision in each case was made using either a rapid, intuitive mode of thinking (n=6, 50%) or a more deliberate comparison of alternative courses of action (n=6, 50%). Rule-based or creative decision making was not used. Risk management involved the perception of threats and assessment of threat impact but was also influenced by personal risk tolerance. Risk tolerance seemed to play a major role during situations requiring a stopping rule. Risk management did not appear to be influenced by time pressure.

Conclusions Surgeons described making key intra-operative decisions using either an intuitive or an analytical mode of thinking. Ophthalmic surgeons’ risk assessment, risk tolerance and decision strategies appear to be influenced by personality.

  • Treatment Surgery
  • Eye (Globe)

https://doi.org/10.1136/bjophthalmol-2012-302642

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Introduction

    Along with technical skills (eg, manual dexterity), non-technical skills such as team working, leadership, judgment and decision making are vital for optimal ophthalmic surgical performance.1 ,2 However, according to American ophthalmic programme directors, 22% of residents display poor intra-operative judgment and decision making, second only to poor manual dexterity (24%).3 Coupled with the rate of adverse events4 ,5 and intra-operative errors6 encountered during eye surgery by inexperienced surgeons, this suggests that training programmes should strengthen their focus on decision making, risk management and judgement skills in addition to the technical aspects of surgery.

    There is very little research on surgeons’ decision making and risk management during operations,7 apart from some recent studies of general surgeons.8 ,9 Across surgical specialties, most research concerns pre-operative decisions such as diagnosis and planning of the intervention.10–13 This is also true for ophthalmic surgery.14 ,15 Although preoperative planning and risk management are vital to ensure successful outcomes, sound intra-operative decision making is especially important during unforeseen circumstances (eg, malfunctioning of equipment, intra-operative flattening of the anterior chamber or reduction in pupil size). Even expected complications, such as rupture of the posterior capsule during cataract surgery, have a highly variable presentation and response to intra-operative manoeuvres.

    From research in other high risk occupations,16 ,17 it appears that the basic components of decision making and risk management are detection, assessment of situational risks and time available, option generation and option selection.18 Flin et al7 adapted a model of pilots’ in-flight decision making for surgery.18 This two stage model proposes that if a deviation from the intended plan is noticed, the surgeon first defines the problem and estimates the level of risk and the time available. Depending on this situation assessment, one of four main decision strategies is then used to choose a course of action: (i) intuitive recognition primed decision (RPD) making17 (rapid recollection of a single course of action based on situational cues); (ii) rule-based decision making (application of a documented procedure; if x, then y, from evidence-based medicine or guidelines); (iii) analytical decision making (several options are generated and compared simultaneously to determine the optimal course); and (iv) creative decision making (devising a novel solution when the situation is so unfamiliar that rule-based or analytical solutions are not available). However, some decisions will involve more than one type of cognitive process, using a mixture of the above strategies.

    In a recent study, our group investigated intra-operative decision making and risk management9 in a range of surgical specialties including urology, general, orthopaedic and vascular surgery. Surgeons were interviewed about a difficult case in order to examine their risk judgements and decision making processes. Interviews were analysed using thematic coding to determine whether a key decision in the case involved one of the four decision strategies described above. The surgeons described using either the analytical (50% of surgeons) or intuitive approach (50%) for key decisions during challenging surgery. These modes of thought are akin to ‘thinking slow’ and ‘thinking fast’, respectively.19 Selection of the method did not seem to be related to situational constraints, such as time pressure, type of operation (laparoscopic or open) or context (emergency or elective). Rather, surgeons’ decision strategy seemed to depend on individual preferences. Risk management was found to be an important part of the decision making process, involving perception of threats and assessment of their impact. Risk tolerance seemed to be influenced by personal disposition towards risk.

    Ophthalmology is of interest because of substantial differences compared to other surgical specialties: there is a high volume of interventions (cataract surgery is the most common operation in the National Health Service (NHS) with over 300 000 interventions per year) in mainly elective cases under local anaesthesia (a factor influencing communication among team members when a complication arises), predominantly in an elderly population, yet there can be a relatively high number of adverse events among inexperienced surgeons. Intra-operative complications are not life-threatening but can cause severe morbidity (ie, visual loss). The goal of this study was to examine decision making and risk management processes among ophthalmic surgeons.

    Methods

    A cross-sectional qualitative study was conducted. Ethics approval was granted by the NHS North-East Scotland Research Ethics Committee. Consultant ophthalmic surgeons (n=14) from four hospitals in Scotland were invited to participate via an email from a senior surgical colleague, and 12 agreed. The surgeons (10 male, 2 female), with a mean age of 48.5 (SD 7.7) years, had an average of 13 years’ experience as an attending/consultant surgeon (range 3–31 years, SD 9.0). Their surgical sub-specialities included vitreo-retinal, strabismus, general ophthalmology (cataract) and glaucoma. Informed consent was obtained before each interview.

    The critical decision method (CDM) was used in interviews.20 In this method, an expert recalls a critical incident, for example, where they made a difficult decision. The goal of the interview is to gather information about the decision process (see above) by probing for information regarding the timeline, cues, analogies, goals, options and the role of experience. We used similar questions as in previous research9 (see online supplementary appendix 1) with two additional items to quantify risk on a scale of 1–3. The cases recalled occurred between 1 week and 2 years prior to the interview (mean 11 months, SD 7 months).

    The surgeons recalled a challenging case from their operative experience. They were asked to describe basic details and the decisions that were made throughout the case. The interviewer (KP) repeated the information back as she understood it and the surgeon corrected the case details if necessary. The interviewer created a timeline to indicate key decisions, and the surgeon selected an important decision point where he or she experienced a major shift in understanding the problem or took some action which altered events. Selecting one decision point of this type enables a deep understanding of the decision making process, while limiting the length of the interview to an hour.20 Then 28 questions (see online supplementary appendix 1) were posed to identify decision making and risk management processes at that decision point.20

    Risk management processes include: (i) threat perception; (ii) risk assessment; and (iii) risk tolerance. Threat perception involves recognition of aspects of the situation (including patient related threats) which pose a threat to safety (eg, the complexity of the operation), the anxiety of the patient (particularly if the patient is under local anaesthesia), the state of the bilateral eye (if it is blind, the consequences of a poor outcome are much worse than if it has good visual function), individual or staff factors (eg, inexperience of staff members (nurses), the surgeon being on ‘auto-pilot’ because the procedure is routine) and culture/organisation factors (eg, only seeing the patient 5 min before the operation). Risk assessment is a cognitive ability and involves the estimation of risk associated with the recognised threat. The surgeons were asked to discuss the general level of risk in the situation and the potential impact on outcome and the patient's quality of life. Finally, risk tolerance is a personality variable, relating to the amount of risk a surgeon is willing to accept in a given situation. Risk tolerance follows risk assessment: the surgeon assesses the risk associated with the considered option(s) and determines whether this is an acceptable level of risk.

    Transcripts were coded according to decision making strategy (analytical, RPD, creative and rule-based10) and risk management (threats,21 risk assessment and risk tolerance). The intra-operative threats encountered by the surgeons were further classified according to origin (patient, task, operating environment, team, individual staff members and culture/organisation).21 A sample of six interviews was coded by a second investigator to assess reliability. The methodology used is particularly suited for understanding complex processes such as decision making.20 The sample size used in this study (n=12) is normally sufficient for data saturation with an in-depth interview method.22

    Results

    The inter-rater reliability (κ) was acceptable (κ values were 0.80 for decision making strategy, 0.67 for risk management and 0.90 for threats).

    The characteristics of the 12 ophthalmic cases and the decision strategies, risk rating and time pressures are shown in Otable 1.

    View this table:
    Table 1

    Characteristics of the 12 cases discussed (ordered by decision strategy)

    Decision making

    Although the interview focused on just one key decision, the cases often involved multiple decisions, similar to the situation with other non-ophthalmic surgeons.9 In all 12 cases, the surgeons described using either an analytical or intuitive (RPD) rather than a rule-based or creative strategy when making their key decisions.

    Analytical decision making

    In six (50%) of the cases, the surgeons described using an analytical decision method to make their key decision. Recalling these cases, the surgeons described weighing up the risks and benefits of more than one possible course of action before making a decision.

    In one case, the surgeon used a limited analytical strategy where he considered all available options, but only very briefly. In this case, the trainee surgeon did not respond to the code word to stop the operation, so the surgeon had to decide how to get the trainee's attention without alarming the patient (who was awake). The surgeon briefly considered the risks of a number of plausible solutions (eg, physically stopping the trainee, saying ‘stop’, or asking for an instrument which was only used in this complication) before deciding to try asking for the other instrument. If this did not work, he would try one of the other options.

    Other surgeons described how they used analytical decision making: How did I arrive at my decision? Analysed the possible options, of which there were two: do something or do nothing and realised that I could do something, simple as that. I weighed them up and decided that leaving it alone was a better option for the patient at the time.

    RPD decision making

    In the other six (50%) cases, the surgeons described using an intuitive (RPD) method to make their key decision. During these cases, the surgeons did not compare the risks and benefits associated with more than one option. There may have been other plausible alternatives, but they did not need to be considered because the surgeons already knew what to do as they had immediately identified their preferred course of action. There may have been some post hoc evaluation of different options (eg, asking ‘why didn't I do X?’), but they did not report doing this during the operation. These surgeons described thinking ahead, considering the positives and the negatives associated with their preferred option, but they never mentioned considering another option. They just thought the proposed response through to see if anything could go wrong if they applied this option; any intra-operative discussion of the option (eg, with junior surgical team members) was done for teaching purposes or to confirm the decision.

    The surgeons described their RPD decision strategy: I think that in retrospect, I probably, maybe, talk about these things slowly but actually it was actually a quick decision, and look at it and think: right, there is not much vitreous, I can see there, I'll just go ahead. You know, it was probably quite a quick, almost subconscious decision at the time. I knew that I had this complication; you're worried that you have this complication, but actually, the decision to go ahead and not do the vitrectomy and implanting was probably quite a quick one, it was a reflex one. I suppose on previous, some limited previous knowledge and a gut feeling that this was probably the best way forward.

    Regarding factors influencing the decision strategy, the six cases where the key decisions were characterised as RPD involved a range of operations, only one of which was cataract surgery. In four of these RPD cases, the surgeons said there was no time pressure. In the six cases where the surgeons reported using analytical decision making, five were cataract surgery. The risk ratings (see below) did not appear to differ in relation to decision strategy. In five cases (two RPD, three analytical), the surgeons described problems that they had not previously encountered. They said they used their prior experience in other situations, their reading around the topic and previous discussions with colleagues to select the appropriate course of action.

    Time pressure was reported in five cases as the patient was under local anaesthetic (difficulties lying still and anxiety that the operation was taking too long) and a shorter time frame was scheduled for each operation compared to general surgery cases. There was an expectation that local surgery was going to be quick (eg, 20 min long), which added to the time pressure. The surgeons said they often ignored this time pressure.

    Risk management

    Rating of surgical risk

    The surgeons rated the patient-related risk as high (n=7) or medium (n=5), and the majority (n=8) rated the risk to themselves as low. These risk ratings did not differ according to the perceived time pressure or decision strategy adopted.

    Risk assessment

    The eye surgeons considered whether the complication could be fixed and how it might affect the patient's quality of life. A second, more complex form of risk assessment was consideration of the risk associated with the different options. This was especially important during analytical decisions where the surgeons compared the risk (likelihood and severity of complications) associated with the different alternatives.

    In a number of cases, the surgeons considered short term versus long term risk. For example, one surgeon described a cataract operation where the haptics supporting the lens broke off. If the surgeon left the lens implant in place, there was a chance that the lens would be misaligned and that the patient would have to return for another operation. On the other hand, if the surgeon took the lens out, the operation would definitely take longer (thus increasing the chance of the patient becoming anxious) and the incision would have to be extended. As well as considering the certainty of the outcome (one was 100% likely, the other was less likely), the surgeon can also compare the immediacy of the negative outcome. If the lens were removed, the negative outcome would be immediate. However, if the lens were retained, the possible negative outcome would occur in several days time.

    Risk tolerance

    Risk tolerance played a major role in the surgeons’ decision making and was discussed by all of them. Issues that affected risk tolerance included the experience of the surgeon and the patient's risk. Risk tolerance seemed to play an especially important role during situations requiring a stopping rule (eg, taking over from the trainee surgeon, removing more scar tissue from the retina).

    The risk management process was illustrated in a number of quotes: I guess that I... that was a risk, I guess that I was risking if I left it, it might cause complications but overall it would be better than going in and risking stabbing the vitreous and having to go in and suck the vitreous out and do the whole thing. I wasn't aware of particular deepening of the chamber, so that made me treat it more conservatively.

    In one particular case, the surgeon described the relationship between threat perception, risk assessment and risk tolerance. He noticed a tear in the posterior capsule (during cataract surgery) based on subtle visual information and decided (based on the same visual information) that very little vitreous was coming through (threat perception) and that it did not threaten the safety of the operation (risk assessment). Therefore, he did not have to worry about performing a vitrectomy, but could proceed and put the implant in the eye (risk tolerance), in other words, he was comfortable taking this level of risk. In this case, time pressure might have been a factor.

    Discussion

    Intra-operative decision making and risk assessment are essential skills for effective and safe eye surgery, and thus should be part of training programmes. This is the first attempt to develop a method to determine how consultant ophthalmic surgeons evaluate and manage risk when taking decisions during eye operations.

    The CDM interview employed in this study is a technique used to measure on-task decision making in high risk activities such as fire fighting, aviation and healthcare.20 CDM interviews are useful to measure intra-operative decision making and risk management, and have been used to develop curricula to inform decision making behind other surgical procedures, for example, percutaneous tracheotomy placement23 and flexor tendon repair.24

    In this study eye surgeons facing a challenging situation during surgery used both analytical and intuitive (RPD) decision making. Similarly, both strategies have been documented during practice in other surgical specialties.9 ,25 ,26 RPD is faster and requires less working memory27 but can only be used by experienced surgeons familiar with the problem in hand. However, with enough time, or when faced with a complex or ambiguous situation, an analytical decision strategy may be adopted. It has been suggested that surgeons use their preferred strategy for 70% of decisions, while situational constraints determine strategy use for 30% of decisions.25 Moulton et al8 described how surgeons have to transition between the more effortful analytical stage to the more automatic, intuitive mode during surgery, with a recent observational study of surgeons’ decision making reporting a cyclical model based on this type of transition.28

    The decision making of car drivers29 and aircraft pilots30 has been shown to be influenced by their risk management, comprising threat perception, risk assessment and risk tolerance. Surgeons’ preoperative risk assessment31 and risk attitudes32 ,33 have been studied previously. In this study, eye surgeons evaluated the risk before and during surgery. Risk ratings and risk management did not seem to be influenced by time pressure or the decision strategy adopted (intuitive or analytical). While the surgeons in our previous study discussed this in terms of the likelihood of patient death, the eye surgeons considered the likelihood of the patient losing an eye or eyesight.

    Behavioural areas of competence should receive explicit attention during ophthalmic surgical training to ensure individuals are aware of what constitutes good practice and that they have the requisite skills to deliver this. Recognition of the importance of these non-technical skills in anaesthesia has led to the development of aviation-style crew resource management training courses, for example, Crisis Avoidance and Resource Management (CARMA) and Anaesthesia Crisis Resource Management (ACRM). A taxonomy of anaesthetists’ non-technical skills (ANTS) and a behavioural scale to rate them is now being used in some anaesthetic training programmes.34 For general surgeons, a non-technical skills for surgeons (NOTSS) taxonomy and rating system was developed using task analysis with subject matter experts and evaluated in trials using standardised video scenarios and real operations.35 ,36 It allows consultant (attending) surgeons to give feedback to colleagues and trainees based on structured observations of non-technical aspects of performance during intra-operative surgery.35 Training in NOTSS is offered by The Royal College of Surgeons of Edinburgh, and the Royal Australasian College of Surgeons have incorporated NOTSS into their new professional standards. Similar developments should perhaps be considered for training ophthalmic surgeons.37

    References

    1. ↵
      1. Azuara-Blanco A,
      2. Reddy A,
      3. Wilkinson G,
      4. et al
      . Safe eye surgery: non-technical aspects. Eye 2011;25:1109–11.
      OpenUrlCrossRefPubMed
    2. ↵
      1. Benjamin L
      . Selection, teaching and training in ophthalmology. Clin Exp Ophthalmol 2005;33:524–30.
      OpenUrlCrossRefPubMedWeb of Science
    3. ↵
      1. Binenbaum G,
      2. Volpe NJ
      . Ophthalmology resident surgical competency: a national survey. Ophthalmology 2006;113:1237–44.
      OpenUrlCrossRefPubMedWeb of Science
    4. ↵
      1. Buchan JC,
      2. Cassels-Brown A
      . Determinants of cataract surgical opportunities and complication rates in UK higher specialist training. Eye 2008;22:1425–9.
      OpenUrlCrossRefPubMedWeb of Science
    5. ↵
      1. Neily J,
      2. Mills PD,
      3. Eldridge N,
      4. et al
      . Incorrect surgical procedures within and outside of the operating room. Arch Surg 2009;144:1028–34.
      OpenUrlCrossRefPubMedWeb of Science
    6. ↵
      1. Gauba V,
      2. Tsangaris P,
      3. Tossounis C,
      4. et al
      . Human reliability analysis of cataract surgery. Arch Ophthalmol 2008;126:173–7.
      OpenUrlCrossRefPubMedWeb of Science
    7. ↵
      1. Flin R,
      2. Youngson G,
      3. Yule S
      . How do surgeons make intraoperative decisions? Qual Safety Health Care 2007;16:235–9.
      OpenUrlCrossRef
    8. ↵
      1. Moulton CA,
      2. Regehr G,
      3. Lingard L,
      4. et al
      . ‘Slowing down when you should’: initiators and influences of the transition from the routine to the effortful. J Gastrointest Surg 2010;14:1019–26.
      OpenUrlCrossRefPubMed
    9. ↵
      1. Pauley K,
      2. Flin R,
      3. Yule S,
      4. et al
      . Surgeons’ intraoperative decision making and risk management. Am J Surg 2011;202:375–81.
      OpenUrlCrossRefPubMed
    10. ↵
      1. Bornstein BH,
      2. Emler AC
      . Rationality in medical decision making: a review of the literature on doctors’ decision-making biases. J Eval Clin Pract 2001;7:97–107.
      OpenUrlCrossRefPubMedWeb of Science
    11. ↵
      1. Larsson G,
      2. Weibull H,
      3. Larsson BW
      . Analysis of the decision-making process leading to appendectomy: a grounded theory study. Scand J Psychol 2004;45:449–54.
      OpenUrlCrossRefPubMed
    12. ↵
      1. Morineau T,
      2. Morandi X,
      3. Le MN,
      4. et al
      . Decision making during preoperative surgical planning. Hum Factors 2009;51:67–77.
      OpenUrlAbstract/FREE Full Text
    13. ↵
      1. Neumayer L,
      2. Vargo D
      . Principals of preoperative and operative surgery. In: Towsend C, Beauchamp R, Evers B, et al. eds. Sabiston textbook of surgery: the biological basis of modern surgical practice. Philadelphia: Sauders Elsevier, 2008:251–79.
    14. ↵
      1. Ainsworth J
      . Clinical decision making in paediatric ophthalmology. Optom Today 1999;39:27–35.
      OpenUrl
    15. ↵
      1. van der Meulen J,
      2. Rahi J
      . Clinical decision making in ophthalmology. Br J Ophthalmol 2002;86:599–600.
      OpenUrlFREE Full Text
    16. ↵
      1. Adams RJ,
      2. Ericsson AE
      . Introduction to cognitive processes of expert pilots. Hum Perform Extreme Environments 2000;5:44–62.
      OpenUrl
    17. ↵
      1. Klein G
      . A recognition-primed decision (RPD) model of rapid decision making. In: Klein G, Orasanu J, Calderwood R, et al. eds. Decision making in action: models and methods. Norwood, NJ: Albex Publishing, 1993:138–47.
    18. ↵
      1. Orasanu J,
      2. Fischer U
      . Finding decisions in natural environments: the view from the cockpit. In: Zsambok C, Klein G, eds. Naturalistic decision making. Mahwah, NJ: Lawrence Erlbaum Associates, 1997:343–57.
    19. ↵
      1. Kahneman D
      . Thinking fast and slow. London: Allen Lane, 2011.
    20. ↵
      1. Crandall B,
      2. Klein G,
      3. Hoffman R
      . Working minds: a practitioner's guide to cognitive task analysis. Cambridge: MIT Press, 2006.
    21. ↵
      1. Catchpole KR,
      2. Giddings AE,
      3. de Leval MR,
      4. et al
      . Identification of systems failures in successful paediatric cardiac surgery. Ergonomics 2006;49:567–88.
      OpenUrlCrossRefPubMedWeb of Science
    22. ↵
      1. Guest G,
      2. Bunce A,
      3. Johnson L
      . How many interviews are enough? An experiment with data saturation and variability. Field Methods 2006;18:59–82.
      OpenUrlAbstract/FREE Full Text
    23. ↵
      1. Sullivan ME,
      2. Brown CV,
      3. Peyre SE,
      4. et al
      . The use of cognitive task analysis to improve the learning of percutaneous tracheostomy placement. Am J Surg 2007;193:96–9.
      OpenUrlCrossRefPubMedWeb of Science
    24. ↵
      1. Luker KR,
      2. Sullivan ME,
      3. Peyre SE,
      4. et al
      . The use of a cognitive task analysis-based multimedia program to teach surgical decision making in flexor tendon repair. Am J Surg 2008;195:11–5.
      OpenUrlCrossRefPubMed
    25. ↵
      1. Abernathy C,
      2. Hamm R
      . Surgical intuition. Philadelphia: Hanley & Belfus, 1995.
    26. ↵
      1. Jacklin R,
      2. Sevdalis N,
      3. Darzi A,
      4. et al
      . Mapping surgical practice decision making: an interview study to evaluate decisions in surgical care. Am J Surg 2008;195:689–96.
      OpenUrlCrossRefPubMed
    27. ↵
      1. Baddeley A,
      2. Hitch G
      . Working memory. In: Bower G, ed. The psychology of learning and motivation: advances in research and theory. New York: Academic Press, 1974:47–89.
    28. ↵
      1. Christancho S,
      2. Vanstone M,
      3. Lingard L,
      4. et al
      . When surgeons face intraoperative challenges: a naturalistic model of surgical decision making. Am J Surg 2013;205:156–62.
      OpenUrlCrossRefPubMed
    29. ↵
      1. Deery HA
      . Hazard and risk perception among young novice drivers. J Saf Res 1999;30:225–36.
      OpenUrlCrossRef
    30. ↵
      1. Hunter D
      . Risk perception and risk tolerance in aircraft pilots. Report number DOT/FAA/AM-02/17. Washington, DC: Federal Aviation Administration, 2002.
    31. ↵
      1. Jacklin R,
      2. Sevdalis N,
      3. Harries C,
      4. et al
      . Judgment analysis: a method for quantitative evaluation of trainee surgeons’ judgments of surgical risk. Am J Surg 2008;195:183–8.
      OpenUrlCrossRefPubMed
    32. ↵
      1. Nakata Y,
      2. Okuno-Fujiwara M,
      3. Goto T,
      4. et al
      . Risk attitudes of anesthesiologists and surgeons in clinical decision making with expected years of life. J Clin Anesth 2000;12:146–50.
      OpenUrlCrossRefPubMedWeb of Science
    33. ↵
      1. Tubbs EP,
      2. Elrod JA,
      3. Flum DR
      . Risk taking and tolerance of uncertainty: implications for surgeons. J Surg Res 2006;131:1–6.
      OpenUrlCrossRefPubMed
    34. ↵
      1. Flin R,
      2. Patey R,
      3. Glavin R,
      4. et al
      . Anaesthetists’ non-technical skills. Br J Anaesth 2010;105:38–44.
      OpenUrlAbstract/FREE Full Text
    35. ↵
      1. Flin R,
      2. Patey R
      . Improving patient safety through training in non-technical skills. BMJ 2009;339:b3595.
      OpenUrlFREE Full Text
    36. ↵
      1. Yule S,
      2. Flin R,
      3. Maran N,
      4. et al
      . Surgeons’ non-technical skills in the operating room: reliability testing of the NOTSS behavior rating system. World J Surg 2008;32:548–56.
      OpenUrlCrossRefPubMedWeb of Science
    37. ↵
      1. Flin R,
      2. Azuara-Blanco
      . Preventing surgical complications by the use of non-technical surgical skills. In: Lois N, Wong D, eds. Complications of vitreo-retinal surgery. Philadelphia: Lippincott, 2013:493–9.

    Supplementary materials

    • Supplementary Data

      This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

      Files in this Data Supplement:

    Footnotes

    • KP is currently working in industry.

    • Contributors All authors made substantial contributions to conception, design, analysis and interpretation of data. KP led the acquisition of data. The three authors contributed to drafting the article and approved the current version.

    • Competing interests None.

    • Ethics approval The North-East Scotland Research Ethics Committee of the National Health Service approved this study (NRS101SR20).

    • Provenance and peer review Not commissioned; externally peer reviewed.