Do simple checklists applied in operating theatres significantly reduce the rates of complication and death from surgery?

Question

Medical professionals are one of the most trusted groups in society. So we tend to trust them to do a good job without asking too many questions. So when a study showed that the simple application of surgical checklists could lead to a 36% reduction in complications and a 47% reduction in mortality not everyone believed the results (original BMJ report here) with one critic commenting:

Little evidence exists for Gawande’s proposal that the WHO checklist is appropriate for modern Western surgical practice. The original article, which showed small overall improvements in outcomes, included Third World hospitals where safety parameters improved from levels unthinkable in European or American practice.

Some of the original work was led by respected writer and medic Atul Gawande and later expanded into a book which also tried to explain why checklists work.

The question which deserves some applied skepticism is: given how professional we expect medics to be, do we believe something so simple as a pre-surgery checklist could lead to a significant improvement in outcomes?

Answer 1

Yes, checklists do make a significant difference in operating theatres in a variety of environments

The original research is reported in the NEJM here. To put the results in context the article provides the following background:

Although surgical care can prevent loss of life or limb, it is also associated with a considerable risk of complications and death. The risk of complications is poorly characterized in many parts of the world, but studies in industrialized countries have shown a perioperative rate of death from inpatient surgery of 0.4 to 0.8% and a rate of major complications of 3 to 17%. These rates are likely to be much higher in developing countries. Thus, surgical care and its attendant complications represent a substantial burden of disease worthy of attention from the public health community worldwide.

The original conclusion are worth reporting directly:

The rate of death was 1.5% before the checklist was introduced and declined to 0.8% afterward (P=0.003). Inpatient complications occurred in 11.0% of patients at baseline and in 7.0% after introduction of the checklist (P<0.001)

The surgeon and writer Atul Gawande became so convinced that he wrote a whole book on the topic of checklists where he examined their history and value in areas such as aviation and sought to explain why they work. The book is The Checklist Manifesto and it is well worth reading for further examples and some convincing rationale.

Answer 2

It is hard to determine what the percentages quoted in your question are referring to without a cite, but a quick search yields this: ( http://www.nejm.org/doi/full/10.1056/NEJMsa021721 )

"Risk Factors for Retained Instruments and Sponges after Surgery"

Patients with retained foreign bodies were more likely than controls to have had emergency surgery (33 percent vs. 7 percent, P<0.001) or an unexpected change in surgical procedure (34 percent vs. 9 percent, P<0.001).

This is certainly one area where a checklist procedure can have a large effect. I do not have any information regarding what proportion of complications are related to this particular type of incident.

Answer 3

The jury is still out as to whether surgical checklists used in a first world environment makes substantial savings of life or reduces complications significantly. The benefit seen in Gawande's trial [1] may have all come from the third world hospitals that were also included but as far as we know, the hospitals were not made public as to which were which. That condition may have been part of the reason why they chose to participate in the first place. They included the hospitals from following locations:

eight hospitals in eight cities (Toronto, Canada; New Delhi, India; Amman, Jordan; Auckland, New Zealand; Manila, Philippines; Ifakara, Tanzania; London, England; and Seattle, WA)

A large trial from Ontario, Canada was published in March 2014, and showed different results. [2]

During 3-month periods before and after adoption of a surgical safety checklist, a total of 101 hospitals performed 109,341 and 106,370 procedures, respectively. The adjusted risk of death during a hospital stay or within 30 days after surgery was 0.71% (95% confidence interval [CI], 0.66 to 0.76) before implementation of a surgical checklist and 0.65% (95% CI, 0.60 to 0.70) afterward (odds ratio, 0.91; 95% CI, 0.80 to 1.03; P=0.13). The adjusted risk of surgical complications was 3.86% (95% CI, 3.76 to 3.96) before implementation and 3.82% (95% CI, 3.71 to 3.92) afterward (odds ratio, 0.97; 95% CI, 0.90 to 1.03; P=0.29).

and they concluded the changes were not significant.

Implementation of surgical safety checklists in Ontario, Canada, was not associated with significant reductions in operative mortality or complications.

Atul Gawande blogged on the results of that trial [3] complaining that the participants lacked adequate training in the use of the checklists, and that there was no independent check on their compliance. The authors deny that these were important aspects.

Interestingly, lacking any well controlled double blinded trials in checklists, Gawande has turned to simulation testing in a crisis situation, and here the results favoured checklists. [4]

A total of 17 operating-room teams participated in 106 simulated surgical-crisis scenarios. Failure to adhere to lifesaving processes of care was less common during simulations when checklists were available (6% of steps missed when checklists were available vs. 23% when they were unavailable, P<0.001). The results were similar in a multivariate model that accounted for clustering within teams, with adjustment for institution, scenario, and learning and fatigue effects (adjusted relative risk, 0.28; 95% confidence interval, 0.18 to 0.42; P<0.001). Every team performed better when the crisis checklists were available than when they were not. A total of 97% of the participants reported that if one of these crises occurred while they were undergoing an operation, they would want the checklist used.

[1] Haynes AB, Weiser TG, Berry WR, Lipsitz SR, Breizat AH, Dellinger EP, Herbosa T, Joseph S, Kibatala PL, Lapitan MC, Merry AF, Moorthy K, Reznick RK, Taylor B, Gawande AA. A surgical safety checklist to reduce morbidity and mortality in a global population. N. Engl. J. Med. 2009 Jan;360(5):491-9. doi: 10.1056/NEJMsa0810119. PubMed PMID: 19144931.

[2] Urbach DR, Govindarajan A, Saskin R, Wilton AS, Baxter NN. Introduction of surgical safety checklists in Ontario, Canada. N. Engl. J. Med. 2014 Mar;370(11):1029-38. doi: 10.1056/NEJMsa1308261. PubMed PMID: 24620866+.

[3] http://theincidentaleconomist.com/wordpress/when-checklists-work-and-when-they-dont/

[4] Arriaga AF, Bader AM, Wong JM, Lipsitz SR, Berry WR, Ziewacz JE, Hepner DL, Boorman DJ, Pozner CN, Smink DS, Gawande AA. Simulation-based trial of surgical-crisis checklists. N. Engl. J. Med. 2013 Jan;368(3):246-53. doi: 10.1056/NEJMsa1204720. PubMed PMID: 23323901.