Article Text

Download PDFPDF
Accuracy of automated machine learning in classifying retinal pathologies from ultra-widefield pseudocolour fundus images
  1. Fares Antaki1,2,
  2. Razek Georges Coussa3,
  3. Ghofril Kahwati4,
  4. Karim Hammamji1,
  5. Mikael Sebag1,
  6. Renaud Duval2
  1. 1Department of Ophthalmology, Centre Hospitalier de l'Universite de Montreal (CHUM), Montreal, Quebec, Canada
  2. 2Department of Ophthalmology, Hopital Maisonneuve-Rosemont (CUO-HMR), Montreal, Quebec, Canada
  3. 3Department of Ophthalmology and Visual Sciences, University of Iowa Hospitals and Clinics, Iowa City, Iowa, USA
  4. 4Department of Electrical Engineering, Ecole de technologie superieure (ETS), Montreal, Quebec, Canada
  1. Correspondence to Dr Renaud Duval, Department of Ophthalmology, Hopital Maisonneuve-Rosemont, Montreal, Canada; renaud.duval{at}gmail.com

Abstract

Aims Automated machine learning (AutoML) is a novel tool in artificial intelligence (AI). This study assessed the discriminative performance of AutoML in differentiating retinal vein occlusion (RVO), retinitis pigmentosa (RP) and retinal detachment (RD) from normal fundi using ultra-widefield (UWF) pseudocolour fundus images.

Methods Two ophthalmologists without coding experience carried out AutoML model design using a publicly available image data set (2137 labelled images). The data set was reviewed for low-quality and mislabeled images and then uploaded to the Google Cloud AutoML Vision platform for training and testing. We designed multiple binary models to differentiate RVO, RP and RD from normal fundi and compared them to bespoke models obtained from the literature. We then devised a multiclass model to detect RVO, RP and RD. Saliency maps were generated to assess the interpretability of the model.

Results The AutoML models demonstrated high diagnostic properties in the binary classification tasks that were generally comparable to bespoke deep-learning models (area under the precision-recall curve (AUPRC) 0.921–1, sensitivity 84.91%–89.77%, specificity 78.72%–100%). The multiclass AutoML model had an AUPRC of 0.876, a sensitivity of 77.93% and a positive predictive value of 82.59%. The per-label sensitivity and specificity, respectively, were normal fundi (91.49%, 86.75%), RVO (83.02%, 92.50%), RP (72.00%, 100%) and RD (79.55%,96.80%).

Conclusion AutoML models created by ophthalmologists without coding experience can detect RVO, RP and RD in UWF images with very good diagnostic accuracy. The performance was comparable to bespoke deep-learning models derived by AI experts for RVO and RP but not for RD.

  • retina
  • imaging

Data availability statement

Data are available by contacting the authors of the Tsukazaki Optos Public Project. The specific subset of images used in this study are available upon reasonable request.

Statistics from Altmetric.com

Data availability statement

Data are available by contacting the authors of the Tsukazaki Optos Public Project. The specific subset of images used in this study are available upon reasonable request.

View Full Text

Footnotes

  • Twitter @FaresAntaki

  • Contributors Conception and design of the study (FA, RGC and RD); data collection and image relabelling (FA and RGC); data analysis (FA, GK and KH); writing of the manuscript and preparation of figures (FA); supervision (KH, MS and RD); review and discussion of the results (all authors); edition and revision of the manuscript (all authors).

  • Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

  • Competing interests None declared.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.