EXTENDED REPORT

The efficacy of automated "disease/no disease" grading for diabetic retinopathy in a systematic screening programme

S Philip¹, A D Fleming², K A Goatman², S Fonseca³, P Mcnamee⁴, G S Scotland⁴, G J Prescott³, P F Sharp² and J A Olson⁵

¹ Biomedical Physics and Grampian Retinal Screening Programme, University of Aberdeen, Foresterhill, Aberdeen
² Biomedical Physics, University of Aberdeen, Foresterhill, Aberdeen
³ Department of Public Health, University of Aberdeen, Foresterhill, Aberdeen
⁴ Health Economics Research Unit, University of Aberdeen, Foresterhill, Aberdeen
⁵ Retinal Screening, David Anderson Building, Foresterhill Road, Aberdeen

Correspondence to:
Dr John A Olson, Clinical Director, Diabetes Retinal Screening Service, David Anderson Building, Foresterhill Road, Aberdeen AB25 2ZP; John.olson{at}nhs.net

Aim: To assess the efficacy of automated "disease/no disease" grading for diabetic retinopathy within a systematic screening programme.

Methods: Anonymised images were obtained from consecutive patients attending a regional primary care based diabetic retinopathy screening programme. A training set of 1067 images was used to develop automated grading algorithms. The final software was tested using a separate set of 14 406 images from 6722 patients. The sensitivity and specificity of manual and automated systems operating as "disease/no disease" graders (detecting poor quality images and any diabetic retinopathy) were determined relative to a clinical reference standard.

Results: The reference standard classified 8.2% of the patients as having ungradeable images (technical failures) and 62.5% as having no retinopathy. Detection of technical failures or any retinopathy was achieved by manual grading with 86.5% sensitivity (95% confidence interval 85.1 to 87.8) and 95.3% specificity (94.6 to 95.9) and by automated grading with 90.5% sensitivity (89.3 to 91.6) and 67.4% specificity (66.0 to 68.8). Manual and automated grading detected 99.1% and 97.9%, respectively, of patients with referable or observable retinopathy/maculopathy. Manual and automated grading detected 95.7% and 99.8%, respectively, of technical failures.

Conclusion: Automated "disease/no disease" grading of diabetic retinopathy could safely reduce the burden of grading in diabetic retinopathy screening programmes.

Abbreviations: DH/MA, dot haemorrhage/microaneurysm

Competing interests: Implementation in Scotland is being considered. If this occurs it is likely that there will be some remuneration for the University of Aberdeen, NHS Grampian and the Scottish Executive.

Funding: This project was funded by the Chief Scientist Office, Scottish Executive Health Department (grant number CZH/4/76).

Ethics approval: Ethics approval was obtained from the Grampian Medical Research Ethics Committee for the use of the anonymised images and grading data.

CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    What's this?

This article has been cited by other articles:

• Olson, J. A., Sharp, P. F., Fleming, A., Philip, S. (2008). Evaluation of a System for Automatic Detection of Diabetic Retinopathy From Color Fundus Photographs in a Large Population of Patients With Diabetes: Response to Abramoff et al.. Diabetes Care 31: e63-e63 [Full Text]  
• Abramoff, M. D., Niemeijer, M., Suttorp-Schulten, M. S.A., Viergever, M. A., Russell, S. R., van Ginneken, B. (2008). Evaluation of a System for Automatic Detection of Diabetic Retinopathy From Color Fundus Photographs in a Large Population of Patients With Diabetes: Response to Olson et al.. Diabetes Care 31: e64-e64 [Full Text]  
• Scotland, G S, McNamee, P, Philip, S, Fleming, A D, Goatman, K A, Prescott, G J, Fonseca, S, Sharp, P F, Olson, J A (2007). Cost-effectiveness of implementing automated grading within the national screening programme for diabetic retinopathy in Scotland. Br. J. Ophthalmol. 91: 1518-1523 [Abstract] [Full Text]