Article Text
PDF
Abstract
Aims To evaluate the effectiveness of glaucoma screening using glaucoma suspect (GS) referral criteria assessed on colour fundus photographs in Singapore’s Integrated Diabetic Retinopathy Programme (SiDRP).
Methods A case–control study. This study included diabetic subjects who were referred from SiDRP with and without GS between January 2017 and December 2018 and reviewed at Singapore National Eye Centre. The GS referral criteria were based on the presence of a vertical cup-to-disc ratio (VCDR) of ≥0.65 and other GS features. The final glaucoma diagnosis confirmed from electronic medical records was retrospectively matched with GS status. The sensitivity, specificity and positive predictive value (PPV) of the test were evaluated.
Results Of 5023 patients (2625 with GS and 2398 without GS) reviewed for glaucoma, 451 (9.0%, 95% CI 8.2% to 9.8%) were confirmed as glaucoma. The average follow-up time was 21.5±10.2 months. Using our current GS referral criteria, the sensitivity, specificity and PPV were 81.6% (95% CI 77.7% to 85.1%), 50.6% (95% CI 49.2% to 52.1%) and 14.0% (95% CI 13.4% to 14.7%), respectively, resulting in 2257 false positive cases. Increasing the VCDR cut-off for referral to ≥0.80, the specificity increased to 93.9% (95% CI 93.1% to 94.5%) but the sensitivity decreased to 11.3% (95% CI 8.5% to 14.6%), with a PPV of 15.4% (95% CI 12.0% to 19.4%).
Conclusions Opportunistic screening for glaucoma in a lower VCDR group could result in a high number of unnecessary referrals. If healthcare infrastructures are limited, targeting case findings on a larger VCDR group with high specificity will still be beneficial.
- Glaucoma
- Telemedicine
- Diagnostic tests/Investigation
- Public health
Data availability statement
Data are available on reasonable request.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
WHAT IS ALREADY KNOWN ON THIS TOPIC
Patients with diabetes have a higher risk to have glaucoma, earlier detection of glaucoma may prevent from irreversible blindness. Diabetic retinopathy screening programmes can provide opportunistic detection for glaucoma based on optic disc features on colour fundus photography, however, due to the low prevalence of glaucoma this may result in a large number of false positives. Evaluation of the effectiveness of photographic screening will better inform glaucoma screening strategy.
WHAT THIS STUDY ADDS
Opportunistic screening for glaucoma using vertical cup-to-disc ratio (VCDR) threshold of 0.65 showed modest sensitivity and low specificity, resulting in more false positives than true positives. Adjusting the threshold for referral to VCDR >0.80 reduces the false positive rate but at the expense of missing early glaucoma.
HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY
A national teleophthalmology diabetic screening must weigh the benefits of early identification of disc suspects in preventing glaucoma blindness against the number of unnecessary referrals. If a healthcare system is concerned about being overloaded with false positive cases, increasing the referral threshold should be considered. However, this strategy must be followed by annual follow-up on the lower VCDR group so that earlier referral can be made whenever VCDR changes are detected over time.
Introduction
Teleophthalmology has enhanced the early detection of vision-threatening eye diseases and improved the accessibility of ophthalmic screening to a broader population.1 2 This has been evident for diabetic retinopathy (DR) where national telemedicine screening programmes have shown to be cost-effective3 4 and reduced the incidence of diabetic blindness.5 6 Singapore has established a nationwide teleophthalmology screening programme for DR, the Singapore’s Integrated Diabetic Retinopathy Programme (SiDRP), at the primary care level since 2010 using colour fundus photographs (CFPs).3 During DR screening, opportunistic screening for other ocular diseases such as glaucoma can be evaluated.
Earlier detection and treatment of glaucoma can reduce the risk of irreversible vision loss. In community screening, evaluation of the optic nerve structure is shown potential to detect early glaucomatous damage when patients remain asymptomatic.7 8 The SiDRP opportunistically refers individuals with glaucomatous optic neuropathy (glaucoma suspects, GS) detected on CFP to tertiary eye centres for further evaluation. However, there is no single definitive feature of a glaucomatous optic nerve. A large vertical cup-to-disc ratio (VCDR), thinning of the neuroretinal rim, disc haemorrhages, an asymmetrical disc between two eyes ≥0.20, or a combination of these, are signs of GS.
Because CFP is the only modality employed in the SiDRP for detecting GS, it is important to evaluate the value of these glaucomatous features given the prevalence of glaucoma is low. A high proportion of false positive referrals may result in unnecessary referrals that can overwhelm a tertiary eye care system and produce unnecessary financial and emotional burden, as well as a high time burden in the evaluation of a GS.9 On the other hand, patients with diabetes may be at a higher risk of glaucoma,10 11 and if GS is not identified and referred in a timely manner, these patients may suffer irreversible vision loss.
The purpose of this paper is to evaluate the effectiveness of using current GS referral criteria on CFP for opportunistic screening for glaucoma in a population with diabetes in the largest eye centre in Singapore. We report the characteristics of subjects with confirmed glaucoma, as well as the performance of GS criteria for detecting confirmed glaucoma, including sensitivity, specificity and positive predictive value (PPV).
Methods
Singapore’s Integrated Diabetic Retinopathy Programme
In Singapore, SiDRP is a national community-based programme that provides annual DR screening to adults with diabetes aged 18 years and above. Screening is conducted among 21 primary care clinics. The screening protocol has been described previously3 and is briefly discussed below.
All patients with diabetes underwent a standard DR screening procedure, which included the collection of sociodemographic data, systemic and ocular histories, visual acuity (VA) testing, and CFP. CFP were taken with a non-mydriatic digital fundus camera by a trained nurse. Two 45° retinal images, one centred on the optic disc and another centred on the fovea, were captured from each eye. Retinal images were graded by certified non-physician graders who underwent intensive training on non-stereoscopic view at ocular reading centres and are audited annually, as briefly discussed in our previous paper.12
Grading process
The SiDRP grading process is a two-tiered model where a primary grader would initially review the quality of retinal images, the presence of media opacity, and the presence of ocular conditions, and escalates any abnormal cases for secondary grader’s confirmation. Ocular conditions that are evaluated including DR, treated DR, diabetic macula oedema (DME), age-related macular degeneration (AMD), GS, disc swelling, macular haemorrhage, retinal detachment, retinal emboli, retinal artery or vein occlusion, central serous retinopathy, pseudo/macular hole, retinitis pigmentosa, epiretinal membrane and disc collaterals. An ophthalmologist’s opinion is sought if a grader requires an urgent escalation to the hospital.
Patients’ information and grading results were entered into the SiDRP grading application that resides in a secure hospital cloud network. An electronic report with recommended actions was generated and uploaded to the primary care electronic medical record (EMR) system within a day. The results were reviewed by the primary care physician who then advised the patient on subsequent management which includes a referral to any of the tertiary eye centres if any significant eye disease, including GS, is detected. Patients decided which tertiary eye centre to visit, and while eye specialists in the tertiary eye centre may have access to the patients’ screening test through the referral letter, all CFP graders did not have access to their final diagnosis.
The definition of GS
GS was diagnosed based on the presence of any of the following disc features identified on non-stereoscopic retinal images: VCDR of ≥0.65 and or other GS features (GF), such as disc asymmetry of two eyes ≥0.2, disc haemorrhages, neural retinal rim (NRR) thinning or any NRR notching. These essential referral criteria were previously defined by the SiDRP steering committee in 2015. The GS criteria, as well as criteria for other ocular conditions, are reviewed regularly to ensure that referral guidelines conform to the latest clinical evidence. During this study period, no changes to the GS referral criteria were implemented.
The VCDR was measured using the Singapore Optic Disc Assessment (SODA) software which used a polynomial curve fitting algorithm to measure the size of cup and disc on CFP. The subsequent cup-disc-ratio was calculated automatically by the programme. The reliability of this programme in measuring VCDR has been evaluated and published,13 with intraclass correlation values ranging between 0.87 and 0.94 for intragrader and 0.82 and 0.94 for intergraders reliability. An example of a patient with a large VCDR value measured by SODA software is shown in figure 1.
An example of a patient with VCDR 0.78, a suspicious appearance of glaucoma optic neuropathy, measured with the Singapore Optic Disc Assessment (SODA) software. VCDR, vertical cup-to-disc ratio.
Study population
The design of this study was case–control based on the presence of GS. The study population in this study were all diabetic subjects screened by SiDRP between January 2017 and December 2018 and referred to Singapore National Eye Centre (SNEC) for further evaluation. All subjects referred to SNEC for further evaluation were either GS, DR (moderate non-proliferative DR to proliferative DR), DME, advanced AMD, other ocular diseases or the combination of the diseases. The referred subjects were then divided into two groups: those who had GS features (GS group) and those who did not (non-GS group).
Evaluating glaucoma outcomes
The EMRs of all SiDRP referrals to SNEC were reviewed until August 2020 to identify outcomes, allowing up to 3.5 years of follow-up from the time of referral. Glaucoma was confirmed if patients had a diagnosis code of primary open-angle glaucoma (POAG), primary angle-closure glaucoma (PACG), normal-tension glaucoma (NTG), secondary glaucoma, juvenile glaucoma, non-specified glaucoma or if they had commenced treatment with intraocular pressure (IOP) lowering eye-drops, glaucoma laser or surgery in any eye. All diagnosis and treatment were established by glaucoma specialists based on the World Glaucoma Association and Asia Pacific Glaucoma guidelines,14 15 which involved a comprehensive clinical examination of risk factors, IOP, gonioscopy and multimodal evaluation of the optic nerve appearance corresponding to visual field loss.
Statistical analysis
The characteristics of GS and non-GS subjects, including demographics, medical history, ocular examination were compared between groups using the t-test for continuous variables, and the χ2 test for categorical variables. If both eyes were confirmed with glaucoma, we used the parameters of the worse eye. The highest IOP reading of all visits to the specialist clinic was chosen in the analysis. At the time of the last review, subjects were divided into two groups: glaucoma and non-glaucoma. All subjects who were not confirmed to have glaucoma at the last review or who were still followed up for other retinal diseases were classified as non-glaucoma.
We estimated the minimum sample size required was 1230, based on an expected 5% prevalence of glaucoma in patients with diabetes16 and both sensitivity and specificity of 80%,17 power of 80% and the p value is set to be less than 0.05. We used STARD reporting guidelines for this study.18
To assess the effectiveness of our GS screening criteria on CFP, the sensitivity, specificity and PPV for glaucoma diagnosis were calculated. We evaluated the test performance on different settings based on the GS referral features: large VCDR alone and large VCDR and other GF.
Sensitivity (true positive rate) was the proportion of subjects with true glaucoma among of all glaucoma cases. Specificity (true negative rate) was the proportion of subjects with false glaucoma among no glaucoma cases. To calculate PPV, the total number of true glaucoma was divided by the total number of GS subjects. A summary of the calculation is shown in online supplemental material 1.
Supplemental material
All data were analysed using Python program V.3.7 and SPSS (IBM SPSS Statistics, software V.26; IBM/SPSS). A p value less than 0.05 was considered statistically significant.
Results
There were 5450 diabetic subjects referred to SNEC for further review and management. We reviewed subjects who were adhered to their appointments and had available EMRs (5023 subjects). Of 5023, there were 2625 GS and 2398 non-GS subjects. Of the 2625 GS, 353 (13.4%) subjects had other concurrent eye diseases such as DR/DME (161 subjects, 6.1%); AMD (19 subjects, 0.7%), DR and AMD (4 subjects, 2%) and other ocular diseases (169 subjects, 6.4%). In the non-GS group, there were 1701 (70.9%), 199 (8.3%), 36 (1.5%) and 495 (20.6%) subjects with DR/DME, AMD, DR and AMD, and other ocular diseases, respectively.
The characteristics of subjects in the GS and non-GS groups are shown in table 1. Both groups had similar mean±SD Logarithm of the Minimum Angle of Resolution visual acuity (logMAR VA) (0.32±0.37 vs 0.31±0.35, p=0.18) and IOP (18.1±3.8 vs 18.5±2.9, p=0.05), but there were significant differences in age, gender, race, duration of diabetes, history of hypertension and family history of glaucoma between the groups (all p<0.001).
Characteristics of glaucoma suspects (GS) and non-GS patients referred to SNEC
The average time from referral to a confirmed diagnosis of glaucoma was 21.5±10.2 months. Figure 2 shows the flow diagram from the referral to confirmed glaucoma at SNEC. At the last review, there were 451 of 5023 (9.0%; 95% CI 8.2% to 9.8%) diabetic subjects confirmed as glaucoma. Of the 451 cases, 369 (81.8%) cases were confirmed from the GS group, and 82 (18.2%) cases were confirmed from the non-GS group. The proportion of glaucoma confirmed from both the GS and non-GS groups was 14.1% and 3.4%, respectively.
Flow chart of diabetic subjects referred from Singapore’s Integrated Diabetic Retinopathy Programme to glaucoma confirmed in Singapore National Eye Centre (SMEC). EMR, electronic medical record.
Characteristics of glaucoma subjects
Table 2 shows the characteristics of glaucoma subjects from the study population. The mean±SD age of the glaucoma subjects was 66.8±10.8 years old and a mean duration of DM 7.8±0.9 years. There were more males (55.0%) and Chinese subjects (75.4%), 77.2% of them had a history of hypertension and 86.9% of them did not have a family history of glaucoma. The mean±SD VA of glaucoma subjects was 0.31±0.38.
Characteristics of glaucoma patients confirmed from the study population
POAG was the most common type of glaucoma in our study population (31.5%), followed by NTG (24.8%), PACG (24.4%), secondary glaucoma (12.4%), non-specified glaucoma (6.7%) and juvenile glaucoma (0.2%).
Comparison of screening performance in different GS features
Table 3 shows the sensitivity, specificity and PPV of the test using the current GS referral criteria. Glaucoma screening for VCDR≥0.65 alone showed a sensitivity of 70.5% (95% CI 66.1% to 74.6%) and a specificity of 55.1% (95% CI 53.7% to 56.6%), with a PPV of 13.4% (95% CI 12.7% to 14.2%). If the VCDR≥0.80 alone was used as the cut-off criterion for GS referral, the VCDR≥0.80 had a sensitivity of 7.7% (95% CI 5.4% to 10.6%) and a specificity of 99.1% (95% CI 96.8% to 99.4%), with the PPV of 46.1% (95% CI 35.5% to 57.0%).
Test performance based on Singapore’s Integrated Diabetic Retinopathy Programme glaucoma referral features on colour fundus photography
When other GF was included in the test, the sensitivity, specificity and PPV of VCDR≥0.65 and other GF were 81.8% (95% CI 77.9% to 85.3%), 50.7% (95% CI 49.2% to 52.1%) and 14.1% (95% CI 13.4% to 14.7%), respectively. When the test only included VCDR≥0.80 and other GF, the sensitivity, specificity and PPV were 11.3% (95% CI 8.5% to 14.6%), 93.9% (95% CI 93.1% to 94.5%) and 15.4% (95% CI 12.0% to 19.4%), respectively.
Discussion
This study found that 141 of 1000 patients with diabetes aged 18 and above who were referred as GS, and 34 of 1000 diabetic subjects who were referred as non-GS cases, were eventually confirmed as glaucoma. Overall, the prevalence of glaucoma in our study was 9.0% (95% CI 8.2% to 9.8%). The number was comparable to other studies conducted across the world, which ranged from 5% to 15%.19 20 The proportion of POAG (31.5%), NTG (24.8%), PACG (24.4%) and secondary glaucoma (12.4%) confirmed in our diabetic subjects was similar to the previous study conducted in the general population in Singapore.21
Using the existing GS referral criteria which included VCDR≥0.65 and other GF, the sensitivity was moderate (81.7%), but the specificity was low (50.7%), and a PPV of 14.0%, resulting in much more high false positives than true positives (2257 vs 368 cases). A high false positive referral rate will result in unnecessary referrals to the tertiary eye centres, which are already burdened with a high patient load, and has financial implication on the patient and healthcare system.
When we assessed the screening performance on different referral criteria, none of the GS referral criteria demonstrated a good balance in sensitivity, specificity and PPV due to the low prevalence of glaucoma (see table 3). Although the specificity of the test increased up to 93.9% for patients with VCDR≥0.80 and other GF, the sensitivity dropped to 11.2% with a similar PPV (15.4%). This reflects the fact that the diagnostic accuracy using glaucomatous appearance of the optic disc for screening is limited, as a large cup in a large disc may not be pathologic,22 23 but a smaller VCDR may also carry a risk to develop glaucoma. Examples of eyes with GS that were identified as true and false-positive cases following a comprehensive examination in the clinic are shown in online supplemental material 2.
Supplemental material
Even though the primary purpose of the telemedicine DR screening programme is to detect DR, the GS cannot be ignored during fundus grading. Therefore, it is important to develop strategies to reduce false-positive cases. Some studies showed that examining IOP may increase the opportunity to detect glaucoma because that patients with diabetes tend to have higher IOP than patients with non-diabetes.10 24 However, our study only found 153 glaucoma subjects of 2625 GS (5.8%) had IOP higher than 21 mm Hg. Furthermore, Tielsch et al’s study demonstrated that IOP measurement has a low sensitivity of 38% and a specificity of >92% at detecting glaucoma.25T
Adding other screening tests such as visual field testing or optical coherence tomography may improve the accuracy of glaucoma diagnosis.26 27 However, these tests require specialised equipment, serial examination, patient reliability and accurate interpretation by glaucoma specialists. Community screening requires a rapid, simple and low-cost test, and therefore, these additional investigation modalities may not be cost-effective, particularly if the screening is performed in a population with a low prevalence of the disease.9 Frequent doubling technology is another diagnostic equipment that has the potential to screen for visual defect in glaucoma since the test is easy and quick, with a sensitivity of 85% and a specificity of 90% to detect early glaucoma.28 29 However, likely with any functional tests, it requires high reliability of patients performing the test.
Other alternative for improving screening accuracy is to have additional examination by optometrists30 31 or ophthalmologists32 33 in conjunction with multiple diagnostic tests before referring them to tertiary eye centres.27 Artificial intelligence has shown encouraging performance in diagnosing, treating and predicting the progression of glaucoma.34 35 Combining these strategies may potentially reduce a large number of false positive cases while also improving the balance of sensitivity and specificity, but their cost-effectiveness must be evaluated.
When a healthcare system is concerned about being overwhelmed with false positive cases but does not want to miss severe cases who are at a higher risk for vision loss at the initial opportunistic glaucoma findings, increasing the threshold of referral to larger VCDR≥0.80 could be considered. Our findings showed that increasing the referral threshold from VCDR of 0.65–0.8 could improve specificity up to 93.9% but at the expense of sensitivity, with glaucoma patients from lower VCDR group potentially missed at the initial screening. However, since our patients with diabetes are screened annually with graders required to reference prior optic disc photographs, using a higher VCDR referral criteria together with observing for disc changes in serial CFP, could be an effective strategy for identifying incidental glaucoma in patients from a lower VCDR group over time.
Our study has limitations. Our study may not be representative of all GS referrals from SIDRP in Singapore. The rest of the population is likely to have been seen at other tertiary institution or were non-compliant with their referral. In addition, the definition of glaucoma was limited to the diagnosis code, glaucoma medication and laser/surgery available on EMR. If the diagnosis codes or treatment were not properly updated in the system, it may potentially generate unintended outcomes.
The strength of our study is that this study is one of the largest community-based glaucoma screening in a telemedicine national programme for DR that has reported the diagnostic value for opportunistic findings of glaucoma cases. We reviewed patients’ diagnoses for 21.5±10.2 months, which may allow us to better determine the true positive of glaucoma cases. The data can help healthcare officials to have a better understanding of the implications of opportunistic glaucoma findings and make an informed decision in their DR screening programme.
In summary, our opportunistic screening for glaucoma in diabetic cohort using CFPs showed a moderate sensitivity if performed on the VCDR≥0.65 group, but yielded a high number of false positives due to the low prevalence of glaucoma. However, opportunistic case findings using a higher VCDR (0.80 and above) definition may be a more rational use of healthcare resources since it would provide a higher specificity, and annual CFP examination may still detect changes in VCDR in those with a VCDR below the cut-off at baseline. To balance the benefit of early detection in preventing glaucoma blindness with the systemic costs from false positives, additional screening strategies and cost-effectiveness analysis need to be established.
Data availability statement
Data are available on reasonable request.
Ethics statements
Patient consent for publication
Ethics approval
This study involves human participants and was approved by SingHealth Centralised Institutional Review Board (CIRB) R1398/84/2016. Participants gave informed consent to participate in the study before taking part.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Contributors RT and GT researched and analysed data, contributed to discussion, wrote, reviewed and edited the manuscript. KYCT, RH, NCT, HH, TW, TA, CYC, ELL, CST and H-TW reviewed and edited the manuscript. QXL contributed to data cleaning and analysis. TYW served as the scientific advisor and reviewed the manuscript. All authors approved the final version of the manuscript. GT is the guarantor of this work and, as such, had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Funding This study was supported by NMRC/TA/0055/2017, MOH-CSAINV21jun-005 and OFLCG/001/2017 from the Singapore National Medical Research Council (NMRC) and, the AIC/RPDD/SIDRP/SERI/FY2013/0018 (HSDP MOH) Funding for Singapore Integrated Diabetic Retinopathy Programme Roll-out, and MH 70:70/4-26 (MOH) Extension of SIDRP grant from Ministry of Health, Singapore.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.
Linked Articles
- Highlights from this issue