SUBJECTS AND METHODS

Study design

A two centre prospective evaluation study of 239 people with diabetes was carried out between December 2000 and July 2001. Subjects were recruited from the Oxford Eye Hospital diabetic retinopathy clinic, or the Bertram Diabetes Centre diabetic and eye clinic in Norwich. A patient information sheet was posted to all patients 1 week before, and informed consent was obtained at the time of their booked outpatient appointment. Ethics committee approval was obtained from the Oxford and Norwich ethics committees.

Screening process

All individuals with diabetes attending the above clinics were considered eligible for inclusion except if they were pregnant, under 18 years of age, known to have learning or significant physical disabilities, or unwell. On arrival, visual acuity was tested using a Snellen chart at 6 metres before dilating both pupils using one drop of tropicamide 1% and phenylephrine 2.5%, repeated not more than three times. In the Norwich clinic phenylephrine 2.5% was restricted to patients with blood pressure less than 180/90 and only given once. Patients were then reviewed and examined in the clinic by their ophthalmic or diabetological team as part of their booked outpatient visit. This was followed by an ophthalmic examination by indirect slit lamp biomicroscopy using a 78D lens and direct ophthalmoscopy, performed by an experienced ophthalmologist (PS). Patients then underwent two field mydriatic digital photography using a 45 degree Canon CR5 retinal camera in Oxford and Canon CR6 camera in Norwich. Both cameras had a Sony three chip video camera capturing an image of 768×568 pixel resolution at 24 bits colour depth. Patients then underwent seven standard field stereoscopic 35 mm slide photography using a Zeiss 30 degree retinal camera in Oxford and a Topcon 50X 35 degree retinal camera in Norwich.

Grading

All three methods of examination were graded independently. The ophthalmoscopic assessments were completed at the time of examination by PHS, the two field digital images by RM, and the seven field stereoscopic 35 mm slide photographs by the Retinopathy Grading Centre, London. The examiner (PHS) was masked regarding each subject’s history of diabetes mellitus, and findings of current or previous ophthalmoscopic examinations although he did, on occasion, have access to the patient’s current visual acuity measurement. The digital and film graders were masked regarding each subject’s history of diabetes mellitus, and findings of current or previous ophthalmoscopic examinations.

The first two methods were graded using the Gloucestershire grading form to define referable diabetic retinopathy (Table 1), and the seven field photographs were graded according to the Modified Airlie House final classification¹ with a comparison table developed for analysis (Table 2). Referable DR was defined as maculopathy, moderate to severe non-proliferative, proliferative and advanced retinopathy as defined by categories 3–6 on the Gloucestershire grading form and/or categories D–G in the comparison table.

Image quality for the two field digital was determined using the following criteria:

• Fully assessable—possible to see the small vessels of the temporal arcades with reasonable clarity

• Partially assessable—possible to see the large vessels of the temporal arcades with reasonable clarity

• Not assessable—the large vessels of the temporal arcades are blurred or more than one third of the picture is blurred unless sight threatening retinopathy is detected in the remainder.

The evaluation of the quality of the seven field images was performed to determine gradeability based on strict definitions of field definition, focus/clarity and stereoscopic effect as outlined in the Early Treatment Diabetic Retinopathy Study² (ETDRS) manual of procedures. Seven field sets, including those defined as “ungradeable” were regraded to include the presence of haemorrhage or exudates less than 1 disc diameter from the foveola and any proliferative or advanced retinopathy, identified in the available images. This was carried out to provide comparative data with the Gloucestershire grading procedures for these features.

Statistical methods

The study was designed to include 250 patients, 100 patients having referable retinopathy, in order to achieve a standard error in the estimate of sensitivity no wider than 4%; 150 patient controls were chosen in order to prevent bias in the ophthalmologist’s examination result and in the photographic grading of the seven field photographs. The ophthalmologist and the retinopathy grading centre did not know the percentage of referable retinopathy in the study population but RM, who graded the two field digital images was aware of this percentage, but not the identity of individual patients.

Data were entered onto an Excel spreadsheet and downloaded into SPSS version 10 for data analysis.

Sensitivity, specificity, together with their 95% confidence intervals and kappa values were calculated for the following comparisons:

• The ophthalmologist’s examination compared with seven field stereophotography

• Two field digital photography compared with seven field stereophotography

• Two field digital photography compared with the ophthalmologist’s examination.

Calculations were based on assessable images from the appropriate reference standard method. Unassessable images were then included in the sensitivity calculations when comparing other methods with the reference standard used.

RESULTS

Technical failure rate and image quality

Ophthalmoscopic examination was technically possible in all patients. Determination of retinal status was not possible in six eyes of three patients (1.3%) from the two field mydriatic digital images. A total of 151 eyes (31.6%) of seven field stereo photosets were technically unassessable using the strict quality criteria, hence were not suitable for assignment of ETDRS retinopathy level. However, when the criteria were supplemented by the additional assessment of lesions lying within one disc diameter of the foveola or presence of proliferative retinopathy, the technical failure rate reduced to 73 eyes (15.3%). All of the six eyes that were unassessable on two field digital photography had assessable seven field stereophotography using the above criteria. Five out of the six eyes showed referable features both on seven field stereophotography and the ophthalmologist’s examination.

The reasons that 73 eyes were unassessable when grading the seven field stereophotographs was:

• 13 eyes—fields 1 and/or 2 were of insufficient quality for assessment

• 50 eyes—fields 3–7 were of insufficient quality for assessment

• 5 eyes—fields 1 or 2 and fields 3–7 were of insufficient quality for assessment

• 4 eyes—images were too dark for assessment

• 1 eye—images were absent

Detection of referable diabetic retinopathy

Comparison of the two reference standard methods, an ophthalmologist’s examination against seven field stereo photography, in assessable eyes (Table 3) gave a sensitivity of 87.4% (confidence interval 83.5 to 91.5) and a specificity of 94.9% (CI 91.5 to 98.3). The measure of agreement was a kappa value of 0.80.

These calculations were based on 405 eyes that had assessable seven field stereophotogaphs.

A more detailed analysis of this result, which includes the 73 eyes that were not assessable using seven field stereo photography is shown in Table 4.

Comparison of the two field digital photography against seven field stereophotography, in assessable eyes (Table 5) gave a sensitivity of 80.2% (CI 75.2 to 85.2) and a specificity of 96.2% (CI 93.2 to 99.2). The measure of agreement was a kappa value of 0.73.

These calculations were based on 399 eyes that were gradable by both methods of examination.

Comparison of the two field digital photography against the ophthalmologist’s examination findings, in assessable eyes (Table 6) gave a sensitivity of 82.8% (CI 78.0 to 87.6) and a specificity of 92.9% (CI 89.6 to 96.2). The measure of agreement was a kappa value of 0.76.

These calculations were based on 472 eyes that were gradable by both methods of examination.

Comparison of the two reference standard methods with the ophthalmologist’s examination as the main reference standard, in assessable eyes (Table 2) gave a sensitivity of 96.4% (CI 94.0 to 98.8) and a specificity of 82.9% (CI 77.4 to 88.4). The measure of agreement was a kappa value of 0.80.

These calculations were based on 405 eyes that had assessable seven field stereo photographs.

Retrospective examination of the seven field stereophotographs in which there was a difference in grading/classification between the two reference standards

There were eight eyes which the ophthalmologist’s examination had classified as referable retinopathy that had been graded as non-referable by seven field stereophotography.

On looking retrospectively at the photographs of these eight eyes five eyes were classified by the ophthalmologist as referable in whom the appropriate abnormalities were detected retrospectively on the photographs—four with maculopathy and one with referable non-proliferative retinopathy.

There were three patients who had abnormalities noted by the ophthalmologist in the superior retina, two with new vessels and one with IRMA that were not detected retrospectively on the photographs, as these features lay outside the standard fields.

There were 31 eyes that had been graded as referable retinopathy by seven field stereophotography and the ophthalmologist’s examination had classified as non-referable.

Of these, 12 eyes had been graded as having IRMA present on seven field stereophotography. Of these 12 eyes, six had received panretinal photocoagulation and the IRMA had therefore been graded as present between panretinal photocoagulation scars. Of these six eyes, IRMA were retrospectively confirmed on the photographs in five eyes. Of the six eyes that had not received panretinal photocoagulation IRMA were retrospectively confirmed in all six on one of the peripheral fields (two eyes field 6, two field 7, one field 5, and one field 3).

There were a further three eyes in whom the seven field stereophotographs had noted small fibrotic NVE following extensive panretinal photocoagulation (two in field 6 and one in field 3) that the ophthalmologist had classified as stable treated retinopathy (that is, non-referable). These were retrospectively confirmed from the photographs.

There were eight eyes that had been graded as having a haemorrhage <1DD from the foveal centre on seven field stereophotography. In all eight of these eyes the ophthalmologist had classified these as more than two microaneurysms <1DD from the foveal centre. Looking at the photographs retrospectively we encountered difficulties in interpretation of the ETDRS definition of haemorrhage/microaneurysm, which will be discussed below.

There were three eyes that had been graded as having a hard exudate <1DD from the foveal centre on seven field stereophotography. Retrospectively, examination of the photographs confirmed that hard exudate was present in one of these eyes, a single probable drusen was present in one and no hard exudate was seen in the third, although the image quality of this third image was only just within ETDRS standard 14.

There were two eyes that had been graded as having a group of exudates >1DD from the foveal centre on seven field stereophotography. In both of these patients the ophthalmologist had graded non-grouped exudate >1DD from the foveal centre.

Looking at the photographs retrospectively we encountered difficulties with the definition “group of exudates” as discussed below.

There was one eye that had been graded as having a multiple haemorrhage on seven field stereophotography. The ophthalmologist had classified this eye as >two haems >1DD from the foveal centre. Retrospectively it was agreed that multiple haemorrhages were present.

There was one eye that had been graded as having NVD present (ETDRS level 65a) on seven field stereophotography and classified as non-referable on the ophthalmologist’s examination. In retrospect no NVD were visible on the photographs and this patient should have been included within the non-referable group.

DISCUSSION

No previous studies have included a complete set of patients in whom a comparison has been made between seven field stereophotography and an ophthalmologist’s examination using slit lamp biomicroscopy, as illustrated in the examples below.

Comparison with other studies in the literature

Moss et al³ compared ophthalmoscopy with fundus photography in determining the severity of diabetic retinopathy. The ophthalmoscopy method was direct ophthalmoscopy supplemented by indirect ophthalmoscopy if the examiner felt this was necessary. Slit lamp biomicroscopy was not used.

Kinyoun et al⁴ compared indirect ophthalmoscopy by a retina specialist with seven standard field fundus photos but slit lamp biomicroscopy was not used.

Schachat et al⁵ compared the use of clinical examination and fundus photographs in detecting diabetic retinopathy in a population that included people with and without diabetes. Slit lamp biomicroscopy with a 78 dioptre lens, three mirror, or both was performed on 1168 individuals, of whom 9.5% had definite diabetes. The photographic method was two 30 degree fields using 35 mm film (Diabetic Retinopathy Study Standard fields 1 and 2 centred on the disc and macula). They concluded that the clinical examination detected most of the cases of diabetic retinopathy identified by disc and macular photographs read by skilled graders, although there would be an underestimate of prevalence using the clinical examination.

Pugh et al⁶ compared four screening methods in 352 subjects, which included an ophthalmologist’s examination through dilated pupils using direct and indirect ophthalmoscopy and seven field stereophotography. Two centres were used, 10 ophthalmologists performed the examinations, and one of the two centres used slit lamp biomicroscopy with a 90 dioptre lens. The number of subjects who had an examination that included slit lamp biomicroscopy was not stated, nor whether this affected the examination results. Sensitivity, specificity, and positive and negative likelihood ratios were calculated after dichotomising the retinopathy levels into none and mild non-proliferative versus moderate to severe non-proliferative and proliferative. Overall the examination results were poor with sensitivity of 0.33, specificity of 0.99, positive and negative likelihood ratios of 72 and 0.67. Of a total of seven cases of proliferative retinopathy, the ophthalmologist’s examination only detected three.

Reasons for differences between the ophthalmologist and seven field stereophotography in this study

A small number of differences were explained by errors being made by both reference methods.

Definitions of referable retinopathy accounted for a significant number of differences. Particular sources of difficulty were:

• Haemorrhage <1DD from the central fovea was a common source of confusion with microaneurysms <1DD often being graded instead of haemorrhage and vice versa

• The definition of a group of exudate >1DD from the central foveola needs to be clearly defined.

The ophthalmologist in this study differed from seven field stereophotography much more commonly in patients who had received extensive laser treatment. Although the grading form did not differentiate between IRMA in patients who had received panretinal photocoagulation and those who had not, he had not looked for IRMA in the former group. This was because he had considered the lack of new vessels to be a stable treated retina and not a referable eye. The grading form used did not allow for this difference. It perhaps illustrates the difference in performing studies to one’s routine clinical practice.

Performance of two field digital photography

Two field digital photography performed well with sensitivities of >80% and specificities of >92% against both reference standards. The technical failure rate was low at 1.5%.

Reasons for unassessable images in this study

During the clinical examination by PS, no patient was recorded as being unassessable. However, patients were excluded from the study by clinic doctors if they had media opacities. These might otherwise have been technical failures for the ophthalmologist.

A strict evaluation of the quality of the seven field images was performed to determine assessability based on field definition, focus/clarity and stereoscopic effect as outlined in the Early Treatment Diabetic Retinopathy Study (ETDRS) manual of procedures (ETDRS Chapter 18). However, seven field retinal photography in the ETDRS and similar major research trials was only carried out by extensively trained, certified, and constantly monitored photographers. Even within the context of such tightly controlled retinal photography protocols it is not unusual to experience cases where the seven field imaging fails to meet the required quality levels, with 10% technical failure rates being reported within WESDR.⁷ Indeed, authors suggested in the same article that use of such a relatively complicated and difficult protocol may not be entirely necessary and fewer retinal fields may be appropriate in the context of diabetic retinopathy imaging. Technical failure rates for studies that have used a seven field stereo protocol are not routinely reported in the literature, nor do studies report how many attempts were made to achieve an assessable seven field set of photographs.

Within the context of the current study the patients had a much higher prevalence of retinopathy than generally among people with diabetes, and consequently were more likely to have media opacities or to dilate poorly. In addition, the photographers in this study perform this technique relatively infrequently compared to the photographers in the Wisconsin studies,⁷ and only one chance was given to obtain a seven field set (no photographs were repeated). Hence, it proved difficult to obtain consistently high quality results in seven field stereo imaging, with approximately 15% being technically unassessable, even when applying less strict definitions for assessability.

A review of these technically unassessable images indicated that in the majority of cases, there were sufficient images present, and of sufficient photographic quality, for the presence of sight threatening retinopathy features to have been detected, should they have been present. seven field sets, including those defined as “ungradeable,” were therefore regraded to include the presence of haemorrhage or exudates less than one disc diameter from the foveola and any proliferative or advanced retinopathy identified in the available images. This was carried out to provide comparative data with the Gloucestershire grading procedures for these features.

CONCLUSION

The current study has shown that slit lamp biomicroscopy by an ophthalmologist, experienced in retinal examination, can compare favourably with seven field stereophotography as a reference standard when assessing different methods of screening for diabetic retinopathy. There are advantages and disadvantages with both reference methods. There is no hard copy for the ophthalmologist’s slit lamp biomicroscopy and one cannot necessarily conclude that this examination, with different ophthalmologists, will produce consistently high quality results. However, this paper has highlighted the high technical failure rate of seven field stereophotography, which even with the most experienced photographers⁷ can be 10%, and the technical failure rate for this procedure has not often been reported in previous literature.