Retinopathy of prematurity (ROP) is a retinal vascular disorder in premature infants and the leading cause of childhood blindness in the United States.1 Because effective treatment exists,2 3 focus has shifted towards ensuring that all patients at risk for development of ROP are screened.

The gold standard for diagnosis of ROP is binocular indirect ophthalmoscopy “by an ophthalmologist who has sufficient knowledge and experience to enable accurate identification of the location and sequential retinal changes of ROP.”4 Two recent events have precipitated increased utilisation of those who provide ROP screening—increase in the eligible screening population and a decrease in the willing screener population. The Joint Statement recommendations for ROP screening implemented the following changes: (1) increased the gestational age cut-off from 28 weeks to 30 weeks postmenstrual age, (2) lowered the initial screening examination to 31 weeks postmenstrual age or 4 weeks chronological age, whichever is later, and (3) defined termination criteria.4 The cumulative effect of these three changes is to increase the number of eligible infants and to prolong their screening period (earlier start, later termination), all of which increase the utilisation. The National Center for Vital Statistics reported that there were 4.1 million live births in the United States in 2004 (the most recent year available).5 Applying the old screening criteria, approximately 60 000 patients were eligible for screening, and the main driver was birth weight (<1500 g).6 Applying the new criteria, approximately 80 000 babies would be eligible for screening, and the main driver is gestational age (<30 weeks). This represents a 33% increase in eligible infants. Additionally, the American Academy of Ophthalmology commissioned a survey of its members who screen for ROP in 2006, which revealed that 23% of present screeners were going to discontinue ROP screening, largely because of financial issues (http://www.aao.org/publications/eyenet/200607/news.cf).7^–9

Telemedicine has the capability to alleviate these screening problems. By training nurses to take standardised wide-angle digital retinal images and allowing remote interpretation of these photographs by an ophthalmologist experienced in ROP, this technology may improve the quality, delivery, accessibility, and cost of ROP care.10^–22 Over the last decade, the sensitivity and specificity of this technology have been continually improving, with recent studies showing comparable outcomes between digital screening and screening with indirect ophthalmoscopy.10^–23

At Lucile Packard Children’s Hospital of Stanford University, the administration purchased 4 RetCam II (Clarity Medical Systems, Pleasanton, CA) for use in a telemedicine outreach initiative termed the Stanford University Network for Diagnosis of Retinopathy of Prematurity (SUNDROP). The 12-month results of the first two sites enrolled in the SUNDROP telemedicine programme are reported herein.

METHODS

Two outlying neonatal intensive care units (NICUs) were enrolled in the first year, and infants meeting published ROP screening criteria4 were screened using the SUNDROP Network protocol between 1 December 2005 and 30 November 2006.

NICU nurse training

All screening examinations were performed by a trained team of NICU nurses using the RetCam II (Clarity Medical Systems, Pleasanton, CA). The team consisted of two or three nurses, one in charge of obtaining the digital imaging per protocol (described below), and the remainder positioning the infant and monitoring vital signs. Clarity Medical Systems provided initial training in the use of the RetCam II with a certified ophthalmic photographer. Following a single training session, the nurses were allowed to obtain photographs on their own following orders from the supervising physician (DMM). No nurse had received previous training in any area of ophthalmic photography. Subsequent training was provided as needed by both the certified ophthalmic photographer and the physician (DMM).

Photography protocol

Pupillary dilation was achieved with 2.5% phenylephrine and 1% tropicamide 30–60 min prior to examination. Aspiration precautions consisted of discontinuing feedings 2 h pre- and postexamination. All examinations were performed with close cardiac and respiratory monitoring, and if the heart rate or oxygen saturation was unacceptably low, the examination was halted until the infant was deemed stable to continue.

A topical anaesthetic, 0.5% proparacaine, was instilled in each eye before examination. A sterile lid speculum was used to open the eye and provide adequate exposure for photography. Hydroxypropyl methylcellulose (2.5%) was used to couple the digital camera lens to the infant’s cornea. Digital images were taken by a trained NICU nurse and stored on the RetCam II computer hard drive. The goal of each imaging session was to obtain five clearly focused images in each eye using the 130° lens (fig 1): (1) optic nerve centred, (2) optic nerve superior, (3) optic nerve inferior, (4) optic nerve nasal and (5) optic nerve temporal. The frequency of imaging sessions was per the Joint Statement recommendations for ROP screening4 with imaging substituted for bedside indirect ophthalmoscopy (ie, Zone II, stage 2 ROP without plus disease would receive a weekly imaging session). Images were captured as necessary until they were deemed to be of adequate quality for examination. An iris shot in each eye was added later during the first year. In cases of inadequate exposure, artefact, poor visualisation of the periphery or lack of a complete standardised image set, repeat telemedicine evaluation was performed within 48 h.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1 Standard image set of 10 total images, five of the right eye and five of the left eye, in a normal exam.

Image transfer

The images were all uploaded to a thumb drive, transferred to a computer and sent via the internet using secure email; alternatively, images were transmitted by courier on DVD to the remote expert reader (DMM). Additionally, demographic data were transmitted by fax to the reader.

Outcomes

Outcomes included referral-warranted ROP (RW-ROP), need for treatment and anatomical outcomes. RW-ROP was defined as any Early Treatment Retinopathy of Prematurity Disease Type 2 or greater, threshold disease, any plus disease2 24 25 and any stage 4 or higher disease. Referral-warranted ROP initiated a bedside examination by the physician (DMM). All decisions to treat were based on bedside examination. All patients were screened within 1 week of discharge from the NICU by one physician (DMM). Patients were no longer screened by the SUNDROP protocol if they received laser or incisional surgery or if they were discharged from the NICU.

Statistical analysis

All data were analysed using statistical software (Excel, Microsoft, Seattle, WA). Sensitivity, specificity, positive predictive values (PPVs) and negative predictive values (NPVs) were calculated and used to assess the usefulness of the RetCam II photographic readings.

RESULTS

Demographics

A total of 42 infants (84 eyes) were enrolled, resulting in 131 unique examinations and 1315 unique images. The mean and median numbers of images per patient per examination were both 10 (range 2 to 21 images). The mean birth weight of the infants examined was 1162.1 g (range 570 to 2119 g), and the mean gestational age at birth was 28.5 weeks (range 24 to 32.4 weeks). See table 1 for baseline characteristics.

Referral-warranted ROP

Four infants were identified with RW-ROP. Each of these infants was examined at the bedside using binocular indirect ophthalmoscopy with scleral depression to confirm their diagnosis. Two of the four infants with RW-ROP received laser photocoagulation with complete disease regression. The two remaining infants did not meet treatment guidelines at the time of bedside examination and were continued to be followed by SUNDROP through the completion of the 12-month period. Following RW-ROP diagnosis, SUNDROP protocol screening was supplemented with bedside examination by the ophthalmologist (DMM).

Safety

No patient progressed to retinal detachment, retrolental mass, macular fold or other adverse anatomic outcomes. All ROP requiring treatments were successfully identified with the SUNDROP protocol. In cases of inadequate exposure, artefact, poor visualisation of the periphery or lack of a complete standardised image set, repeat RetCam II evaluation was performed within 48 h.

Analysis

ROP screening using the SUNDROP protocol has a sensitivity of 100% and a specificity of 95% in detecting referral-warranted ROP. The positive predictive value of digital photography was 50%, and the negative predictive value was 100%. See table 2 for clinical RetCam II findings.

DISCUSSION

The 12-month results for the SUNDROP telemedicine screening initiative for ROP are favourable with a sensitivity of 100% and a specificity of 95%. The NPV is 100%, while the PPV of telemedicine screening is 50%. These data summarily suggest that the SUNDROP telemedicine screening initiative for ROP is safe in terms of identifying clinically significant ROP with no cases of treatment warranted disease missed.

Telemedicine’s high sensitivity, expressing the probability that a patient with ROP requiring intervention will test positive, speaks to its suitability as a screening tool. The high specificity we report, indicating the probability that a child without RW-ROP will be screened as such, is useful in preventing significant over-referral and thus undermining the purpose of screening in the first instance. The predictive values we report for this study offer a related measure of telemedicine’s utility. Analogous to the sensitivity, the negative predictive value of 100%, in representing the percentage of individuals with negative test results who are correctly diagnosed, is perhaps most important for a screening test and is reassuring in predicting that any infant with ROP requiring medical treatment would be detected by telemedicine screening and ensured a recommendation for a binocular indirect ophthalmoscopic exam.

The foregoing reflects the SUNDROP initiative not as reconfirmation of telemedicine’s theoretical efficacy by numerous prior studies but rather as their clinical culmination. For this reason, a concurrent bedside indirect ophthalmoscopic exam was not performed, as this is precisely what telemedicine aspires to replace. This seemingly precarious approach is exonerated in the context of prior literature conclusively establishing its safety and necessitating the current study.

Several reports have previously examined the sensitivity and specificity of ROP screening using telemedicine. An early study by Yen et al compared the RetCam 120 system (an older model of the camera employed here) and indirect ophthalmoscopy at two specific postgestational time points, finding “insufficient” sensitivity (46–76%) but good specificity (95–100%) of the images.21 22 The low sensitivity the authors claim, however, is misleading as it was defined as predictive accuracy in determining which eyes would go on to prethreshold or threshold disease with the precise criteria for this evaluation not immediately apparent.21 22 A similar study by Schwartz et al, in its comparison of bedside indirect ophthalmoscopy with telemedicine screening, noted a sensitivity of 95%.17 More relevant to this study is that the authors note difficulties with the RetCam they used, precluding proper fundus screening. It is notable that the newer RetCam model (utilised in later studies, including this one) does not suffer from similar hindrances.17 Roth et al evaluated the RetCam 120 system against indirect ophthalmoscopy in detecting the presence and progression of any stage of ROP and plus disease, reporting 82.4% sensitivity and 93.8% specificity.16 However, when telemedicine was evaluated in its ability to identify treatment-warranted ROP disease, the authors note a sensitivity of 100% as any case of ROP missed was either stage 1 or stage 2 disease (always in zone 2 or 3) and in all cases regressed spontaneously on follow-up. A later study by Sommer et al screened 145 premature infants using the RetCam 120 and noted neither false positives nor, most importantly, false negatives.26 Similarly, Ells et al reported 100% sensitivity and 96% specificity11 of telemedicine for detecting RW-ROP. In related studies by Chiang et al,10 23 27 28 the authors note that using a diagnostic cut-off of “ROP requiring treatment” yielded a sensitivity of remote telemedicine diagnosis by three different readers of at least 85%10 with an inter-reader reliability (κ) of 0.74328 for stratifying the type of ROP. An important caveat is that none of the readers in the study had more than limited experience in performing ROP examinations (bedside or using telemedicine). This notwithstanding, it is interesting to note that as the diagnostic cut-off level for image interpretation was shifted to detection of type 2 prethreshold ROP or worse (from detection of “ROP requiring treatment”), the sensitivity of each reader using telemedicine increased to 100%.10 Also noteworthy is that in 87% of eyes, RW-ROP was diagnosed by digital photography before or at the same time as indirect ophthalmoscopy.11 Wu et al also report a sensitivity of 100% and specificity of 97.5% in detecting prethreshold and threshold ROP with an NPV of 100%.20 A related study, the PHOTO-ROP trial, is the only trial to date to offer a prospective, multicentre evaluation of telemedicine screening for comparison against the gold-standard bedside examination.29 A total of 51 patients with 612 photos were obtained.29 The results that have been presented at the American Academy of Ophthalmology Retina Subspecialty day indicate that the sensitivity was 100%, and the specificity was 97%, when compared with the ophthalmologist’s examination.30

Summarily, the above studies16 17 22 23 citing a seemingly less favourable sensitivity of telemedicine reflect only subtleties surrounding the definitions of what telemedicine is screening. Thus, in every study where expert evaluation of telemedicine acquired images (using the latest RetCam model) was evaluated against a bedside indirect ophthalmoscopic exam, the sensitivity of telemedicine was actually 100% for detecting disease necessitating treatment, despite variation in the efficacy of identifying milder degrees of ROP for which no intervention is indicated. Therefore, the SUNDROP initiative was implemented. The ideal sensitivity is somewhat unsurprising given that ROP that requires treatment usually indicates either zone 1 disease (which has been shown to be easily imaged by telemedicine) or plus disease (which is diagnosed in the posterior pole).2 Telemedicine’s technical limitations in visually accessing the peripheral retina largely underlie these circumstances, although the RetCam II can actually image all stages of ROP.17

The above draws attention to what constitutes the proper assay of sensitivity—the ability of telemedicine screening to identify any stage of ROP identifiable by a bedside examination or its ability to identify all cases of ROP warranting treatment. Here we favoured a latter leaning definition in adhering more strictly with the essence of a screening test—namely, its fundamental purpose in identifying disease which lies above a critical “action” threshold and, therefore, for which either intervention is mandated or which necessitates further evaluation (ie, by a “gold standard” diagnostic test).31 The data presented here as well as the literature preceding them indicate that telemedicine as applied to ROP is almost perfect when measured as a screening test by this standard. It is important to note that, assuming 100% sensitivity, selection of a critical threshold below treatment warranted disease would likely result in no additional benefit in outcomes but would result in suboptimal cost-effectiveness. A further confounding factor of ROP screening is its requirement by any NICU wishing to remain accredited despite its increasingly prohibitive cost.32 For community hospital NICUs without the mandated “by an ophthalmologist who has sufficient knowledge and experience to enable accurate identification of the location and sequential retinal changes of ROP”4 telemedicine may represent the only opportunity to remain in licensing compliance. Though exact information regarding the cost-effectiveness of telemedicine in the United States is elusive, a recent editorial has commented that the strictly financial cost of infant transportation, bedside paediatric retinal consultations and medicolegal settlement of a single infant becoming blind (from no screening being implemented) would far surpass the cost of the RetCam.33

Resistance to adopt telemedicine screening for ROP is perhaps based on the disconcerting transition away from an ophthalmologist evaluating a patient at their bedside. However, it is imperative to note that telemedicine has surpassed the safety standards of ROP screening by the aforementioned reports comparing it with a bedside examination and has proven entirely safe in this trial of its use as a sole ROP screening tool. Furthermore, telemedicine in this report is not being used to diagnose ROP requiring treatment, only to identify cases suspicious for ROP requiring treatment with confirmation still performed by the gold-standard bedside examination by a trained professional. The lack of an accompanying ophthalmological exam accompanying every telemedical evaluation, though useful in initial studies, was unnecessary to determine sensitivity and specificity in our current report as any ROP requiring treatment and missed by telemedical screening would have been identified during the bedside indirect ophthalmoscopic exam each patient received upon discharge.

Additionally, other disciplines have applied telemedicine strategies in the management of their patients following less impressive reports of its sensitivity in those medical circumstances. Such disciplines include cardiology (deeming 87–94% sensitivity and a negative predictive value of 75% as safe in the detection of paediatric heart disease)34 and even the evaluation of diabetic retinopathy which now utilises stereoscopic images as the standard for its detection and evaluation by trained readers.35

One aspect of this study is that all screening was done by a single Ophthalmologist (DMM) skilled in diagnosing ROP. As Ells et al note, future remote screeners will benefit from telemedicine-specific training as evaluators gain skills in “remote interpretation of digital ROP images during the real-time video presented on the computer screen during the actual examination.”11 In our trial, the lack of an accompanying indirect ophthalmoscopic exam may also have had the effect of both more closely simulating future use of telemedicine as a sole screening tool and enhancing sensitivity, as a remote screener was perhaps more likely to refer a case knowing there would be no accompanying bedside exam. It is notable that specificity of screening remained quite high despite this. Additionally, our use of telemedicine in this fashion permitted outlying community NICUs to retain their patients while still gaining access to quaternary care.17 The benefits include keeping the patient in the community, cost-savings from reduced transport of the patient and potential health benefits by avoiding unnecessary and potentially dangerous transport of premature infants.