AIMS To assess non-cycloplegic screening for amblyopia with the hand held Nikon Retinomax autorefractor in 3 year old kindergarten children.
METHODS 427 three year old children were examined in kindergarten with the Retinomax without cycloplegia. A gold standard was established in all children by two orthoptic examinations in kindergarten. If there were missing, abnormal, or inconsistent findings, children were referred for ophthalmological examination. If, by the ophthalmological examination, a new case of amblyopia requiring treatment was diagnosed, the gold standard was set “positive.”
RESULTS In 404 children the gold standard was obtained. 10 children (2.5%) had a “positive” gold standard of unknown and untreated amblyopia. Screening sensitivity was 0.80, specificity 0.58, accuracy 0.58, and the likelihood ratio 1.89.
CONCLUSION Non-cycloplegic refractive screening with the Retinomax led to many false positive referrals due to instrument myopia and “inconclusive” results. Hence specificity, accuracy, and the likelihood ratio were too low to conduct screening effectively.
Statistics from Altmetric.com
The Nikon Retinomax autorefractor is a monocular refractor, using a fogging technique. In several studies, this autorefractor was found to be a reliable instrument compared with other autorefractors and retinoscopy.1-3 Being a hand held, easy to use device, it was also tested for early screening for refractive errors in infants, in a preschool population at risk, and preschool children.4 5
Non-cycloplegic screening offers advantages since it increases compliance and participation rates, is more rapid, and avoids the side effects associated with cycloplegia.6 Some authors found the Retinomax useful for non-cycloplegic screening7 for refractive errors. However, other researchers did not,8and attributed this to instrument myopia, which is frequent in young children.
To examine the effectiveness of a screening device or procedure, it is desirable to choose a setting which will come close to the actual screening situation and to include different examiners. It is also preferable to compare the screening results directly with the target condition itself—that is, amblyopia and the need for treatment, and not just with risk factors like refractive anomalies.
Therefore, the purpose of this study was to assess non-cycloplegic screening for unknown and untreated amblyopia with the Nikon Retinomax, under realistic screening conditions in 3 year old kindergarten children, with several examiners.
Subjects and methods
The institutional board approved the study design, and the tenets of the Declaration of Helsinki were applied. Parents were asked for informed consent in writing. Local and regional kindergartens with at least five to ten 3 year old children were contacted systematically. Any 3 year old child (after the third, before the fourth birthday) was eligible in kindergartens whose management opted for a participation in the programme. To our knowledge, no parent refused participation in those kindergartens which opted for the programme, except for one mother whose child was already being treated for strabismus.
A succinct history of each child (known ocular diseases, wearing of glasses, current ophthalmological treatment) was collected from the parents on the consent form. Children were recruited and examined regardless of their history.
About 9% of the children who were enrolled could not be examined on the scheduled morning owing to absence (illness, vacation, or other); 427 children participated in 29 morning screening sessions in 29 kindergartens in 1999.
Refractive screening with the Nikon Retinomax autorefractor was carried out by five experienced orthoptists familiar with tabletop autorefractor measurements in children.
All orthoptists received training to operate the Retinomax refractor: (a) oral introduction to the device, its functions, and operating buttons; (b) one individual training session in the laboratory with trials in one to two test subjects, and explanation of the quick reference sheet; (c) 2 training sessions in two different kindergartens with 5–10 children on each occasion. There training cases were not included in the recruited study population.
The Retinomax device was operated in the “normal” measuring mode—that is, at least eight full measurements per eye were collected by the device sequentially, and the global refraction computed by the refractor from these eight measurements was used to decide on referral.
Orthoptists were explicitly instructed to watch for and avoid bias induced by accommodation: fluctuations of the spherical components, or myopic shifts during the measurement meant that orthoptists should prolong the measurement until a relaxation of the accommodation would occur, or the orthoptist would switch to the fellow eye and repeat the measurement afterwards.
Retinomax screening results were computed by a customised database application. “Referral” resulted if the spherical equivalent of either eye was outside of the interval −1D to +3D, or cylindric power >1.5D, or spherical equivalent anisometropia >1D.
If no reliable measurement in both eyes could be achieved because of bad fixation or accommodation, or if a child did not cooperate well enough, the screening result would be “inconclusive.” If the refractive measurement could be performed in both eyes, and was within the thresholds cited above, then the refractive screening result was rated “no referral.”
A second set of refraction thresholds was used,7 to test whether a threshold variation would improve the screening effectiveness: “referral” was chosen if the spherical refraction was outside −3D to +1.5D in either eye, cylindrical refraction = 2D, cylindrical anisometropia = 1.5D, or spherical anisometropia = 1.5 D. Otherwise, the same criteria were applied. This set of thresholds was labelled No 2 as opposed to the first set, labelled No 1.
Presence or absence of unknown and untreated amblyopia defined the gold standard, which was ascertained through a diagnostic procedure based on
double orthoptic examination, and if there were any abnormal or missing, or inconsistent findings
an additional ophthalmological examination.
The first orthoptic examination was performed directly after the refractive screening by the same orthoptist. The second orthoptic examination was performed several weeks to several months later by a different orthoptist.
Orthoptic gold standard examinations took place in kindergarten including monocular visual acuity testing with the Lea single symbol test for 3 metres (Precision Vision, Villa Park, IL, USA), eye alignment with the cover/uncover/alternate cover test, eye motility, and abnormal head posture.
The uncorrected pass visual acuity threshold was set at 0.8 (10/12.5) monocular visual acuity in both eyes, or at least 0.5 (10/20) in both eyes and less than two log lines difference between visual acuity of the right and left eye. Visual acuity testing of an eye was discontinued when 1.0 (10/10) was reached in that eye, to save time.
Otherwise, visual acuity testing criteria followed the recommendations of the manufacturer's user instructions.
Any abnormality in at least one orthoptic examination item in the first or second examination resulted in a referral; likewise, if in the second orthoptic examination, cooperation was so insufficient that findings could not be classified within the normal range, the child would be referred to an ophthalmologist.
Parents were given the choice to have their child examined as an outpatient by the strabismology department of the university eye clinic Tuebingen, or in the office of a practising ophthalmologist in their vicinity. About half of the ophthalmological examinations were provided by the strabismology department and the other half by office based ophthalmologists.
Thus, all children with any abnormal or missing results in the orthoptic screenings received an ophthalmological examination to complete the gold standard. In addition, ophthalmological examination reports from all children who were currently seen by an ophthalmologist were collected.
The ophthalmological criteria for a “positive” gold standard (amblyopia) were:
any newly administered patching therapy,
any newly administered spectacle therapy,
if the visual acuity was ⩽ 0.4 (20/50) in either eye, or
if the difference of visual acuity between right and left eye was ⩾ 2 log steps.
In other cases of spectacle prescription, the condition was not rated truly vision threatening in terms of permanent vision loss, and thus the gold standard was set “negative” for amblyopia.
Applying this procedure meant that any visual abnormality detected in kindergarten had to be confirmed by an ophthalmological examination.
EFFECTIVENESS OF RETINOMAX NON-CYCLOPLEGIC SCREENING
Retinomax screening “referrals” were compared with the gold standard for amblyopia. Sensitivity was calculated as the ratio of the number of subjects with a “positive” gold standard rated “referral” by the refractive screening, and the number of all subjects with a “positive” gold standard. Specificity was computed as the ratio of the number of the subjects with a “negative” gold standard rated as “no referral” by the refractive screening and the number of all subjects with a “negative” gold standard. Accuracy was computed as the sum of true positives and true negatives divided by the number of all subjects.
The likelihood ratio—that is, the ratio of the probability to screen a positive subject “referral,” and of the probability to screen a negative subject “referral” was computed as sensitivity/(1 − specificity).9
For 404 (95%) of all 427 children the gold standard could be obtained; 23 children for whom no gold standard could be obtained (mostly because the family moved away with unknown destination, rarely because parents refused ophthalmological examination), were excluded from the data analysis.
The mean age at the time of refractive screening was 42.7 months. According to the information collected from the parents 9.7% of the children were currently treated or followed by an ophthalmologist, and 2.2 % had glasses.
In 10 children, amblyopia was newly detected and treatment was started for the first time, which was equivalent to a prevalence of the target condition of 2.5%.
RETINOMAX SCREENING RESULTS
Using thresholds No 1, Retinomax screening was “referral” in 49 cases, “no referral” in 229 cases, and “inconclusive” in 126 cases, of 404 cases with gold standard. For the purpose of screening, “inconclusive” results were also counted as “referral.” This resulted in 175 cases being rated “referral,” of which 167 were false positive. Using thresholds No 2, Retinomax screening was “referral” in 37 cases, “inconclusive” in 126 cases, and “no referral” in 241 cases. Now 156 of 163 screening results rated “referral” or “inconclusive” were false positive.
Sensitivity, specificity, accuracy, and the likelihood ratio for the two threshold sets are summarised in Table 1.
Screening for refractive amblyopia with the Retinomax was reasonably sensitive for threshold set No 1, but not sufficiently specific to conduct screening effectively. The same conclusion could be drawn from the low accuracy and likelihood ratios.
WHY WAS NON-CYCLOPLEGIC REFRACTIVE SCREENING NOT SUITED TO PREDICT REFRACTIVE AMBLYOPIA EFFECTIVELY?
Non-cycloplegic refractive thresholds indicative of amblyopia, determined through studies, have not been established to date, and could not be established in this study either.
One difficulty when performing non-cycloplegic screening with a device like the Retinomax was that there was no safe end point criterion for a “successful examination,” unbiased by accommodation or unsteady fixation, which could easily be applied by different examiners. The main reason for the high percentage of false positive Retinomax screening results was “inconclusive” results: in many children there were unreliable results owing to badly controlled accommodation and/or fixation. One consequence was erroneous anisometropia.
Could a device which screens binocularly avoid such errors? This is supported by a study using the Topcon PR2000 Pediatric Refractor.10 Nevertheless, with that binocularly measuring instrument, and without cycloplegia, instrument myopia was still a concern.
Could another set of thresholds have led to a more favourable outcome? We have presented results for two sets, and both were unsatisfactory. Following this, one could examine a larger range of thresholds and try to find an optimised cut off, or prove that there was no better combination.
Our data indicate that other combinations could not improve the test properties so that they would come close to the gold standard with reasonable specificity. Nor is it likely that another combination of thresholds could compete with, for instance, orthoptic visual acuity screening. The main problem with the Retinomax screening results was the large number of “inconclusive” cases: these are likely to keep specificity down, no matter how the thresholds would be defined, because all “inconclusive” cases have to be referred, unless one were to accept a still lower sensitivity. If the thresholds were chosen so that sensitivity may rise considerably, then the specificity would inevitably decrease to even lower levels.
The findings of this study are different from those of two other studies. In one study,7 the Retinomax autorefractor was used in the “quick” measuring mode with one examiner in a hospital outpatient setting, and the screening results were not compared with the necessity of amblyopia treatment.
In the present study, aimed at realistic circumstances, with various examiners, the Retinomax “quick” measuring mode was not used because fluctuations of the spherical component could be judged much better in the “normal” measuring mode. This mode displayed up to eight valid measurements collected consecutively, and this gave a better end point for ending the refractive assessment. In the “quick” measuring mode, there would not have been an objective measure of the technical quality of the refractive measurement, and no end point. Another study which found non-cycloplegic refractive screening with the Retinomax effective was conducted in a population of 3–5 year old native Americans with a high prevalence of astigmatism and amblyopia, the results of which were not corrected for lower prevalences of amblyopia.4 Instrument myopia may have been less pronounced, and cooperation better in the children older than 3 years of age. Therefore, the authors' conclusion that visual acuity screening was less effective than non-cycloplegic refractive screening deserves further investigation.
We should like to emphasise that it was not because of the kindergarten setting that the portable autorefractor had a poor performance compared with the gold standard. On the contrary most children were, as a result of a group phenomenon, cooperative, if not very motivated, to participate in the visual acuity examinations, more so in the absence of their parents, and in well known surroundings and accompanied by the kindergarten staff. Nor was the handling or operation of the Retinomax to blame.
The authors rather believe that in this study, the Retinomax performed like other autorefractors when used without cycloplegia in children: some 3 year olds do not cooperate well, and readings should be interpreted with care and have to be confirmed by cycloplegic measurements.3 11
Nevertheless, non-cycloplegic refractive screening might be advantageous in younger children who cannot yet be tested for visual acuity. While refractive amblyopia is the most common type of amblyopia it is also the least vision threatening. Consequently, evidence that treatment of children younger than 3 years of age for refractive amblyopia is worthwhile, and cannot be delayed until the age of 3, would be required.
It is undeniable that there is some correlation betweencycloplegic refraction and the risk of amblyopia. Nevertheless, in the long run, a certain level of refractive error may entail amblyopia in one subject, but not in another. Further, one may question if there is enough evidence at all to treat children with refractive amblyopia at the age of 3.12 This assumption was the implicit basis for the present study, which defined amblyopia in 3 year old children as the target disease. The present study cannot address this important issue.
The main conclusions which can be drawn from this study are, firstly, that screening for refractive amblyopia should be compared with the visual impairment associated with amblyopia (age permitting). Secondly, if one wishes to screen for refractive amblyopia at the age of 3, non-cycloplegic screening via refractive findings with a monocular autorefractor like the Retinomax cannot be recommended.
We acknowledge Medical Faculty Tuebingen grant fortuene 447, grant from Ernst und Berta Grimmke Stiftung, Duesseldorf, donations from Trusetal Verbandstoffwerk, Schloss Holte-Stukenbrock, and from Carl Zeiss, Aalen.
We should like to thank the participating ophthalmologists of the Tuebingen and Reutlingen regions for providing their examination records: A Riester, F Lorch, A Stanowsky, L Floegel, W Dechant, E Bader, I Hirsch, S Boddin, E Ellinger, K Koenig, B Kreutzer, KP Krug, D Lemke, M Reichel, N Wenzel, H Engelke, A Fries-Woye, A Sperber, M Sperber-Szymankiewicz, R Mittelstaedt, C Pohl, I Schuster, D Seuffer-Schulze, G Stehberger, R Zevallos-Moell.
We should also like to thank the staff of the Strabismology Department for performing the outpatient gold standard examinations: V Herzau, K Siepmann, H Hettesheimer, D Müller, B Rapp, B Nüssle, E Oechsner.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.