Background: The Plusoptix Vision Screener (PVS) is a new non-cycloplegic videoretinoscopy autorefractor. Refractive accuracy may affect its performance as a screening tool.
Aims: Study 1: To determine the intra- and interobserver variability of PVS measurements. Study 2: To compare PVS measurements with gold-standard manual cycloplegic retinoscopy (MCR).
Methods: Study 1: PVS refraction of 103 children with mean (SD) age 5.5 (0.6) years by two observers. Study 2: PVS and MCR refraction of 126 children with mean (SD) age 5.5 (1.5) years, including 43 children with manifest strabismus ⩾5 PD, comparing mean spherical equivalent (MSE) and Jackson cross cylinders J0 and J45.
Results: Study 1: Repeatability coefficients (observer 1): MSE: 0.63 D, J0: 0.24 D, J45: 0.18 D; those of observer 2 were nearly identical. The mean difference (95% limits of agreement) between the two observers for MSE, J0 and J45 were, respectively, 0.03 (−0.62 to 0.68 D), −0.008 (−0.25 to 0.23 D) and 0.013 (−0.18 to 0.20) D. Study 2: MSE tended to be lower on PVS than MCR, with differences of up to 8.00 D. Less than 20% of values were within ±0.50 D of each other. Agreement was better for J0 and J45. Strabismus was associated with an odds ratio of 3.7 (95% CI 1.3 to 10.5) of the PVS failing to obtain a reading.
Conclusions: The PVS may underestimate children’s refractive error.
Statistics from Altmetric.com
Amblyopia is the most common visual deficit in children, and treatment implemented within the first years of life is highly effective.1–6 Vision screening programmes use a range of tests, and the ideal screening method, which is both sensitive and specific as well as cost-effective, still remains to be found.7 Automated photoretinoscopy has the advantage of being user- and subject-friendly, but the accuracy of refractive measurements in children has been questioned.8–10 A new autorefractor, the Plusoptix Vision Screener (Plusoptix GmbH, Nuremberg, Germany), has recently become available and is marketed as a vision screening tool for young children. First data have been published on its refractive performance compared with gold standard manual cycloplegic retinoscopy (MCR).11 12 However, these studies excluded children with defects of binocular vision and strabismus, or even children with high hypermetropia. In addition, refractive analysis was limited to direct comparison of spherical equivalent and cylinder power, which does not reflect the true blurring effect and optical relevance of aberrant measurements.
We set out to evaluate the intra- and interobserver repeatability of Plusoptix Vision Screener (PVS) measurements and to compare its measurements with those of MCR by experienced ophthalmologists in children attending our paediatric hospital eye service, including those with strabismus and high refractive errors. We used vector analysis to determine the optical significance of the difference between measurements and to allow mathematical comparison of refraction findings.13
The PVS performs real-time videoretinoscopy of reflected infrared light in three meridians. The refractive data provided as final output are the median value of six frames of the acquired video sequence on which the software recognises both pupils. We used the PVS in screening mode, configured with software version CR03 V.184.108.40.206. The local ethics committee approved the study, and all parents gave informed consent.
We conducted two studies: study 1 evaluated intra- and interobserver repeatability of PVS measurements, and study 2 compared PVS measurements with MCR. For the first study, we recruited 103 children with mean (SD) age of 5.5 (0.6) years seen as part of a community vision screening study. Fifty-one children (49.5%) were girls, and 52 (50.5%) boys. We analysed duplicate sets of PVS measurements obtained by two observers to determine inter- and intraobserver variability. The same two observers acquired all data. With 100 subjects and two repeated measurements per subject, the within-subject standard deviation (and repeatability coefficient) of the PVS would be estimated to ±14 relative to its estimate.
For study 2, we recruited 126 children with a mean (SD) age of 5.5 (1.5) years attending our hospital-based paediatric eye service. Sixty-two children were females (49.2%) and 64 (50.8%) male. One observer acquired one non-dilated (non-cycloplegic) PVS measurement. If the device failed to obtain a simultaneous reading from both eyes, we switched to sequential monocular measurement mode. The children then underwent funduscopy, full orthoptic and ophthalmic examination and MCR 30 min after instillation of G Cyclopentolate 1%. With 130 subjects, the limits of agreement would be about ±0.30 SD relative to their estimate.
Over the course of study 2, 10 observers operated the PVS (four ophthalmologists, two ophthalmic nurses, four orthoptists). Four ophthalmologists masked to the PVS measurements performed MCR and ophthalmic examinations. We recorded the refractive error of both eyes determined by PVS and MCR and angle of ocular misalignment (if present) on orthoptic assessment without glasses for distance and near in prism dioptres (PD). We classified any manifest deviation ⩾5 PD for distance and/or near as “strabismus present.”
From the primary refractive values, we calculated the mean spherical equivalent (MSE) = sphere+(cylinder/2), Jackson cross cylinder at axis 0° with power J0 = −cyl/2×cos(2×axis) and at axis 45° with power J45 = −cyl/2×sin(2×axis),13 and spherical equivalent anisometropia = |MSE(left eye)−MSE (right eye)|.
With the exception of the anisometropia calculation, we analysed data from right eyes only to avoid bias from enantiomorphism.14
In study 1, the repeatability of consecutive readings for the three variables of interest was examined with the repeatability coefficient. Agreement between observers was explored with the 95% limits of agreement, with the averages of consecutive readings taken by each researcher being implemented. The 95% limits of agreement method was also used in study 2, when J0 agreement between PVS and MCR was explored. Due to a clear relation seen between the difference and mean, agreement in J45 and MSE was explored in a non-parametric way.14 All presented analyses were performed with SPSS version 14.0.
Study 1: intraobserver repeatability
The repeatability coefficient for the MSE measurements of observer 1 was equal to 0.63 D, and that of observer 2, 0.64 D. Thus, 95% of consecutive MSE readings by the same observer would be expected to be within ±0.63 to 0.64 D of each other. For both observers, the repeatability coefficient for J0 was found to be equal to 0.24 D. For J45, it was 0.18 D for observer 1 and 0.19 D for observer 2.
The mean MSE difference between the two observers was equal to 0.03 D with 95% limits of agreement from −0.62 to 0.68 D, indicating that in 95% of cases the measurements of observer 1 would be expected to be from 0.62 D above to 0.68 D below those of observer 2 (fig 1). For J0 the mean difference was equal to −0.008 D with 95% limits of agreement from −0.25 to 0.23 D. For J45 the mean difference was equal to 0.013 D, and the 95% limits of agreement were equal to −0.18 to 0.20 D.
Study 2: comparison of PVS and MCR measurements
In 18 children the PVS was unable to obtain refractive data from one or both eyes in either binocularly simultaneous or sequential monocular mode. These cases were excluded from the comparison of refractive measurements, leaving 108 cases for analysis (table 1).
In the majority of cases, the MSE was lower on PVS than on MCR. The mean (SD) MSE on MCR was +2.7 (2.5) D, and the mean (SD) difference between PVS and MCR was 1.9 (1.8) D. Plotting the difference (MCR – PVS) against the average MSE readings revealed differences of −0.88 to 8.00 D (fig 2A). Only 21 (19.4%) of the MSE values were found to be within ±0.50 D of each other, and 38 (35.2%) within ±1.00 D.
Deriving the difference of the J0 readings (MCR – PVS), we found that the mean difference was equal to 0.125 D. On average, the PVS measurements were lower than the MCR measurements, with 95% limits of agreement from −0.67 to 0.92 D (fig 2B). The precision of the lower J0 limit was (−0.71 to −0.62 D) and the precision of the upper limit was (0.88 to 0.96 D). The majority (92/108 = 85.2%) of J45 readings were 0 on MCR, and those of the PVS between −0.50 and 0.50 D. Plotting the differences between the two methods against the average values (fig 2C) showed that 104 observations (96.3%) were between ±0.50 D of each other and 107 (99.1%) between ±1.00 D.
We analysed whether strabismus was associated with an increased incidence of the PVS failing to obtain a refraction reading. Indeed, we found that children with strabismus had a higher risk of being excluded from refractive comparison due to PVS failure to obtain a reading (odds ratio 3.7, 95% CI 1.3 to 10.5, p = 0.001). Of the 108 children in whom the device did acquire refractive readings, 32 had manifest strabismus ⩾5 PD for distance and/or near with a mean (SD) distance deviation of 20.4 (13.3) PD. As can be seen on fig 2A, the difference between MSE on PSV and MCR tended to be greater in the presence of strabismus. The mean (SD) difference in MSE between the two refraction methods was 2.75 (1.65) D in children with strabismus versus 1.59 (1.74) D for those without.
Lastly, as the PVS is intended for use as a screening tool, we analysed its performance in correctly identifying refractive errors potentially associated with amblyopia (table 2). In line with the above comparison of refractive data, the sensitivity in detecting hypermetropia >+3.00 D was low at 19% (95% CI 8.6 to 34.1%), while that for anisometropia >1.00 D was better at 50% (95% CI 30.6 to 69.4%).
The main aims of this study were to investigate intra- and interobserver variability of PVS readings and to compare its refractive measurements in children with those found on MCR. Study 1 delivered unexpected results. The PVS is an automated photorefractor which obtains readings when the acquired images are in focus and in alignment as determined by the device itself. We therefore did not expect to find significant variability between readings. The observed variability of readings acquired by the same observer may be related to the device itself or to fluctuations in the refractive state of the non-cyclopleged subject. Changes in refractive state by fluctuations in accommodation would not affect the astigmatic component. Indeed, J0 and J45 were less variable than MSE, supporting the notion that accommodation contributes to the observed intraobserver variability.
The subsequent observation of interobserver variability is then less surprising, as it is in the same range as the variability of subsequent measurements acquired by the same observer. These are useful figures for any future study of PVS performance.
Study 2 has two limitations. The main limitation relates to the number of observers involved. Ten observers operated the PVS, and four ophthalmologists performed MCR. However, the PVS measurement variability demonstrated above indicates that it only contributes to a small degree to the lack of agreement between PVS and MCR we observe. We did not establish the interobserver variability of MCR measurements, but all participating refractionists were experienced in prescribing refractive corrections for children, and previous studies indicate low degrees of interobserver variability for this test.15–18 The second limitation is the low number of myopic subjects included in this study, which reflects the distribution of refractive errors in our study population. We therefore cannot comment on PVS performance in myopia.
Since we began our investigation, three studies on the same autorefractor have been published, two of which have studied its refractive performance compared with MCR.11 12 19 Due to differences in study design, our work reveals information that was not included in the previous reports.
First, both previously published studies excluded children with ocular misalignment exceeding 5 PD11 or with any defect of binocularity.12 We found that in the presence of strabismus, the PVS is often not able to obtain a refractive measurement, either in binocularly simultaneous or in monocular sequential mode. However, when it does acquire refractive measurements, the difference between autorefractor and MCR values for MSE tends to be larger in strabismic than in non-strabismic subjects. In 32 children with strabismus, only three MSE values were within ±0.50 D of each other, and four within ±1.00 D. Although the number of subjects in this subgroup is small, users of the PVS might need to take into consideration that in the presence of strabismus, the device might still provide a refractive reading but that this reading is less likely to agree with MCR than in non-strabismic subjects.
Second, while our results concerning differences in MSE between PVS and MCR are similar to those reported by Ehrt et al, who studied a similar cohort of children attending a hospital eye clinic,11 they differ from a recent report by Erdurmus et al.12 Case selection may explain this difference, as Erdurmus et al included children up to 14 years of age, with a mean cohort (SD) age of 7.1 (2.4) years and a far narrower range of refractive errors (−1.50 to +3.88 D), with a mean (SD) of +0.90 (0.76) D on MCR. Accommodation in these subjects may have been lower than in our study cohort. This highlights the importance of viewing reports about device performance in the context of the population studied.20
Third, this is the first study to use vector analysis to analyse the agreement between the astigmatic component measurements.13 Our analysis confirms that the measurement agreement is better for the astigmatic than for the spherical component. Based on the limits of agreement analysis, the J0 vector determined by PVS would be expected to lie between −0.67 to 0.92 D of the MCR value in 95% of cases. Agreement for the J45, the Jackson cross cylinder at axis 45°, was even better, but as oblique astigmatism is rare, all J45 values were equal or close to 0.
The poor agreement in MSE between PVS and MCR is a cause for concern. Only 20% of values were within ±0.50 D of each other, and 35% within ±1.00 D. The PVS underestimated the refractive state by up to 8.00 D. This failure to measure hypermetropia is probably caused by subjects having to fixate onto a target at 1 m distance and exerting individually different degrees of accommodation in the process, a phenomenon called “fixation myopia” or “pseudomyopia”.10
Previous studies have attempted to overcome accommodation-induced measurement errors by cycloplegia (PowerRefractorII10) or the addition of +3.00 D lenses (Plusoptix Vision Screener11), but have found a loss of accuracy in the measurement of cylinder power and axis10 and loss of specificity in detecting hypermetropia.11
We attempted to improve PVS accuracy by cycloplegia in a subgroup of our subjects but found that the testability rates decreased dramatically (data not shown), that is the device was unable to give a refractive reading after cycloplegia even in cases where it had delivered a non-cycloplegic refraction result earlier. The PVS requires non-dilated pupils to determine the horizontal axis and fixation; hence, cycloplegia cannot be used. Another way of overcoming accommodation would be to ask subjects to fixate on a distance target. Future work is needed to determine whether this might improve the accuracy of the device.
In conclusion, the present study has determined the intra- and interobserver variability of PVS measurements and has found a lack of agreement mainly of the spherical component between PVS and MCR. Using distance fixation might overcome the current apparent source of error, that is accommodation onto the fixation target. This study shows the limitations of the PVS as a stand-alone screener to detect amblyogenic factors in this susceptible age group of children.
Competing interests: None.
Ethics approval: Ethics approval was provided by Suffolk Local Research Ethics Committee.
Patient consent: Obtained from the parents.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.