Article Text

Download PDFPDF

Validation of a computerised logMAR visual acuity measurement system (COMPlog): comparison with ETDRS and the electronic ETDRS testing algorithm in adults and amblyopic children
  1. D A H Laidlaw1,2,
  2. V Tailor2,
  3. N Shah1,
  4. S Atamian1,
  5. C Harcourt2
  1. 1
    Vitreo Retinal Unit, St Thomas’ Hospital, London, UK
  2. 2
    EEMU Maidstone General Hospital, Maidstone, UK
  1. Mr D A H Laidlaw, Consultant Vitreo Retinal Surgeon, St Thomas’ Hospital, Lambeth Palace Rd, London SE1 7EH, UK; allaidlaw{at}btinternet.com

Abstract

Background/Aim: The COMPlog clinical visual acuity measuring system is being developed for both routine and research use. This study aimed to validate its performance in amblyopic children and both normal and diseased adults against the gold standard ETDRS chart and the E-ETDRS computerised acuity measurement algorithm.

Method: Timed test and retest fully interpolated five letters per line logMAR visual acuity measurements were taken for 70 adults and 59 amblyopic children using the ETDRS chart and the COMPlog visual acuity measurement system. 39 of the adults also underwent computerised acuity testing using the E-ETDRS testing algorithm. The tested adults included normals as well as subjects with a range of ocular diseases. The methods of Bland and Altman were employed with test–retest variability (TRV) expressed as 95% confidence limits for agreement.

Results: No significant bias was observed between the gold standard ETDRS acuity measurements and those taken with either COMPlog or E-ETDRS. TRVs of ±0.12 logMAR and ±0.10 logMAR were respectively found for COMPlog measurements in the amblyopic children and adult groups compared with ±0.12 logMAR for the ETDRS chart in both groups. The TRV of the E-ETDRS system was slightly greater at ±0.16 logMAR. Median testing times for COMPlog and ETDRS were 95 and 85 s and 66 and 56 s respectively in the paediatric and adult groups and 120 s for the E-ETDRS measurements on adults.

Discussion: COMPlog measurements agree well with and are similarly reliable to the gold standard ETDRS chart with comparable test times. E-ETDRS algorithm measurements took approximately twice as long.

Statistics from Altmetric.com

The COMPlog computerised clinical visual acuity measuring system is being developed as an alternative to hard-copy acuity charts for routine and research use. Any new acuity test must be shown to perform well against the current gold-standard ETDRS chart.1 2 Contour interactions present in amblyopia may influence clinical acuity measurement; any letter test designed for clinical use should therefore be shown to be applicable to literate children as well as adults.3 The only other computerised acuity measurement system of which we are aware is the E-ETDRS algorithm running on the Electronic Visual Acuity tester.46 The aim of this method comparison study was to validate the performance of the COMPlog visual acuity measurement system using amblyopic children, normal adults, and adults with a range of acuity impairing ophthalmic diseases as subjects. Performance was determined in terms of (1) agreement (bias) with gold standard ETDRS acuity chart measurements, (2) by comparison of the test–retest variability (TRV) of COMPlog, ETDRS and E-ETDRS measurements and (3) test time for each measurement system.

PATIENTS AND METHODS

Fifty-nine children undergoing treatment for amblyopia were invited to participate, as were 70 adults with either normal or diseased eyes. Each subject underwent timed single letter scoring test and retest measurements of the visual acuity of their poorer acuity eye using ETDRS charts 1 and 2 and the COMPlog system (ie, four tests in total). The last 39 of the 70 adult subjects recruited also underwent computerised acuity measurements using the E-ETDRS algorithm. All tests were performed in random sequence in order to control for fatigue and learning effects. All measurements were conducted by one of three trained examiners (VT, SA, NS) under consistent lighting conditions. Subjects wore their habitual spectacle correction with their fellow eye occluded.7

Responses and computerised test scores were recorded on specifically designed proformata.

COMPlog is a computerised visual acuity measurement system consisting of a laptop PC capable of running Micrsoft Windows XP®, a 21 inch 1600×1200 resolution LCD flat panel secondary monitor and a software programme running within the Microsoft dotnet® framework. The examiner controls the test through a series of sequential control screens presented on the laptop monitor (fig 1): these enable collection of demographic data, control of various aspects of the testing algorithm (see below), response recording, and presentation of results in one of two logMAR formats (decimal logMAR and number of ETDRS letters) and one of three Snellen equivalent format scores (Decimal Snellen, UK Snellen and US Snellen).

Figure 1 COMPlog acuity measurement system.

The COMPlog testing algorithm consists of two phases, “range finding” and “thresholding”, both of which require forced choice responses from the subject and input of the response to each letter as correct or incorrect by a technician. Range finding aims to roughly identify threshold and consists of presentation of sequentially smaller single crowded Sloan letter steps starting from 0.8 logMAR, with a step size of 0.2 logMAR. If the 0.8 logMAR letter cannot be correctly identified, then sequentially larger letters are presented. The thresholding phase commences 0.4 logMAR larger than the range finding result, unless this exceeds 1.2 logMAR in which case it commences at 1.6 logMAR. The thresholding phase consists of presentation of sequential lines of randomly chosen Sloan letters surrounded by a crowding box. No letters are repeated on any line. The line size increment employed in the thesholding phase is set at 0.1 log units. The other aspects of the test are however user-controllable: these include letter spacing, the presence or absence of crowding bars, the number of letters per line, number of letters per line incorrectly identified to terminate the test and number of tests upon which the score is based. In this study, five letters per line spaced half a letter width apart and surrounded at the same separation by a crowding box of one stroke width thickness (fig 1) were employed. The termination criterion was set at all five letters on one line. In the event of letters being incorrectly identified on the first thresholding line sequentially larger lines are presented until an entire line is correctly read, with the programme then descending to threshold, but only presenting lines of each size of letters once. If all five letters cannot be fitted simultaneously on to the secondary monitor, the line is broken up into fractions with as many letters as possible of each size being presented. Letters larger than 1.6 logMAR are not presented, if a patient is unable to correctly identify any letters of 1.6 logMAR size, the programme invites scoring on a count fingers, hand movements, perception of light and no perception of light scale. In this way, with a 21 inch monitor and a single viewing distance of 3 m, an acuity range of up to 1.68 logMAR (one ETDRS letter, 1.5/71 UK Snellen or 20/957 US Snellen) may be measured without moving the patient.

Once the termination criterion has been met, the test automatically terminates with calculation and presentation of a fully interpolated logMAR acuity score and its Snellen equivalent in the selected formats, along with the total test time.

The COMPlog programme was also adapted to run the E-ETDRS test algorithm.46 This test measures a five letters per line fully interpolated logMAR acuity and also employs 0.1 logMAR line size increments but does so by presenting one letter at a time in a surrounding crowding box spaced half a letter width from the test letter.

ETDRS charts 1 and 2 were used and displayed in the standard Lighthouse® Low Vision Products light box.2 The ETDRS charts were read from a distance of 4 m unless the subject misnamed any letters on the top line. In this event, the subject was moved to 1 m, and the remainder of the measurements taken at this distance with 0.6 logMAR being added to the score.

The end point for all tests employed in this study was defined as an entire line of letters being misread. A fully interpolated logMAR acuity score was calculated for each test result as previously described.810

STATISTICAL ANALYSIS

The methods of Bland and Altman11 were used to quantify (1) bias (mean and 95% confidence interval of the mean) between ETDRS logMAR and both COMPlog and E-ETDRS algorithm scores (2) test–retest variability (TRV) expressed as 95% confidence limits for agreement (mean±2SD) for the paired ETDRS, COMPlog and E-ETDRS algorithm scores. Time scores were compared using paired t tests.

RESULTS

Fifty-nine amblyopic children (37 male) aged between 5 and 10 years (mean age 7.3 years) participated in the study. Twenty-two subjects had strabismic amblyopia, 15 had anisometropic amblyopia, 17 had a combination of strabismic and anisometropic amblyopia, and five had various types of meridional amblyopia. The median visual acuity of the tested eyes was 0.56 logMAR range 0.00 to 0.96 logMAR (UK Snellen equivalents, 6/22, 6/6 to 6/55).

Seventy adult patients (39 male) aged 23–81 years (median age 65 years) were also recruited. Their tested eyes had had a median acuity of 0.32 logMAR, range −0.12 to 1.30 logMAR (UK Snellen equivalents, 6/13, 6/4.6 to 3/60). Eight were normal, six had cataract or corneal disease, 27 surgical retinal disease and three glaucoma or other optic neuropathy. Mixed pathology was present in 26.

The COMPlog secondary monitor had a screen luminance of 41 cd/m2 and the contrast of letters was measured to be 94%.

Histograms of the distribution of the test–retest and between test acuity variability data suggested that the data sets conformed reasonably to a normal distribution. Scatter plots of the observed test–retest variation plotted against the average of the test and retest measurements suggested that there were no systematic association between TRV and the underlying acuities (fig 2). The parametric methods of Bland and Altman were therefore employed. The results of the studies are presented in tables 1–3.

Figure 2 Scatter plot of the difference between COMPlog test and retest logMAR acuity measurements measurements in 59 children (circles) and 70 adults (squares) plotted against the mean of the test and retest measurements.
Table 1 COMPlog and ETDRS intertest and test–retest agreement: results from 59 amblyopic children
Table 2 COMPlog agreement with ETDRS measurements: results from 70 adults
Table 3 E-ETDRS algorithm and COMPlog agreement with ETDRS measurements: results from 39 adults

In the paediatric study, the median (range) test times for ETDRS logMAR and COMPlog measurements were respectively 85 (40–251) s and 92 (38–233) s (p = 0.52). The corresponding median test times in the adult study for ETDRS and COMPlog acuity tests were 56 (range 17–148) and 66 (range 27–275) s (p<0.001) and for E-ETDRS 120 (range 60–300) s (p<0.001).

DISCUSSION

The aim of these studies was to validate the performance of a computerised visual acuity measurement system which we have named COMPlog. There are many potential advantages of such a computerised visual acuity measurement system: These include:

  • controllable and standardised measurement rigor, thereby minimising training and technician based errors;

  • automated score calculation and presentation in any of the routinely encountered acuity formats (decimal, UK and US Snellen; decimal logMAR and number of ETDRS letters read);

  • random-letter generation avoiding memorisation effects;

  • measurement from one testing distance of 6/3 to 1/60 acuity.

COMPlog employs the Sloan letter set and follows the standard ETDRS chart design format of one stroke width components and a 5×5 stroke width design grid.1 2 The between letter spacing and the separation of the letters to the surrounding crowding box however is based on half a letter width rather than the full letter width separation on an ETDRS chart. We have previously shown in both adults and amblyopic children that measurements made on a hard-copy logMAR chart using half letter width letter, line and crowding bar separation agree well and are comparably reliable to the gold standard ETDRS chart.3 7 A notable difference is however that these charts presented multiple lines of letters rather than a single line in a crowding box.

With the use of a 1600×1200 pixel 21 inch monitor and a viewing distance of 3 m, it is possible to present a single letter in a crowding box of 1.6 logMAR size, this is equivalent to the size of the top line of an ETDRS chart viewed from 1 m. This means that the COMPlog system is potentially able to measure the entire range of clinically encountered acuities from one test distance, as opposed to the two test distances usually employed with hard-copy charts. It also means that an acuity of Count Fingers (CF) is known to be worse than one ETDRS letter or 1.68 logMAR (6/288 Snellen); in routine clinical practice, CF can mean anything from worse than 6/60 to worse than 1/60 depending on the rigor with which the test was performed.

The data presented here are the results of the first of a series of experiments which aim to rigorously determine and document the performance of the COMPlog visual acuity measurement system. The ETDRS chart has been used for comparison purposes, as it is the current gold standard and the E-ETDRS algorithm employed as the only other available computerised acuity measurement system of which we are aware. Limited performance data from one other computerised acuity measurement system have been published. This system is no longer available and was limited to measuring acuity of 0.6 logMAR (Snellen 6/24 or better from its designed test distance.12

The results of these studies have shown that COMPlog measurements agree well (ie, show no evidence of bias) when compared with the ETDRS chart when measuring the acuity of the defective eye of amblyopic children and in adults with widely ranging acuity and ocular pathology. This suggests that the method of presenting a single line of letters spaced half a letter width apart and surrounded by a crowding box does not result in either adverse or beneficial contour interaction. COMPlog also differs from the ETDRS chart in terms of letter selection, screen luminance and letter contrast: COMPlog letters are randomly selected, with the exception that no repeats occur on any line, whereas the combinations of letters on each line of an ETDRS chart were deliberately chosen in an attempt to ensure equal legibility of each line.2 COMPlog screen luminance and letter contrast are also both lower than that on the ETDRS chart (35 cd/m2 vs 111 cd/m2 and 92% vs 100%). The observed lack of bias between COMPlog and ETDRS measurements suggest that these differences do not exert a clinically important effect on acuity measurements.

The TRV of ETDRS measurements was found to be ±0.12 logMAR in both our paediatric amblyopia and adult groups. (The 39 patients reported in table 3 are a consecutive sub set of the 70 described in table 2.) These results are comparable with other studies on the repeatability of ETDRS measurements in which TRV values ranging between ±0.07 and ±0.20 logMAR have been reported.3 4 7 1318 In this respect our subjects appear to be similarly reliable to those reported elsewhere. The TRV of the five letters per line COMPlog measurements in the 59 paediatric amblyopes which we studied was found to be ±0.12 logMAR and ±0.10 logMAR in the 70 adults. This suggests that the TRV of COMPlog measurements might reasonably be considered to be at least comparable with those taken using ETDRS. The TRV of E ETDRS measurements was found to be ±0.16 compared with ±0.14 for ETDRS and ±0.10 for COMPlog in the 39 adults on whom all three test and retest measurements were taken. It was not our aim to test the significance of these differences.

Beck et al reported no bias when comparing EVA E-ETDRS algorithm measurements to hard-copy ETDRS chart measurements in 265 adults.4 Two further validation studies have been carried out using the EVA and E-ETDRS algorithm on children, 100/311 of whom had amblyopia.5 6 Neither study documents agreement between hard-copy ETDRS and E-ETDRS measurements, so potentially important bias could conceivably have been present. The TRV of EVA E-ETDRS measurements in 265 adults reported was ±0.14 logMAR4 and between ±0.12 and ±0.14 logMAR in the paediatric subjects.5 These TRV data and those presented above are broadly comparable with published data for the TRV of the ETDRS chart in adults and children and provide further validation of computerised visual acuity testing devices.

A five-letters-per-line COMPlog measurement was found to take slightly longer than an ETDRS measurement (median test times 92 compared with 85 s and 66 compared with 56 s respectively) in the paediatric and adult populations. All test times include the ETDRS score calculation time; it is however our experience that the calculation of an ETDRS score may be prone to technician error. The adult time difference reaches statistical significance (p<0.001), but the paediatric does not (p = 0.52). The clinical significance of the 10 s time penalty of using COMPlog as opposed to ETDRS is up to the user to decide. E-ETDRS measurements were found to take nearly twice as long at a median of 120 s in the tested adult population. This is because the E-ETDRS algorithm presents a single crowded letter at a time (p<0.001). We believe that this might be considered a clinically significant time penalty. We are not aware of any other data on test times with the E-ETDRS algorithm.

COMPlog allows user control of testing rigor in terms of numbers of letters per line and termination criteria. The influence of these factors on testing time and TRV will be explored in future studies. In general, it might be reasonably expected that fewer letters per line and less stringent termination criteria would result in faster but less repeatable measurements.3 7

In summary, these studies have shown that COMPlog visual acuity measurements agree well, show a similar test–retest variability and take a similar length of time to the gold standard ETDRS chart. E-ETDRS measurements took approximately twice as long.

REFERENCES

Footnotes

  • Competing interests: None declared.

  • Ethics approval: Ethical approval for this study was granted.

  • Patient consent: Informed consent to participate was obtained.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.