Statistics from Altmetric.com
Editor,—We read with interest the article by Viswanathan et al.1
We were surprised by the authors’ progression criteria for theprogressor method; their criteria are extremely sensitive and very non-specific: only one point needs to show significantly negative slope when tested on the 5% level, together with an observed slope of <−1 dB/year and <−2 dB/year for inner and outer points, respectively. With 74 test points, it goes without saying that one here faces a problem with mass significance testing, which explains the very low specificity. It is obvious that almost any stable glaucoma patient would be detected as progressing with the reported method.
To illustrate the effect, we studied 10 patients with manifest glaucoma which, after at least 5 year follow up studies and at least eight visual fields, did not show visual field progression on the same clinical grounds as those used by Viswanathan et al. We had used same type of visual field test as the authors—that is, Humphrey 30-2 full threshold. In order to eliminate the possibility of any significant subclinical progression, the order of field tests were randomised before tests were analysed by the progressormethod. This randomisation secures that all significant progression would be the results of pure chance. Using the progression criteria of Viswanathan et al, stated above, five visual fields showed progression at the fourth visit, and already at the sixth visit eight eyes (80%) had shown false progression at one or more visits.
Further, we performed a Monte Carlo simulation study with the same criteria: 10 000 fields were simulated and tested 10 times and the regression slope was tested at each point starting at the third test occasion. The field was divided into two regions with 50 inner points and 24 outer points, respectively, as stated by Viswanathan et al, and the field generated by Gaussian variables with one homogeneous random component common to all points in the region (SD 1.1 dB), and with an independent inhomogeneous variation (SD 0.9 dB) between points.2 All fields were assumed to be stable, with constant mean at all test occasions. We assumed three tests to be performed each year. With the criteria of Viswanathan et al, 60% of the simulated fields showed significant progression in at least one inner point and 29% showed progression in at least one outer point. This means that at least 72% showed false significant progression on at least one occasion. In fact, 35% of the simulated fields showed progression on two consecutive test occasions. If we had included the short term fluctuation, the number of false significances would have been even larger. This finding of 72% false significances is in agreement with our experience with the 10 glaucoma patients.
Viswanathan et al must have been fooled by their own selection of progressing patients, and the use of remaining statistical significance at the last visit as a criterion for reliability. Naturally, this is erroneous. It is not surprising that many points in subjectively progressing visual field tests show significant progression at the end of the follow up period, and this yields no information as to the specificity of the analyses employed.
It is generally and correctly considered that, because of the large random variability of glaucomatous visual fields, real progression can seldom be demonstrated in glaucomatous visual fields in only 1 year. In view of this we are quite satisfied that the mean time until progression was demonstrated with statpac 2 was more than 2 years. In fact, we recommend stricter progression criteria forstatpac 2 than those used by the authors, again in the interest of specificity.
Linear regression analysis has advantages in visual field follow up; one is that all available data are used. Such analyses can be applied both properly and improperly. Analyses that lack specificity are only misleading, however, and may even result in improper clinical management. A more thorough analysis of linear regression analyses for visual field follow up is needed, and we will address this subject in a future article.
Editor,—We thank Heijl et al for their interest in our paper. Their letter raises a number of issues, all of which are worthy of discussion.
Our paper was a continuation of previous work (ref 25 in our paper) comparing a new analysis program (progressor) with the widely available visual field analysis program statpac 2. In the earlier paper we showed that the two programs identified the same locations in the visual field as “progressing”. In this paper we showed that “progression” as we defined it occurred much earlier with progressor than statpac 2. As an unwelcome outcome of glaucoma management is (further) visual field loss, any approach looking for earlier detection of visual field loss seemed to have merit.
Drs Heijl, Bengtsson, and Lindgren criticise us on our methodology and approach. Specifically they express concern at the use of a significance level of p<0.05, suggest that our patient selection would bias the results, and express satisfaction that the mean time beforestatpac 2 can detect progression is more than 2 years.
We set the significance level for our study at the 5% level because this is the only significance level available with statpac2. With this in mind, we set out to compareprogressor and statpac 2 in as fair a trial as possible. Since there is no external “gold standard” for glaucomatous visual field progression it is impossible to measure the true specificity of any analysis. Our patient selection, which would have a bearing on the results, was from a cohort of patients with normal pressure glaucoma who had progression identified on clinical examination of a large series of visual fields. This group may well be unrepresentative of glaucoma patients as a whole but as examples of undoubted progression they allowed us to perform the study. We agree with Heijl et al that the 5% significance level is likely to result in overdiagnosis of progression. In fact, even using the 5% level progressor would be expected to produce less overdiagnosis than statpac 2 sinceprogressor contains a rate (that is, slope) criterion for progression which statpac 2 does not: this also mitigates against the “problem with mass significance testing” which “goes without saying”. When we first compared progressor withstatpac 2 (in terms of numbers of progressing points detected rather than speed of detection) we expected the 5% level to be an overly lax criterion for progressor: we were surprised to find that this criterion had to be relaxed still further to the 10% level in order for progressor to emulatestatpac 2.
It is not surprising that Heijl et al found a high rate of false progression using a rudimentary numerical simulation. This is to be expected when the 5% significance level is specified, and their simulation echoes research on true clinical data. The authors have seemingly fallen into the trap of equating statistical significance with clinical significance. Any change in the visual field must be placed in the context of the overall management of the patient (which will include consideration of intraocular pressure, optic disc features, current and previous therapy, together with the patient’s age and general medical condition) before the clinical implications may be assessed. Statistical significance in visual field change, no matter how it is obtained, does not itself equate to clinical significance. The subject of which progression criteria correspond most closely to “true” clinically observed worsening of glaucoma is currently under investigation.
Because setting the progression criteria at the p<0.05 level will provide a large number of false positive responses, we do not recommend the use of this level for clinical use. As progressor(unlike statpac 2) is not widely available the chance of “improper clinical management” is therefore negligible. This cannot be said to be the case for the widely available statpac 2 for the original criteria established by the authors (ref 20 in our paper) are considered by many as the gold standard for progression. We are interested that Heijl et al now set stricter criteria for progression than is currently available on statpac 2 and would be interested to learn what these might be as they have not, to our knowledge, appeared in print.
We are surprised that Heijl et al are “quite satisfied” with a detection time of more than 2 years forstatpac 2. This seems a nihilistic approach when a more sensitive analysis, such as progressor, coupled with clinical perception, offers the hope of earlier detection and treatment (if required) of an irreversibly blinding condition.
Finally, we are delighted that Heijl et al are engaged in “a more thorough analysis” of linear regression, particularly as they dismissed linear regression as “the best way to diagnose cataracts” at the open glaucoma meeting of the ICO in 1990. We await their results with interest.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.