Statistics from Altmetric.com
Treatment claims not supported by data
Recently Sabel, Kenkel, and Kasten co-authored a report showing that vision restoration therapy does not improve field defects in patients with cortical lesions.1 This finding was a disappointment because it dashed hopes that vision restoration therapy might benefit patients who suffer visual field loss from stroke, tumour, or trauma involving the occipital lobe. In a new twist, Sabel and colleagues have now written an editorial stating that “we have no objections to the data as presented” (an unusual remark from the co-authors of a study), followed in the next breath by a long argument repudiating the main thrust of their report.2 If readers are confused, they are not to blame.
Vision restoration therapy was described previously in a series of papers by Sabel and colleagues.3–8 In brief, it attempts to restore visual field defects by having patients practise perimetry every day at home using a software package loaded onto their personal computers. The idea is that repeated visual stimulation, especially just inside a scotoma boundary, can salvage neurons in damaged cortex at the fringe of a lesion. Sabel’s previous studies suffered from a major flaw: eye movements were not recorded or controlled. Patients with homonymous field loss often compensate by making surveillance saccades into their blind hemifield. Although Sabel and colleagues used the blind spot position to monitor fixation, they never reported fixation losses, false positives, or false negatives in their papers. Moreover, the blind spot position is an imperfect method for detecting small saccades, and useless for fixation assessment in an eye with a temporal hemianopia. For these reasons, most neuro-ophthalmologists were sceptical of Sabel’s claims for vision restoration therapy.
To his credit, Sabel responded by undertaking a collaborative study with scientists employing the scanning laser ophthalmoscope.1 This instrument allows one to present stimuli while monitoring fixation with great precision. Trials in which the patient sneaks a saccade can be discarded, solving the problem of fixation instability. Under these testing conditions, Sabel and co-workers found no improvement in the visual fields after vision restoration therapy.
Sabel had hoped that proper monitoring of fixation with the scanning laser ophthalmoscope would vindicate vision restoration therapy. Instead, he saw its apparent therapeutic benefit evaporate once the artefact of eye movements was eliminated. Rather than accept this negative outcome, he has written a commentary defending vision restoration therapy and criticising the methods used in his own paper.2 This is worthy of further comment, if only to highlight the inconsistencies behind this about face.
Sabel’s rebuttal relies on a paper in press elsewhere, showing that the same patients who failed to show improvement with the scanning laser ophthalmoscope did improve when tested with Tübingen automated perimetry and high resolution perimetry.9 It is difficult to comment on a paper that is still unpublished, but it should be recalled that Sabel has reported previously that patients with homonymous field loss do not improve after vision restoration therapy when testing is done with Tübingen automated perimetry.3 Once again, Sabel has placed himself in the position of refuting his own work. High resolution perimety refers to the technique used by Sabel to measure the visual fields before and after vision restoration therapy.3 Its drawback, as mentioned earlier, is poor control over eye movements. This deficiency was the reason for turning to the scanning laser ophthalmoscope in the first place.
Sabel’s data will remain uninterpretable until he adopts a technique that eliminates the artefact created by small saccades
In his editorial, Sabel reproduces a figure from his upcoming paper, comparing the fields before and after vision restoration therapy in a patient with a hemianopia.2 Before treatment, the scanning laser ophthalmoscope shows a field cut that deviates only about half a degree from the vertical meridian. This reflects the excellent control of eye movements afforded by that technique. The fields plotted by Tübingen automated perimetry and high resolution perimetry deviate by 1°–8° from the vertical meridian, yet they are measuring the same field defect defined by scanning laser ophthalmoscope, the gold standard. Immediately, this discrepancy should raise a warning flag. How can one define an improvement equal to only 2.5°–3.5° degrees azimuth, when one’s baseline measurement of the field defect is inaccurate by this amount? Sabel’s data will remain uninterpretable until he adopts a technique that eliminates the artefact created by small saccades. Parenthetically, it should be pointed out that the figure shows an unexplained discrepancy between Tübingen automated perimetry and high resolution perimetry in the elevation of the field defect before treatment.
After vision restoration therapy, Sabel shows a thin sliver of residual field loss that hugs the horizontal meridian, extending no further from the vertical meridian than it did before treatment. He asserts that such a change following treatment could not be produced by undetected horizontal saccades. In this single anecdotal case he is correct, but why rely on such an indirect, inferential approach to deal with the problem of fixation control? And in this particular case, why did the field cut improve dramatically in elevation but not azimuth? Anyone familiar with the visual field map in striate cortex will be puzzled by the pattern of field improvement attributed to vision restoration therapy in this case.
Sabel asserts that field testing using the scanning laser ophthalmoscope was difficult for patients, preventing vision restoration therapy from showing any benefit. It is true that patients had to report verbally their perception of three vertically aligned targets, rather than simply push a buzzer.1 However, there is no evidence that this requirement made their task more difficult or that it led to selective inaccuracy in the post-treatment assessment of their visual fields. It is also true that the scanning laser ophthalmoscope targets were dark against a bright background, to avoid light scatter. Sabel states that “Simultaneous stimulus discrimination and detection of negative stimuli on a bright background are probably tasks beyond the abilities of a damaged visual system.” In fact, cells in the visual cortex respond overall equally well to stimuli that are dark, rather than light, compared to background. The retina contains approximately equal numbers of on-centre and off-centre cells, and it is no harder for a subject to detect a dark spot than a light spot, provided the contrast is high.
The stimuli used with the scanning laser ophthalmoscope were large (0.33°) and high contrast, chosen deliberately to detect absolute scotomas (much like the V4e stimulus of the Goldmann perimeter). Sabel states that “The SLO method appears to be insensitive to relative defects describing areas with residual function as being absolutely blind.”2 In fact, the opposite is true. A technique that measures only absolute defects will characterise relative defects as normal. Sabel argues that the scanning laser ophthalmoscope missed regions of relative field depression that might have improved from vision restoration therapy. He forgets that with a cortical lesion, the first indication of recovery is provided by shrinkage of the absolute portion of the scotoma, even while the relative portion persists. To draw again an analogy with Goldmann perimetry, the V4e isoptre will sometimes expand in a recovering field defect, whereas the I2e isoptre will continue to show a defect. Thus, a technique that defines the patient’s absolute scotoma is the most sensitive to any potential improvement.
Sabel et al write that “Horton is concerned that vision restoration therapy improvements may simply be a result of placebo effect.”2 That is not quite an accurate paraphrase of my position. In my editorial, I noted that patients had the subjective impression that they had benefited from visual restoration therapy, despite lack of improvement in their fields.10 I attributed this discrepancy between negative field results and positive patient perception to a placebo effect. I expressed concern about using patient satisfaction as an outcome criterion, because patients will clamour for a treatment they believe works, even if it’s humbug.
Although neuroplasticity is active in many regions of the brain, this fact does not mean that vision restoration therapy can promote visual field recovery following lesions of the striate cortex. Sabel notes that “normal adult subjects are capable of perceptual learning, and there is an entire body of evidence on activity dependent use and neuroplasticity, such as studies on adult receptive field expansions following retinal or brain lesions.”2 These statements are true, yet when examined closely they are irrelevant to Sabel’s position.
Perceptual learning refers to the improvement in psychophysical performance that comes with practice. For example, anyone who takes a computerised visual field test a few times will show a slight improvement in retinal sensitivity. This phenomenon is well known, and must be taken into account when assessing the response to any proposed therapy, such as pressure lowering medications in glaucoma.11 For vision restoration therapy, perceptual learning is a confounding factor that must be controlled for by incorporating a placebo arm into studies. All subjects show a slight degree of improvement with practice, whether they have had vision restoration therapy or not.
Several investigators have reported that after retinal lesions (Sabel mistakenly refers to retinal or brain lesions), cells in the visual cortex become responsive to stimulation just outside the zone of retinal damage.12,13 The visual field does not improve, nor does the brain suffer a direct lesion. Individual cortical cells undergo expansion of their receptive fields, without any special therapy, simply because they have lost their normal input. Sabel advocates vision restoration therapy for an utterly different scenario: the restoration of lost visual field after a lesion that has injured the brain.
Sabel reminds us that the visual system is not purely sensory, because “it utilises many cognitive mechanisms as seen, for example, in the phenomenon of physiological blindspot “filling in.”2 It is unclear how the ability of the visual system to fill in blind areas is relevant here. Such areas remain blind, and subjects cannot detect visual stimulation. The fill-in phenomenon has nothing to do with the concept behind vision restoration therapy, and makes it no more plausible.
Sabel states that “the Food and Drug Administration has cleared vision restoration therapy to be offered in the United States and has done so in recognition of the results from the Tübingen-Magdeburg trial.”2 It is true that the FDA granted a 510 (k) clearance to NovaVision’s vision restoration therapy on 22 April 2003, in response to an application filed on 25 October 2002. A 510 (k) clearance is required before marketing certain types of new medical devices in the United States. The applicant must demonstrate that the device is “substantially equivalent” to a legally marketed device introduced previously. In this case, the predicate device was DynaVision 2000, a similar program for treating amblyopia. A 510 (k) clearance is not based on recognition of the results of a clinical trial, and Sabel’s application to the FDA did not rely on the Tübingen-Magdeburg trial. (Readers can judge the veracity of Sabel’s statement by inspecting his application at www.fda.gov/cdrh/pdf2/k023623.pdf)
Sabel asserts that “several clinical centres throughout the United States are now beginning to observe similar improvements with their first patients.”2 The NovaVision website features anecdotal case vignettes of patients who experienced huge recovery from scotomas, hardly representative of the mean 2.5°–4.9° improvement reported by Sabel in his studies.3,9 The website also announces that NovaVision has raised $20 million in venture capital funds to finance its expansion into the US marketplace. Sabel’s financial stake is undisclosed in all his publications.
The saga of Sabel’s visual restoration therapy provides a cautionary tale. An investigator proposes a new therapy for a condition that has no treatment. He adduces supporting evidence by carrying out a number of trials, but fails to control properly for a source of artefact. Meanwhile, he launches a commercial venture, based on his own research, and becomes financially involved. When his data are challenged, he agrees to an independent test of his therapy in collaboration with a third party. When the results prove him wrong, he rejects them. Meanwhile, trusting patients continue to sign up for the treatment programme, motivated by hope and the knowledge that nothing else is available.
Note in Proof
Treatment claims not supported by data