What should policy makers do with systematic reviews that fail to find evidence of effectiveness for interventions currently provided in the NHS? Is no evidence of effectiveness sufficient evidence of no effectiveness? The systematic review process has made great progress in minimising the influence of personal bias in the selection and critical appraisal of evidence. By bringing together in one place and critically appraising all the relevant studies, these reviews can show the fragility of the evidence on which some current practice is based.

Policy makers in the NHS take the results of these reviews seriously and expect to be able to act on their findings. But clinicians argue that it is wrong to close down a service, which might be doing good, just because the research evidence of effectiveness is poor. The review of pre-school vision screening commissioned by the NHS Health Technology Assessment Programme1 raises this dilemma and presents policy makers with a difficult decision. This report’s most startling finding was that the reviewers could not identify any robust studies showing that amblyopia (the main target condition of pre-school vision screening programmes2) causes any problems to children or adults. One researcher who tried to find performance differences between amblyopic students (as opposed to people blinded in one eye in adulthood or blindfolded in one eye for the purpose of an experiment) and students with monocular refractive errors was not able to document disabilities that he considered likely to affect everyday life.3 Health professionals have always assumed that reduced vision in one eye must cause problems, and in occupational health services health professionals have been responsible for the development of policies that exclude people with amblyopia from certain occupations on these grounds. The best documented consequence of amblyopia, however, appears to be that blindness may ensue if the good eye becomes damaged or diseased in later life. The incidence of this problem has been reported in a single study from Finland,4which was not methodologically robust. The problem is certainly rare compared with childhood amblyopia, which has a prevalence of 1–2% and is unlikely to be sufficiently common to justify treating all childhood amblyopes. Surprised by these findings, the authors of the review went on to undertake a qualitative study that suggested that people who have suffered from amblyopia since childhood have difficulty in specifying any way this has affected them. It may, however, be difficult for people to perceive that they cannot perceive something of which they have no experience.

The second finding of the review was that it was not possible to identify trials of treatment for amblyopia that included a “no treatment” control group. The natural history of amblyopia has only been studied in an indirect way and for short periods, and the studies that have been done suggest that amblyopia may sometimes regress spontaneously. Given the lack of studies on the natural history of the condition, the results of observational studies of treatment impact (which show an average improvement of around two lines on a Snellen chart) and studies comparing one treatment regimen with another, cannot be interpreted with any confidence. The evidence for clinicians’ belief in the efficacy of amblyopia treatment therefore comes largely from animal studies in which amblyopia has been artificially created in kittens and monkeys. The pharmaceutical industry would never be allowed to market a drug that had not been thoroughly tested in humans. It seems unlikely that the public would, if they knew, approve of other types of interventions being done on their children on the basis of animal studies alone.

An incidental finding of the qualitative study was that many adults and children were clear that the treatment (wearing glasses and intermittent patching of the good eye) had had an important and negative impact on their quality of life as children and on the lives of their families.5 The children did not like wearing glasses or patches, and some parents had felt distressed at enforcing these on their children. They persisted for two reasons, however. Firstly, because they were concerned that if they did not their children’s sight might cause problems in later life and secondly, because they felt that good parents should do for their children what was prescribed by health professionals.

The review found one prospective controlled study and a small number of retrospective observational studies that confirmed beliefs that primary orthoptic screening is an efficient way of identifying children with this condition. In the face of the evidence that the condition may not be disabling, that there are no robust studies of treatment impact, and that the psychological and practical consequences of treatment could be a problem for families, this evidence may be redundant. This review set out to investigate the efficacy of screening and therefore concluded that screening is not effective, not because the screening programme cannot identify children efficiently, but because there is no evidence that treatment is either effective or necessary. It has therefore called into question the provision of orthoptic services as well as screening services and has presented policy makers with an added dilemma.

Although amblyopia is regarded as the main target condition for this programme, pre-school vision screening also identifies other conditions that are sufficiently common to warrant screening, for example, non-cosmetically obvious squints (phorias and microsquints) and refractive errors. Cosmetically disturbing squints are by definition obvious to the casual observer and therefore do not need to be screened for. Spectacle correction is effective in correcting refractive errors. There is, however, no clear evidence that pre-school children benefit from correction of the minor refractive errors that are common in childhood, and there is good evidence that children with these defects, who are old enough to make choices for themselves, often choose not to wear glasses.6 Glasses are sometimes prescribed to children with refractive errors to prevent the development of amblyopia or squint, but the reviewers found no controlled studies of any sort showing that this was effective at the age of 3 years. Phorias and microsquints are only treated if there is accompanying amblyopia or refractive error; if not, they are watched so that if the squint progresses an operation can be undertaken.

It may be that in the future more detailed studies will define and quantify difficulties suffered by people with amblyopia and other common childhood visual problems, and that properly conducted randomised controlled trials of treatment will show that these difficulties can be reduced by intervention. Such studies may alternatively show that the difficulties are indeed insignificant and that the detrimental impact of intervention on families and children is significant and important. In the meantime, the NHS is supporting an intervention for which there is no evidence base. This intervention consumes scarce resources, but more importantly it requires parents to enforce a potentially distressing treatment on their children, on specious premises.

The results of this report suggest that health authorities should stop purchasing pre-school vision screening services. An invitation for screening carries with it the implicit assumption that children will benefit from being identified and it is therefore unethical to screen in the absence of robust evidence of treatment impact.7 In the case of screening, “no evidence of effectiveness is sufficient evidence of no effectiveness.” The situation with regard to treatment services is slightly different. It is regarded as ethical for a clinician to treat someone to the best of his ability on the basis of his clinical experience. However, because amblyopia is asymptomatic, children with amblyopia rarely present spontaneously unless they also have another problem.

A decision to stop purchasing orthoptic services for the treatment of amblyopia would have serious consequences for orthoptists. Such a decision might also be difficult for parents, teachers, and other members of the public to accept after they have been told for many years how important it is to treat children with these problems. It would result in the dismantling of a service that could not be re-established quickly overnight. If policy makers do not feel able to take this step, a reasonable and ethical compromise could be that they insist that treatment be offered only after a clear statement of the limits of medical knowledge about the conditions, their treatment, and their side effects. Then parents could make up their own minds about whether the potential benefits are worth the problems that treatment causes.