Cataract blindness is a major public health problem. As a result of population growth and ageing the number of people blind as a result of cataract is calculated to double in the next 20 years if services are not increased and improved. Cataract surgery has been demonstrated to be one of the most cost effective health interventions.¹ The cost effectiveness is related to short operation time, the potential for high volume, high success rates, and the low cost of consumables. However, when success rates are low, cost effectiveness is reduced.² Recent clinical trials^3–5 have demonstrated that cataract surgery can provide a good visual outcome (presenting visual acuity of 6/18 or better) in 90% or more of operated eyes. However, these trials usually exclude individuals with ocular co-morbidity, and are usually undertaken in tertiary institutions by highly trained surgeons. In contrast, population based surveys and rapid assessments conducted over the past decade, have shown that of all eyes operated for cataract, 21–53% had a presenting visual acuity of less than 6/60 (poor outcome).^6,7–11 A significant cause of poor outcome is uncorrected aphakia due to loss or breakage of spectacles. However, even in eyes receiving an intraocular lens (IOL) poor outcome is reported in 5–15%.

Outcome data from population based surveys may not do justice to recent improvements in surgical techniques, but they may well reflect what the public perceives, influencing their expectations of regaining sight if they have surgery. Several studies have demonstrated that fear of not regaining sight and fear for the operation are important barriers which prevent patients coming for surgery.^12,13 It is therefore imperative that, besides increasing the number of operations, the outcome of cataract operations should also be improved. If the causes of poor outcome are known through monitoring then it should be possible to address these causes and improve the results of surgery. This article describes a multicentre study to develop a simple instrument for monitoring cataract outcome in low and middle income settings. The goal is a system that can be used in a range of circumstances from non-ophthalmologists operating in remote rural areas to experienced ophthalmologists operating in training institutions.

The word “discharge” in this article is used as “discharge from inpatient care following cataract surgery.” The phrase “follow up visits” is used to indicate outpatient assessment at least 1 week after surgery.

In case of day care surgery, the postoperative assessment at the first or second postoperative day should be used as the “discharge” assessment. In many countries, follow up is poor, but all patients will have an assessment at discharge.

METHODS

Development of recording forms

In March 2000 eye surgeons from eight centres in Africa and Asia participated in a workshop to define the essential data needed to monitor results of cataract surgery. The centres were selected to represent a range of service providers—that is, outreach surgical services in rural Africa; district level eye units in Africa and Asia; and well equipped tertiary level training institutions in Africa and Asia. The output of these centres varied from 300 to 10 000 cataract operations per year (table 1).

It was agreed to focus on uncomplicated age related cataract, and to exclude from the monitoring process people younger than 40 years, those with traumatic cataract, and those having cataract surgery combined with another procedure. Drafts of a simple manual system (manual cataract surgical record form, MCSRF), and of a more detailed recording form for data entry and analysis (computerised cataract surgical record form, CCSRF) were developed by the participants over 2 days. Over the following 3 months a database was created for data entry, and software written for automated analysis of data.

Field testing recording forms

In June and July 2000, the principal investigator (HL) visited each centre that elected to evaluate the CCSRF. The objectives of the study were explained to all the staff, giving details of how to complete the recording form. The data entry software was installed, and a local data operator trained. After the visit further assistance was provided through email, if required. One centre decided after a few months to use the manual system only because they did not have any dedicated staff for data entry.

Each centre recorded patients having cataract surgery between June and December 2000. Follow up of patients continued into 2001. Data were regularly sent by email to the principal investigator for consistency checks and, in case of doubt, compared with the original case sheets. Contacts between the principal investigator and the centres were limited intentionally, in order to explore what problems the centres encountered and how they solved them.

Issues raised

During discussion at the workshop concerns were raised about (1) the workload entailed in recording information for all operated patients in high volume settings, and (2) the value of recording visual acuity within 1–2 days of surgery in settings where not all patients return for routine follow up. In some countries patients with poor results do not return to the place of surgery, but seek advice from other clinicians. It is not, therefore, possible to assume that those who do not return have a good outcome. All these concerns were developed into operational research studies, which were undertaken by the centres over the ensuing 6 months.

Operational research studies

The following operational research studies were undertaken:

• The first study was set up to determine whether results obtained using the two systems were comparable. Four centres agreed to collect data on the same patients using both systems.

• The second study was set up as a means of addressing the workload issue, by determining whether the findings in a randomly selected sample of 20% of patients having cataract surgery gave similar findings to all patients having cataract surgery. In four centres a random sample of 20% was selected at the time of surgery, by drawing a marble from a black bag in the operating theatre which contained five marbles, one of which was a different colour from the others. The recording form was only completed if the marble with a different colour was selected.

To address issues of follow up, two studies were undertaken.

• Five centres agreed to trace patients who failed to return for follow up and measure their visual acuity in their home. Visual acuities in those who did return for follow up were compared with those who did not.

• In a second study analysis was limited to patients with complete follow up data to compare visual acuity at discharge with visual acuity at various periods after surgery, to evaluate changes in visual acuity over time. Data from six centres were used for this purpose.

The participants met again in 2001 to evaluate the results and finalise the instrument based upon the experience gained.

RESULTS

Cataract surgery record forms

Versions of the manual and the computerised cataract surgery records used in the pilot studies are shown in figures 1 and 2. In each form the number of parameters has been limited to those identified as being essential. The following World Health Organization¹⁴ (table 2) definitions were used for the operated eye:

• Presenting vision: visual acuity with available correction, if any

• Best corrected vision: visual acuity with best possible correction

• Good outcome: can see VA 6/18

• Borderline outcome: cannot see 6/18, but can see 6/60

• Poor outcome: cannot see 6/60.

Causes of poor visual outcome following surgery can be recoded as follows:

• Selection: due to pre-existing concurrent eye disease

• Surgery: due to intraoperative or immediate postoperative complications

• Spectacles: due to inadequate optical correction

• Sequelae: due to late postoperative complications.

Software accompanying the CCSRF

Two computer packages were developed for the CCSRF—one is programmed in Epi-Info 6.04¹⁵ and runs under MS-DOS and Windows, the other is in Visual FoxPro 6.0 and runs in Windows only. The software generates reports selected through a menu. The user can select all records for analysis, or use filters to select a particular year, surgical centre, or surgeon. Reports can also be produced in chronological order by date of operation, giving the results from 100 consecutive cases, allowing the user to follow trends over time with meaningful percentages. Data can also be generated by month and year depending on local volume and requirements. The reports produce tables on:

1. operative complications: total and type of complication

2. percentage of good, borderline or poor outcome at discharge

3. causes of poor outcome (VA<6/60) at discharge

4. good, borderline or poor outcome at 4 weeks or more postoperatively

5. cause of poor outcome (VA<6/60) at 4 weeks or more postoperatively.

Information by age and sex; first and second eyes; existing ocular pathology; surgeon and place of surgery can also be accessed.

Operational research studies

(a) Comparison of MCSRF with CCSRF

One centre discontinued CCSRF in favour of the MCSRF. The other three centres used the computer system first and the manual system later as an addition. However, the data in the two systems differed considerably, with more data entry errors in the manual system. Motivation to correct the MCSRF was low because of the extra work.

(b) Comparison of a randomly selected sample with the whole study population

Three centres were unable to take a random sample for a variety of reasons. In the other study centre 122 patients were selected by “the marble in the bag” system, out of the total of 454 cases operated on (27%). There were no significant differences in outcome between the two groups.

(c) Comparison of findings in those attending follow up with non-attenders

Different centres had different findings. Centre 1 compared 117 non-attenders visited at home at least 6 months after surgery with 133 cases who returned for the first and/or second and third follow up visit. At more than 6 months postoperatively, the outcome data for the non-attenders did not differ significantly from the data for those who did attend.

Centre 6 reported that 69% of the traced patients had good visual outcome while only 33% of those who came voluntarily had good vision. However, the postoperative periods in the two groups differed. Similarly, centre 8 reported that follow up cases traced at home had better visual outcomes than those attending for follow up. In this case, the traced group was examined at a mean of 67 days after surgery compared with a mean of 24 days in the self reporting group. Differences in outcome could, therefore, be attributed to the longer postoperative period in patients traced in their homes. Centre 5 managed almost 100% follow up and, hence, no comparison was possible. Centre 7 could not conduct home visits because of civil unrest. In summary therefore, one centre had evidence that those returning for follow up are representative of the total, but this finding could not be substantiated in the other participating centres.

(d) Change in visual acuity between that at discharge and at 8–25 weeks after surgery

Presenting visual acuity data at discharge as well as at 8–25 weeks follow up were available for analysis on 2819 eyes that had received a variety of different surgical procedures at the various centres. Overall, the proportion with good vision increased from 30.8% at discharge to 68.5%, while the proportion with poor outcome fell from 17.5% to 5.1% (table 3). There was, therefore, a very significant improvement in results from discharge to 8–25 weeks follow up time.

DISCUSSION

Two major problems were experienced during the pilot study. The first concerned the flow of CCSRFs through the hospital. Data are collected up to discharge while the patient is in hospital and then the CSRF goes to the computer operator for data entry. Afterwards it is returned to the outpatient department where patients are seen for review and again back for data entry. This cycle has to be repeated for subsequent visits. This requires strict logistic procedures and timely data entry.

The second problem was frequent data entry errors as data entry was not always done by trained, designated staff. In order to reduce this problem consistency checks were later incorporated into the software.

The operational research questions posed at the beginning of the study could only be partly answered. Centres could manage either the paper manual system or the computer system, but using both for comparison was found too time consuming. Patients seen at follow up in one centre were representative of all operated eyes in that centre, but this could not be substantiated in other centres. This is an important question which should be investigated further. Efforts to monitor outcome in a randomly selected sample of all operated cases caused much confusion. Only one centre could demonstrate no significant differences in outcome at any postoperative period between the two groups, although the sample was 27% of all operated cases, suggesting that the randomisation process had not been reliable (that is, 20% should have been selected). The consensus was that monitoring all operated cases rather than a random sample was less confusing and not much extra work. Centres with a large output might consider monitoring every other month, or during the first week of each month. However, the risk is that work attitudes may differ during monitoring periods and thereby affect outcome. Lastly, as anticipated, visual acuity at discharge did not accurately reflect visual acuity at 8–25 weeks after surgery. There was an improvement in good outcome from 30.8% to 68.5% of operated eyes and a decrease in poor outcome from 17.5% to 5.1%. The surgical technique used in most centres was extracapsular cataract extraction (ECCE) with IOL surgery, and only one of the centres was performing phacoemulsification. As small incision cataract surgery gains acceptance in developing countries it is anticipated that visual rehabilitation will be quicker than is currently the case following ECCE. Under these circumstances it seems likely that visual acuity at discharge will more closely relate to the final visual acuity, and so become more reliable for monitoring.

The monitoring system presented here follows the guidelines formulated by the WHO on visual outcome after cataract surgery¹⁴ (table 2). It is primarily intended for use in low and possibly middle income countries where gains from regular monitoring of outcome are expected to be greatest. Early in 2003 a CD-ROM with installation files for the manual monitoring and computer monitoring packages was distributed to another 20 eye centres for further testing. Based on their feedback and recommendations optional fields on preoperative and postoperative refraction, targeted postoperative spherical equivalent, biometry, and surgical details were added. The number of complications was expanded and a faster method for data entry was incorporated. An additional report on surgical details was added. In December 2003, this updated version was sent to the same 20 centres and a few new ones. Figure 2 shows the latest version of the CCSR.

For all cases with poor outcome a cause should be identified. This helps the surgeon decide whether current practices need modification to improve results. It is important to keep surgical causes of poor outcome or inadequate optical correction to a minimum. The proportion of cases with poor outcome due to pre-existing eye disease can also be reduced, by not operating on cataract patients with known pathology. However, these patients should not be denied surgery if there is the prospect of an improvement in functional vision; these patients should be informed of the likely outcome. When late postoperative complications, such as posterior capsule opacification are common, investing in a Yag laser may become cost effective.

Implementation of systems to monitor the outcome of cataract surgery should avoid confrontation and criticism, but rather create a culture of audit and support, to promote overall improvement in results. The purpose is to help surgeons analyse their own cases, to find out the causes of poor outcome so that action can be taken by the surgeon to improve his or her results. This will reduce blindness and poor vision after cataract surgery, and help to alleviate anxiety about surgery among those with operable cataract in the community.