Aims: To report the item specific responses of the VF-14 in a population of patients undergoing cataract surgery in their first eye and to determine whether or not the VF-14 can be reduced without compromising its performance as an index of cataract related visual impairment.
Methods: The item specific responses to the VF-14 were analysed before (771 patients) and 4 months after (552 patients) cataract surgery in one eye to determine if the VF-14 index can be reduced without compromising its performance. Patients studied were selected from a cross sectional longitudinal study of patients undergoing cataract surgery in 72 ophthalmologist's offices located in three metropolitan regions of the United States.
Results: Pairwise correlations between items in the VF-14 were all less than 0.6, indicating that no items could be removed solely on the basis of redundancy. 10 items correlated moderately with change in trouble, and 11 correlated moderately with change in satisfaction (r >0.15) at 4 months after cataract extraction. Eleven items demonstrated an effect size >0.4 at 4 months. These 11 items were either important for detecting cataract related functional disability or for quantifying the extent to which cataract impaired function. Additionally, 11 items were needed to detect adequately individuals with functional impairment. Three items (recognising people, cooking, and reading large print) were less responsive to cataract extraction and were more strongly associated with ocular comorbidities.
Conclusions: While previous reports indicate that the VF-14 can be significantly shortened, our analysis only justifies removing three items. While the resulting VF-11 has properties similar to the VF-14, the limited time savings do not justify altering this already validated instrument.
Statistics from Altmetric.com
Cataract extraction remains the most commonly performed operation on Medicare beneficiaries, with 1.4 million surgeries in 1998 (Kevin Hayes, personal communication, Medicare Payment Advisory Commission, Washington, DC, 19 August 1999). To quantify the functional limitations associated with cataract, researchers have developed standardised questionnaires designed to measure the impact of impaired vision on patients' ability to perform daily activities.1–4 Patients' responses have correlated well with patient reported trouble and satisfaction with vision before surgery and have improved significantly after cataract surgery.1,5–7 In contrast, preoperative Snellen acuity testing does not assess the functional difficulties experienced by patients with cataract and has been documented to be poorly correlated with patient reported trouble and satisfaction with vision.5–8
Reports issued by the Agency for Health Care Policy and Research Cataract Guidelines Panel and the American Academy of Ophthalmology state that the goal of cataract surgery is functional improvement,9,10 and the indication for cataract surgery is cataract induced functional impairment that is considered to be significant by the patient.
Despite the widespread availability of reliable, valid, and responsive instruments for quantifying functional impairment related to vision, most practising ophthalmologists still do not use them. Possible reasons for this low utilisation include lack of belief in the utility of the available instruments, low awareness of them among ophthalmologists, and a reluctance to spend the time required to administer a standardised questionnaire to a patient. The purpose of this study was to determine whether the VF-14 could be reduced in length without substantial compromise of its performance. Expanded use of standardised methods of obtaining this information would improve the assessment of cataract patients in clinical practice and evaluation of the care they receive in different settings.1
A recent publication indicated that the VF-14 might be able to be reduced to seven questions without compromising performance.11 In order to further examine these findings, we sought to determine which of the individual items within the VF-14 were most responsive to cataract surgery among the population of patients enrolled in the Cataract Patient Outcomes Research Team (PORT) study and to assess the correlation between alternative combinations of these items and self reported trouble and satisfaction with vision.5,12
Data collected during the Cataract PORT longitudinal study of cataract outcomes were used for this analysis. The methods employed in that observational study of cataract surgery have been described in detail elsewhere.1,8 In brief, patients of age 50 years or older scheduled to undergo first eye cataract surgery by any of 72 ophthalmologists in three metropolitan areas were recruited between July and December 1991 for participation. Data were collected preoperatively and at 48 hours, 4 months, and 12 months after cataract extraction. The VF-14, which asked patients about 14 vision dependent activities, was used to assess functional impairment related to vision.1 The VF-14 is administered by asking patients the following: “Do you have any difficulty, even with glasses, in doing any of the following activities?” If the answer to a question regarding a particular activity is “yes,” the patient is asked whether his/her level of difficulty with performance of the activity is “a little,” “a moderate amount,” “a great deal,” or whether he/she is unable to perform the activity because of his/her vision. In addition, the patient can state that the activity is “not applicable” (that is, he/she does not perform the activity for reasons unrelated to his/her vision). The activities addressed by the VF-14 are:
Reading small print such as labels on medicine bottles, or a telephone book
Reading a newspaper or book
Reading a large print book or large print newspaper or numbers on a telephone
Recognising people when they are close to you
Seeing steps, stairs, or curbs
Reading traffic, street, or store signs
Doing fine handwork like sewing, knitting, crocheting, or carpentry
Writing cheques or filling out forms
Playing games such as bingo, dominoes, card games, or mahjong
Taking part in sports like bowling, handball, tennis, or golf
The VF-14 is scaled from 0 to 100, with 0 indicating that the patient is unable to perform any applicable activities and 100 meaning that the patient can perform all applicable activities without difficulty.
Patients enrolled in the Cataract PORT study also were asked to rate their trouble with vision and their satisfaction with vision, using scales with four possible responses. For trouble with vision, response options were “none,” “a little,” “a moderate amount,” or “a great deal.” For satisfaction with vision, response options were “very dissatisfied,” “dissatisfied,” “satisfied,” or “very satisfied.”
A total of 888 patients were referred for enrolment during the Cataract PORT study; 772 (86.9%) agreed to participate. Baseline analyses were performed on 771 individuals who completed the VF-14. Analyses of the effect of cataract surgery on patient reported outcomes were limited to the 552 patients who had both preoperative and 4 month postoperative VF-14 data and had cataract surgery in only one eye.
Differences in the proportion of patients enrolled in the Cataract PORT study who reported particular levels of difficulty with specific VF-14 items were assessed using a χ2 test. Associations between preoperative visual acuity, ocular comorbidities, and item specific responses were assessed using multiple logistic regression. Preoperative versus postoperative changes in proportions for paired data were assessed using McNemar's test. Associations between preoperative versus postoperative change in reported trouble with vision and satisfaction with vision and preoperative versus postoperative responses to each item in the VF-14 index were evaluated using Spearman's rank correlation coefficient. Effect sizes were calculated for each item in the VF-14 as well as for the VF-14 and indices we constructed that included fewer than the original 14 items. The effect size is a measure that can be used to quantify changes in health status for a group produced by an intervention. While there are several approaches to measuring effect size,13,14 we used the mean change in the item or index score following cataract surgery divided by the standard deviation of that item or index score at baseline. An effect size of one, therefore, means that the score, on average, changed by one standard deviation. The effect size for the VF-14 (0.99) has been reported previously.12 An instrument demonstrating an effect size greater than 0.8 is considered to be responsive.15
We explored the impact of removal of an item from the VF-14 if it did not appear to detect uniquely a common type of functional impairment. Specifically, we assessed the impact of removal of an item if few Cataract PORT study enrollees reported that the activity applied to them, if over 90% of subjects reported no difficulty with the activity at baseline, if the item correlated poorly with baseline trouble and satisfaction with vision, or if the item did not uniquely identify a functional impairment for a given patient. We also assessed the association between patient responses to each item and an index of ocular comorbidity (comorbidity was present if the subject had macular degeneration, glaucoma, or diabetic retinopathy in the opinion of the examining ophthalmologist) to determine if conditions other than cataract may have been responsible for patient reported functional impairment. Items that were strongly associated with ocular comorbidities were considered as candidates for removal from the VF-14. A final component of our strategy for paring down the VF-14 was to remove items that did not appear to be responsive to cataract surgery, as demonstrated by a small effect size, and/or poor correlation with patient reported change in trouble and satisfaction with vision after cataract removal. Factor analysis on the final, reduced index was performed to determine whether there were subscales within the overall scale that reflected clinically meaningful domains of function. Cronbach's α16 was also calculated to measure the internal consistency of shortened forms of the VF-14.
Subjects enrolled in the Cataract PORT study were mostly white (94%) and female (63%, Table 1). The 552 subjects who underwent surgery in only one eye by 4 months were similar to those who had both eyes in age, race, and sex, but had higher baseline scores on the VF-14 (p < 0.01, adjusted for age, race, and sex).
Performance of individual items of the VF-14
The proportion of study enrollees for whom individual VF-14 activities were applicable among the 771 individuals with baseline data ranged from 25.3% for taking part in sports to 100% for recognising people (Table 2). The proportion of respondents for whom specific VF-14 activities were applicable at both the preoperative and postoperative evaluations ranged from 16.7% for taking part in sports to 99.8% for reading small print and recognising people (Table 3). Over 300 patients (55%) participated in each activity except for sports at both baseline and 4 months postoperatively.
For both the entire enrolled cohort (n = 771) and the 552 patients with baseline and 4 month postoperative data and no second eye surgery before the 4 month postoperative examination, 10% of patients or fewer reported any difficulty at baseline in recognising people, reading large print, or cooking. In contrast, over half reported at baseline that they either had a great deal of difficulty or were unable to drive at night or read small print. The proportion of patients who reported moderate or severe impairment with an activity decreased significantly for each activity after cataract extraction (p <0.001 for all activities).
Pairwise correlations between the scores on individual items in the VF-14 were all <0.6, indicating that variables could not be removed from the VF-14 on the basis of redundancy alone. Correlations between the score on each item and the score on the remaining 13 items were lowest for difficulty reading large print, recognising people, cooking, and night driving (<0.4) suggesting that these items were measuring impairments that were the most different from the other 10 items. These differences could be due to these activities identifying types of cataract related functional limitations not captured by the other items, or to these activities being more affected by non-cataract ocular disorders. Night driving may be particularly impaired by cataract induced glare. Hence, it is possible that patients may have considerable difficulty with night driving without having much difficulty with other activities included in the VF-14. In contrast, the comparatively low correlation between scores for reading large print, recognising people, or cooking and the other items may be due to the fact that the vast majority of individuals had “no difficulty” performing these three activities at baseline.
To assess quantitatively the unique contribution of individual items to the VF-14 score, as well as the responsiveness of individual questions to cataract surgery, we examined, for each item: (1) the correlation between the change in the item score and the change in trouble and satisfaction with vision after cataract extraction; (2) the effect size of the item; (3) the effect size of the remaining 13 item index score after removing that item from the VF-14; (4) the frequency with which a patient reported any difficulty in that activity; and (5) the likelihood that a patient would have difficulty with at least one item in the remaining VF-13 after an item was removed from the VF-14.
Postoperative changes in ability to read large print, recognise people, and see steps or curbs were the most weakly correlated with both change in trouble and change in satisfaction with vision (r ≤0.15 for both trouble and satisfaction, Table 4). The items in which the change in score was most strongly correlated with change in trouble and satisfaction (r ≥0.25) were reading small print and doing fine handwork. The effect size of individual questions (how many baseline standard deviations a measure changed after cataract extraction) ranged from 1.0 for reading small print to 0.21 for recognising people (Table 5). The activities that were most responsive to surgery (as measured by effect size ≥0.8) were near vision activities (reading small print, reading the newspaper, and doing fine handwork). The activities that were least responsive to surgery (as evidenced by an effect size ≤0.3) were recognising people, reading large print, and cooking. The poor responsiveness of these items is likely due to the large number of patients who could not improve on these activities given the lack of difficulty they had with them at baseline. The effect sizes of the various VF-13 indices that result from removing items one at a time were similar to those found for the full VF-14.
Some questions included in the VF-14 provided little information about the functional status of most patients. For example, four out of five patients did not participate in sports either preoperatively or postoperatively, meaning that, for the majority of patients, this question did not contribute to their VF-14 score. In addition, some items (reading large print, recognising people, and cooking) asked about activities with which ≥90% of individuals had no difficulty. To determine quantitatively which items contributed unique information we assessed (using data from all subjects at baseline) the frequency that an individual activity was the only one for which a patient reported having difficulty. One could anticipate, for example, that a patient might have trouble playing sports as her or his sole complaint. In this analysis, for each activity we used the number of patients reporting moderate or greater difficulty preoperatively as the denominator. For these patients, the mean number of other questions for which they reported moderate or greater difficulty was calculated. This mean number ranged from 2.3 for night driving to 5.9 for playing games (Table 6). In addition, for each activity, of the individuals reporting moderate or greater difficulty, the percentage reporting moderate or greater difficulty on less than two (that is, none or one) other activities was calculated. Of the 771 individuals in our analysis, 26.1% reported moderate or greater difficulty on driving at night but had either zero or only one other activity for which they reported moderate or greater difficulty. Over 15% of those enrolled reported difficulty reading small print and had either one or no other items with which they had moderate or worse difficulty. Therefore, the reading small print and driving at night items often identified individuals with visual limitations who might have been missed if these questions were removed. Conversely, every patient who reported moderate or greater difficulty reading large print or watching TV had at least two other activities for which he/she reported moderate or greater difficulty. Less than 1% of the study population reported moderate or worse difficulty preoperatively reading large print, recognising people, seeing steps or curbs, playing games, playing sports, cooking, or watching television who did not have at least two other items on which they reported moderate or worse difficulty.
Of the 771 individuals enrolled in the Cataract PORT study, 722 had at least one activity in the VF-14 with which they had at least a little difficulty (all patients had already elected to have cataract surgery). Removing reading small print led to the greatest decline in the number of individuals identified as having at least one question with moderate or worse difficulty (Table 7), indicating that for many patients this item captures functional difficulty that is not identified by other items in the VF-14 index. Night driving was the only other item with a similar profile.
Based on the preceding analyses, we concluded that the items recognising people, reading large print, and cooking contributed little to detection of functional impairment in cataract surgery candidates. Over 90% of cataract surgery patients had no difficulty preoperatively with these three tasks. In addition, less than 1% of the study population had moderate or greater difficulty on any of these three questions and did not report moderate or greater difficulty on two or more other questions. Finally, these three items showed poor correlation with the overall VF-14 score (r <0.4) and had low effect sizes.
These items had originally been included in the VF-14 to provide an indication of disease severity. Those with severe visual impairment, it was believed, would be more likely to have difficulty on these relatively simple tasks.
Table 8 shows that the presence of one or more of three ocular comorbidities (diabetic retinopathy, macular degeneration, or glaucoma) was associated with an age adjusted threefold increased likelihood of having moderate or worse difficulty recognising people. This association with ocular comorbidity and difficulty recognising people was driven primarily by an association between difficulty recognising people and an association between difficulty with cooking and the presence of macular degeneration (OR = 3.3; 95% CI 1.6, 6.7) or glaucoma (OR = 2.0; 95% CI 0.9 to 4.2). Moderate or worse difficulty in cooking also was significantly associated with ocular comorbidity (OR = 1.9; 95% CI 1.1, 3.5), again largely due to macular degeneration (OR = 2.3; 95% CI 1.0, 5.0) or glaucoma (OR = 1.9; 95% CI 0.9, 4.1). The only other VF-14 item with a statistically significant correlation with the ocular comorbidity index was seeing steps or curbs (OR = 1.6; 95% CI 1.1, 2.4), which was due to increased difficulty with this item among individuals with glaucoma (OR = 1.9, 95% CI 1.1, 3.2).
Removal of recognising people, reading large print, and cooking from the VF-14 resulted in a VF-11 with good internal consistency (Cronbach's α = 0.83). The VF-11 also had a slightly larger effect size than the VF-14 (1.09 versus 0.99), as well as a comparable degree of correlation with change in trouble with vision and satisfaction with vision after cataract surgery (Table 9). The Spearman correlation between the VF-14 and VF-11 was 0.99 (p <0.0001).
Several studies have shown that functional status is more strongly correlated with self reported trouble and satisfaction with vision than is Snellen acuity,1 at least in the range of vision loss represented among people presenting for cataract surgery.5–7 This analysis was undertaken to assess the contribution of each item in one measure of functional impairment related to vision, the VF-14, as well as to evaluate the impact of deleting various items from the VF-14. Several of our findings provide a rationale for removing three items from the VF-14 (Table 10 summarises these). Fewer than 10% of patients awaiting cataract surgery who were enrolled in the Cataract PORT study had any difficulty recognising people when they are close, reading large print, or cooking. Less than 1% of the total population reported moderate or greater difficulty in one of these activities in the absence of comparable difficulty with another activity in the VF-14. These items are therefore plagued by a ceiling effect in which almost all subjects are at the top of the scale and therefore cannot improve with treatment. In addition, difficulty recognising people and difficulty cooking were both more strongly associated with ocular comorbidity than with cataract in our population. Removing recognising people, reading large print, and cooking from the VF-14 results in an 11 item index (the VF-11). This shorter index has good internal consistency, and is as strongly correlated as the VF-14 with preoperative trouble with vision and satisfaction with vision. In addition, the correlation between change in the VF-11 score after cataract surgery and changes in trouble with vision and satisfaction with vision were of a similar magnitude to those for the VF-14.
Responsiveness of individual items to cataract surgery was measured in a subset of subjects who only had one cataract removed at 4 months. These individuals were similar to those who had both cataracts removed in terms of age, race, and sex, but had higher VF-14 scores at baseline. It is possible that the responses of this subset to the VF-14 at 4 months may differ from that of all patients undergoing first eye cataract surgery. Those who elected not to have the second surgery performed may have been less satisfied with the surgery than those who chose to undergo a second operation. Conversely, those who were most satisfied may have elected to hold off on a second procedure. It is therefore difficult to predict how the bias introduced influenced the results.
Uusitalo and colleagues recently proposed removing seven items from the VF-14 to create a VF-7.11 The authors relied solely on the correlation between change in individual items and change in patient satisfaction following cataract surgery to select items to remove from the VF-14. While they demonstrated good correlation between change in trouble and satisfaction after cataract surgery and change in the VF-7 score (as would be expected given the methodology used to develop the VF-7), they did not report on the internal consistency of the VF-7 in their patient population, or the likelihood that the VF-7 would fail to identify at least one type of functional impairment in patients undergoing cataract surgery. Our analysis found that the VF-11 could not be reduced further without jeopardising one or more important aspects of the performance of the index. Decreasing the number of questions posed to a patient reduces the time needed to use a standardised instrument and may increase the likelihood that clinicians will perform a systematic evaluation of functional status. While we had hoped to be able to significantly shorten the VF-14, our analysis justified removing only three items. Removal of more items from the index would only weaken its utility in quantifying functional limitations caused by cataract. However, removing only three items will result in minimal time savings. Given the long track record of the VF-14, and the documented responsiveness and reliability of the instrument, we believe it is not advisable to remove items from this instrument which has been validated as a tool to measure disability related to cataract and corneal and retinal diseases.
Funding: HS-06280; K-23 EY00358.
Dr Friedman also is funded by a the Robert E McCormick Scholar Award from Research to Prevent Blindness
The authors have no proprietary interest in the instrument evaluated in this paper.