Eye health indicators for universal health coverage: results of a global expert prioritisation process

Introduction In its recent World Report on Vision, the WHO called for an updated approach to monitor eye health as part of universal health coverage (UHC). This project sought to develop a consensus among eye health experts from all world regions to produce a menu of indicators for countries to monitor eye health within UHC. Methods We reviewed the literature to create a long-list of indicators aligned to the conceptual framework for monitoring outlined in WHO’s World Report on Vision. We recruited a panel of 72 global eye health experts (40% women) to participate in a two-round, online prioritisation exercise. Two-hundred indicators were presented in Round 1 and participants prioritised each on a 4-point Likert scale. The highest-ranked 95 were presented in Round 2 and were (1) scored against four criteria (feasible, actionable, reliable and internationally comparable) and (2) ranked according to their suitability as a ‘core’ indicator for collection by all countries. The top 30 indicators ranked by these two parameters were then used as the basis for the steering group to develop a final menu. Results The menu consists of 22 indicators, including 7 core indicators, that represent important concepts in eye health for 2020 and beyond, and are considered feasible, actionable, reliable and internationally comparable. Conclusion We believe this list can inform the development of new national eye health monitoring frameworks, monitor progress on key challenges to eye health and be considered in broader UHC monitoring indices at national and international levels.


INTRODUCTION
In its first World Report on Vision released in 2019, WHO included the strengthening of health information systems (HIS) among its five global priority areas for action. 1 This recognises the critical role of HIS to provide information-from populationbased surveys, facility-based sources and administrative data-to guide health policy, management and clinical care. Among WHO's recommended actions were to strengthen national capacity to collect, analyse and use data on eye health, and the creation of a global indicator menu for eye health from which countries can select relevant indicators. 1 The priority placed on HIS in the World Report on Vision also reflects the limited progress made to date. Several lists of indicators have accompanied global eye health initiatives over the past two decades. [2][3][4][5] Inconsistent reporting against these lists over time may be due to under-investment in district-level HIS capacity in low-income and middle-income settings, the vertical nature of many eye health systems, variable levels of engagement from national eye care planners and limited public-private sector cooperation. [6][7][8] In addition, a lack of policy imperative may be due to an absence of eye health indicators in WHO's global health monitoring frameworks to date. 9 10 Based on these and other challenges, in its Universal Eye Health: A Global Action Plan 2014-2019 4 (hereafter 'GAP') in 2013, WHO emphasised the need for eye care to be integrated into broader health planning. The World Report on Vision went further to state that eye health should be considered an essential component of universal health coverage (UHC). 1 Monitoring global eye health as part of UHC and the United Nations' Sustainable Development Goals 11 requires an updated menu of indicators aligned with the UHC dimensions of access, quality, financial risk protection and equity.
Here we report a collaborative prioritisation process to generate a menu of indicators that may be used by governments to monitor and improve eye health and eye health services at the national level, and to support progress towards achieving UHC. This work was undertaken as part of the Lancet Global Health Commission on Global Eye Health. 12

Study design
A two-round, prioritisation exercise was undertaken between February and April 2020 using an online survey platform (www.qualtrics.com). All panellists' responses were de-identified throughout, however, individuals were provided the option to join a study authorship group.

Participants
A project steering group (the co-authors) was convened to guide the development of the initial long-list of indicators, nominate panellists from a network of global eye health experts, review Clinical science indicator scoring and develop the final menu. We aimed to recruit panellists from all Global Burden of Disease (GBD) Super Regions, 13 with equal numbers of men and women per region. In total 74 out of 84 invited panellists participated in Round 1 and 72 went on to complete Round 2 (response rate after Round 2, 85.7%). Men were 59.7% of the Round 2 panel, similar to the proportion among all invitees. Eleven members of the steering group participated, five from a 'global' (non-Regional) perspective. Thirty-nine countries and all GBD Super Regions had participants in both rounds and 85% of the Round 2 panel represented low-income or middle-income countries (table 1). Round 2 panellists most frequently reported their roles within eye health as 'management/leadership' (25.0%), 'epidemiology' (12.5%), 'clinician/practitioner' (12.5%), 'eye health services research' (9.7%), 'government/Ministry of Health', 'clinical research' and 'international institution' (all 6.9%).

Initial indicator selection
A long-list of indicators was compiled with reference to previously proposed eye health indicators and existing international health and health systems indicator lists, adapted for relevance to the eye health sector where necessary. This long-list was mapped to the domains of measurement of HIS used in the World Report on Vision (adapted from the 2012 WHO Framework and standards for country health information systems) (figure 1). When panellists were invited to participate, they were asked to suggest additional indicators for consideration. The steering group reviewed all indicators identified, only excluding obvious duplicates in order to avoid biassing the pool of potential indicators. At the end of this process 200 indicators were included (online supplemental appendix 1).

The prioritisation exercise: round 1
Panellists scored the indicators based on perceived priority in their context. Priority was scored from 1 to 4 on a Likert scale, with 1 representing the lowest priority ('no need to collect') and 4 the highest priority ('essential to collect') (online supplemental appendices 2 and 3). A fifth option, (0 = 'redundant') was included to allow for the fact that the long-list had not been heavily edited and some overlap of indicator concepts was possible. A priority score for each indicator was calculated by summing the products of two dimensions: the Likert scale score (1)(2)(3)(4) x the number times each indicator received that score.
At the end of Round 1, an initial threshold for continued inclusion was set at or above the median score. Indicators scoring in the top half were merged where there was sufficient overlap in  Figure 1 Domains of measurement of health information systems (reproduced from the World Report on Vision).
concepts to do so. Indicators not scoring in the top half were reviewed to determine if any concepts deemed essential to score in Round 2 had been omitted and should be included to ensure representation (online supplemental appendices 2 and 3). In total, 95 indicators were forwarded to Round 2.

The prioritisation exercise: round 2
Each of the 95 indicators were scored against four new criteria. The panel were asked to indicate their agreement on a 4-point Likert scale (1 = 'strongly disagree', 2 = 'disagree', 3 = 'agree', 4 = 'strongly agree') as to whether each indicator was feasible, actionable, reliable and internationally comparable (table 2). In addition, the panel selected 10 indicators they considered to be 'core' indicators, described as those which all countries could be encouraged to adopt. These were ranked 1 (most important) to 10. Scores were calculated in the same way as Round 1. Each indicator was scored on the criteria separately and a composite score of all four was calculated, with all criteria weighted equally. Each indicator was assigned a rank position from 1 to 95 for each of the four criterion and the overall composite score. The ranking of indicators 1 to 10 as core indicators was calculated in a similar way: a vote for first place awarded 10 points, second place awarded 9 points and so on. Points were multiplied by the number of times an indicator received that vote position for an overall core score (online supplemental appendices 2 and 3). A ranking of 1 to 95 was given based on this scoring and this ranking was used in all subsequent analysis. Indicators with the same score were ranked equal.
We arrived at a list of 30 priority eye health indicators by ranking the Round 2 selections using two metrics: 1. The rank of the indicator based on the core score.

The rank of the indicator based on the composite score
We plotted the core and composite scores against each other and selected the 30 indicators that scored most highly by both ranking methods, by expanding the 'gating' equally along both axes until the selected area included 30 indicators (online supplemental appendices 2 and 3). The selected indicators, therefore, scored relatively highly for both.

Development of the indicator menu
Starting with the top 30 indicators from Round 2, we developed the detailed indicator menu presented in box 1. In this step we aimed to: ► Ensure alignment with UHC dimensions of access, quality, financial protection and equity ► Avoid repetition or misclassification of themes within and across domains ► Avoid over-representation or under-representation of domains ► Identify any omissions related to the five most prevalent causes of vision impairment globally (cataract, uncorrected refractive error, glaucoma, age-related macular degeneration, diabetic retinopathy). 14 No major edits to key concepts were undertaken. This process is summarised in figure 2.

RESULTS
Twenty-two distinct eye health indicator concepts were identified (box 1). In compiling the menu, we articulated broader concepts by specifying 39 'sub-indicators' (see bullet points under indicator titles). It is anticipated that these could be used in support of defining the broader indicators, for example, whether or not eye health is integrated into national health planning. Subindicators for the concept of eye health financing integration are not yet developed and, once included, will increase the scope of the menu in this domain.
The steering group selected seven core indicators for monitoring eye health as part of countries' progress towards UHC. These are set out in table 3.

DISCUSSION
This process engaged a large panel of global eye health experts representing all GBD Super Regions and developed a quantitative approach to prioritise existing indicators. The steering group refined the highest ranked selections to produce a menu of indicators for governments to monitor and improve eye health and eye health services, aligned with UHC and the Sustainable Development Goal on health and in keeping with WHO's call for such a menu in the World Report on Vision. We believe the core indicators highlighted here, if collected by all countries, could allow governments, and supranational organisations, to track progress on key challenges within eye health and UHC. Otherwise, the menu is not intended to be prescriptive; countries could select indicators according to priorities based on population need. We recognise that some countries will likely benefit from collecting and reporting fewer, more important eye health indicators as accurately as possible.
The core indicators include two candidate WHO UHC service coverage indicators: effective cataract surgical coverage (eCSC) and effective refractive error coverage (eREC). 15 16 Effective coverage has been acknowledged as a useful measure of progress towards UHC as it includes dimensions of quality, access and, where disaggregated, equity. Both eCSC and eREC were omitted from a recent global UHC analysis because of limited data availability, 17 an issue which must be addressed.
The standard UHC financial risk protection indicators (catastrophic and impoverishing expenditure) adapted to eye health scored lowest among the 95 indicators in Round 2. This likely reflects anticipated complexities in data collection and the possibility that, for non-emergency healthcare, they may not be sufficiently discriminatory. Instead, we have proposed two new proxy measures for financial risk protection. These are not intended as direct replacements for catastrophic and impoverishing expenditure indicators, rather what might be achievable within the constraints of eye health data availability. They will require additional work to develop a full metadata description but provide a way to track eye health insurance coverage (for multiple conditions) and out-of-pocket (OOP) payments for Continued treatment of cataract, the most common cause of blindness globally. The WHO has acknowledged that monitoring the intersection between service coverage and OOP expenditure is key to assessing progress towards UHC. 18 There may be value in expanding this OOP payment indicator to include refractive error correction and a pilot test for one or both of these will be undertaken in the near future within the Rapid Assessment of Avoidable Blindness (RAAB), a well-established populationbased survey method. 19 We acknowledge that relying on OOP payment data obscures those people who do not present to care due to unaffordable cost, an essential group to identify and reach for UHC to be realised. 20 Place of residence should not be a barrier to accessing care, however, human resources for eye care are skewed towards Clinical science urban settings. 21 22 We have included two provider-side measures of access: human resources and infrastructure for eye care per capita. These can be reported more frequently than service coverage estimates and at little cost. The density of eye health workers (by cadre) features in all previous eye health indicator lists. In order to be UHC-aligned, countries must expand beyond aggregate numbers of personnel, and monitor their distribution by rural/urban and public/private settings. Disaggregation should also be applied to the monitoring of eye care facility density and distribution. This metric could be enhanced by geocoding infrastructure and integrating geographical coverage with other population data to apply a spatial component to eye health planning. 23 The final concept represented in the core indicators selection is the impact of countries' eye health systems on population health in terms of the prevalence of vision impairment. Vision impairment affects individuals' quality of life but is also a broader development issue affecting education and employment. 24 Vision impairment prevalence has been the key measure of eye health for decades and continues to be useful at the national level for monitoring and planning and at the global level for advocacy.
Beyond the core indicators, key concepts represented in the menu include the integration of eye health into national health planning and financing. We acknowledge that defining and measuring the degree of integration in these areas requires further discussion, particularly for health financing. The integration of primary eye care into primary healthcare planning, financing and training programmes is also included, along with integration of eye health into HIS.
The importance of data disaggregation to monitor eye care equity across the menu should not be understated. We have included an equity statement for consideration across the list and believe equity-relevant monitoring is essential to ensure the most gains are made among population groups with the most need.
Our menu has substantial overlap with WHO's GAP indicator list, including its six key indicators on vision impairment, human resources for eye care and cataract surgical services. New concepts in our list include eye care insurance coverage and affordability, the UHC dimensions of quality and equity, eye health infrastructure and information systems, primary eye care, child eye health, refractive error and diabetic retinopathy. Our proposed indicators are not only UHC-aligned but also address many areas of global eye health prioritised in a recent 'grand challenges' global Delphi process. 12 There are some notable absences from the final menu. The concept of 'people-centred' eye care proposed in the World Report on Vision is not represented. We presented 13 'responsiveness' indicators in Round 1, including 7 patient-reported outcome indicators, but the panel prioritised none; this will require further study. Disease-specific indicators for glaucoma and age-related macular degeneration were potentially underrepresented in the initial long-list and not prioritised by the panel, despite their prominence as causes of vision loss globally. 14 This may be because the natural history of these conditions make monitoring more complex than for cataract or refractive error. Appropriate coverage indicators for these conditions will require further investigation, and as the menu evolves, more 'difficultto-measure' concepts would ideally be included. 12 Trachoma and onchocerciasis were not prioritised, likely reflecting the progress made in these areas in recent years. However, we expect endemic countries would continue to report against indicators aligned with their elimination programmes. Unilateral vision impairment, associated with, for example, infectious corneal ulcers, is not included in the menu but was identified as a knowledge gap in the World Report on Vision and could be included in future as it gains priority in eye health planning. The GAP indicator for evidence of research on the cost-effectiveness of eye health programmes was not prioritised, but more evidence of costeffectiveness may strengthen the case for resource allocation. Finally, broader health and financing indicators potentially relevant to eye health (demographics, non-communicable diseases, water and sanitation, government health spending) were not prioritised but could be obtained from other national reporting mechanisms to support eye care planning as appropriate.
We recognise that generating this list is insufficient in isolation, and several challenges must be addressed for these indicators to be successfully integrated into countries' HIS and monitoring frameworks. Fortunately, the priority given to HIS in the World Report on Vision and the potential inclusion of eCSC and eREC in the next list of WHO UHC indicators provides impetus for action. In addition, countries will benefit from the ongoing refinement of tools such as WHO's Eye Care Services Assessment Tool 25 and RAAB and its Planning Module. 19 These tools strengthen national HIS capacity by providing guidance on data collection and interpretation for a range of indicators included in our list. Several of the new indicators proposed here require indicator metadata which would ideally be generated by subject-specific expert working groups working collaboratively with countries. Alongside indicator development, appropriate target-setting also requires consultation. Further, there are financial and logistical challenges for countries to routinely collect national-level population health data, so rapid surveys of vision impairment and eye care services have often been carried out at the subnational level to aid local planning. In the absence of increased national-level data collection, modelled estimates will be required to provide data for global estimates and regional and national comparisons with any degree of regularity.

Clinical science
We propose that new indicators in this menu be field-tested in several contrasting settings, and that the menu be regularly reviewed and updated according to user feedback. Such reviews would ideally assess whether data collection and indicator usage are viable and valuable for both national and subnational planning, as well as for generating global eye health estimates. These steps require ongoing engagement and resourcing to develop and maintain the utility of the menu. This may be encouraged by a centralised eye health data repository.

Limitations
This study has several limitations. First, inherent in a study that recruits experts, the indicators prioritised reflect the preferences of those invited to participate. We aimed to be as geographically representative as possible, however, the North Africa and Middle East, High Income and Central Europe, Eastern Europe and Central Asia Super Regions had few panellists. Further, despite aiming for gender parity, only 40% of the panel were women. A more diverse panel may have generated a different set of indicators. Second, the online exercise was only available in English, however, no nominated panel members were unable to participate due to language constraints. Third, personal interests and familiarity with some concepts over others may have led to confirmation bias in scoring by panel members. The overlap with existing GAP indicators may be a reflection of this, however, the menu does include many new concepts. Finally, detailed explanations of new concepts are required which was beyond the scope of this prioritisation project.

CONCLUSION
This process sought a broad consensus from 72 eye health experts from all world regions to produce a menu of indicators for countries to monitor eye health as part of UHC. From a long-list of 200, the final menu consists of 22 indicators that represent important concepts in eye health for 2020 and beyond, and are relatively feasible, actionable, reliable and internationally comparable. The new direction in global eye health set by the World Report on Vision must be supported with investment in HIS that include eye health data collection and data monitoring via internationally acceptable indicators. We believe this list is well-placed to inform the development of new national eye health monitoring frameworks and shows where eye health metrics might be incorporated into broader UHC monitoring indices at national and international levels.