Computer interface evaluation using eye movements: methods and constructs
Introduction
The software development cycle requires frequent iterations of user testing and interface modification. These interface evaluations, whether at initial design or at later test and evaluation stages, should assess system functionality and the impact of the interface on the user. Earlier, design evaluation methods include cognitive walkthroughs, heuristic, review-based, and model-based evaluations. At more mature product phases, performance-based experiments, protocol/observation, and questionnaires are frequently used as a basis for evaluation (Dix et al., 1998). Performance-based studies assess errors and time to complete specified operations or scenarios (Wickens et al., 1998).
Interface evaluation and usability testing are expensive, time-intensive exercises, often done with poorly documented standards and objectives. They are frequently qualitative, with poor reliability and sensitivity. Provision of an improved tool for rapid and effective evaluation of graphical user interfaces was the motivating goal underlying the present work assessing eye movements as an indicator of interface usability.
While using a computer interface, one's eye movements usually indicate one's spatial focus of attention on a display. In order to foveate informative areas in a scene, the eyes naturally fixate upon areas that are surprising, salient, or important through experience (Loftus and Mackworth, 1978). Thus, current gazepoints on a display can approximate foci of attention over a time period. When considering short time intervals, however, one's attentional focus may lead or lag the gazepoint (Just and Carpenter, 1976). By choosing long enough sampling intervals for eye movements, temporal leads/lags should be averaged out.
Applied eye movement analysis has at least a 60 yr history in performance and usability assessments of spatial displays within information acquisition contexts such as aviation, driving, X-ray search, and advertising. Buswell (1935)measured fixation densities and serial scanpaths while individuals freely viewed artwork samples, noting that eyes follow the direction of principal lines in figures, and that more difficult processing produced longer fixation durations. Mackworth (1976)noted that higher display densities produced 50–100 ms longer fixation durations than lower density displays. Non-productive eye movements more than 20° from the horizontal scanning axis strongly increased as a percentage of all eye movements as the display width and density increased. Kolers et al. (1981)measured eye fixations (number, number per line, rate, duration, words per fixation) as a function of character and line spacing in a reading task. More fixations per line (and fewer fixations per word) were associated with more tightly-grouped, singled-spaced material. Fewer, yet longer fixations were made with smaller, more densely packed text characters. Yamamoto and Kuto (1992)found improved Japanese character reading performance associated with series of sequential rather than backtracking eye movements. Eye tracking has aided the assessment of whether the order of product versus filler displays in a television commercial influences one's attention to that product (Janiszewski and Warlop, 1993). Using eye movement analyses while scanning advertisements on telephone yellow pages, quarter-page ad displays were much more noticed than text listings, and color ads were perceived more quickly, more often, and longer than black and white ads (Lohse, 1997).
Prior eye movement-based interface and usability characterizations have relied heavily upon cumulative fixation time and areas of interest approaches, dividing an interface into predefined areas. Transitions into and from these areas, as well as time spent in each area, are tallied. While these approaches can signal areas where more or less attention is spent while using a display, few investigations have considered the complex nature of scanpaths, defined from a series of fixations and saccades on the interface. Scanpath complexity and regularity measures are needed to approach some of the subtler interface usability issues in screen design.
Eye tracking systems are now inexpensive, reliable, and precise enough to significantly enhance system evaluations. While the hardware technology is quite mature (Young and Sheena, 1975, for a general review), methods of evaluating data from eye tracking experiments are still somewhat immature and disorganized. The objective of the present paper is to provide an introduction and framework for eye movement data analysis techniques. These eye movement measures and algorithms are presented in light of results from an experiment presenting users with both “good” and “poor” interfaces.
Section snippets
Methods
Scanpaths were collected from 12 subjects while using both “good” and “poor” software interfaces. The resulting scanpaths were characterized using a number of quantitative measures, each designed to characterize different aspects of scanpath behaviors and relate to the cognitive behavior underlying visual search and information processing. A comparison of expected user search behavior using each interface with the results of scanpath measures were used to determine the relative effectiveness of
Classification of measures
Scanpaths are defined by a saccade–fixate–saccade sequence on a display. For information search tasks, the optimal scanpath is a straight line to a desired target, with relatively short fixation duration at the target. The derived scanpath measures discussed below attempt to quantitatively measure the divergence from this optimal scanpath in several ways. The measures each provide a single quantitative value, with some requiring no knowledge of the content of the computer interface. Table 1
Measures of search
Illustrated descriptions of each of the eye movement measures and algorithms are provided below. Results from the good versus poor interfaces and other factors are also presented here. The same hypothetical scanpath is used in all examples below, for easy comparison. All of these measures may be used for a given scanpath, with each offering a slightly different interpretation of the data. Scanpaths may also be viewed as directed or undirected graphs, allowing additional characterizations of
Measures of processing
Visual search is conducted to obtain information from an interface, where more extensive search allows more interface objects to be processed. This does not consider the depth of required processing, however. In the present study, as the same representations were used in both interfaces, the depth of processing required to distinguish and interpret a component was not expected to differ.
Discussion
Successful interaction with a computer clearly requires many elements, including good visibility, meaningfulness, transparency, and the requirement of simple motor skills. Eye movement-based evaluation of the interface, as espoused here, can only address a subset of critical interface issues that revolve around software object visibility, meaningfulness, and placement. Though not a panacea tool for design evaluation, characterization of eye movements can help by providing easily comparable
References (23)
- et al.
Eye fixations and cognitive processes
Cognitive Psychology
(1976) - et al.
A probability vector and transition matrix analysis of eye movements during visual search
Acta Psycholgica
(1995) On line measurements, analysis and control
- Belofsky, M.S., Lyon, D.R., 1988. Modeling eye movement sequences using conceptual clustering techniques. Air Force...
How People Look at Pictures
A Study of the Psychology of Perception in Art
(1935)- et al.
Human–Computer Interaction
(1988) - et al.
Eye-gaze determination of user intent at the computer interface
- et al.
Eye-gaze contingent control of the computer interface: Methodology and example for zoom detection
Behavior Research Methods, Instruments and Computers
(1995) - Jones, R.E., Milton, J.L., Fitts, P.M., 1949. Eye fixations of aircraft pilots; IV: Frequency, duration and sequence of...
- et al.
Eye movement measurement of readability of CRT displays
Human Factors
(1981)
Spatial interactions in rapid pattern discrimination
Spatial Vision
Cited by (771)
Visual inspection problem-solving strategies at different experience levels
2024, Applied ErgonomicsA machine learning study for predicting driver goals in contingencies with leading and lagging features during goal determination
2024, Expert Systems with ApplicationsGazeGraphVis: Visual analytics of gaze behaviors at multiple graph levels for path tracing tasks
2023, Advanced Engineering InformaticsMachine learning models for stroke detection by observing the eye-movement features under five-color visual stimuli in traditional Chinese medicine
2023, Journal of Traditional Chinese Medical SciencesUsability Study of UUM Student Portal Using Eye Tracker
2024, Communications in Computer and Information Science
- 1
Present address: Lucent Technologies, Bell Laboratories, Holmdel, NY.