CS-TR-3793

UMIACS-TR-97-43

Apparency of Contingencies in

Pull Down Menus

Diane Lindwarm Alonso and Kent L. Norman

Department of Psychology and

Human-Computer Interaction Laboratory

University of Maryland

College Park, MD 20742




Abstract

In many computer interfaces the underlying structures and contingencies are often hidden from the user's view. Users high in Spatial Visualization Ability (SVA) are able to quickly determine and manage the contingencies of these relationships and are not severely affected by this problem. Low SVA users, however, have difficulty visualizing these contingencies and often get lost. We examined the performance of 160 undergraduate students to determine whether revealing hidden contingencies though visual cues would facilitate Low SVA users enabling them to approach the level of performance of High SVA users on a computerized path finding task. It was found that using color and displaying paths improved performance, however, there is no indication that it is more beneficial to low than high SVA users.

Apparency of Contingencies in Pull Down Menus

In today's interactive world, the ways in which we navigate through systems has become an important issue. To some, navigation is a trivial exercise, but to others, it can be a confusing and frustrating experience. Problems in navigation arise for 3 main reasons. The first has to do with individual differences. The way in which different users visualize the layout of the system and its underlying structure and contingencies may be due in a large part to their level of Spatial Visualization Ability (SVA). Studies have shown a strong correlation with an individual's ability to navigate through a hierarchical data base (e.g. Butler, 1990, Vincente, Hayes, & Williges, 1987). High SVA individuals are often able to visualize 3 dimensional space in an efficient and relatively accurate manner, whereas low in SVA tend to use less efficient means to store and manipulate their visions of 3 dimensional space. According to Lohman (1989) high spatial subjects often solve figural tasks by "generating mental images that they transform holistically" and that these subjects are better able to generate, manipulate and also preserve information about the figure than low SVA subjects. Low SVA subjects tend to use devices such as general reasoning skills or external aids, although most subjects use more than one strategy often switching back and forth among them.

Another factor affected by individual differences in SVA is Working Memory. In a study by Salthouse, Babcock, Mitchell, Palmon, & Skovronek (1990) it was suggested that low SVA subjects may need more "workspace" than high SVA subjects for the same processing. Low SVA subjects appear to have more difficulty when there are storage and processing demands, because these demands may exceed capacity. In another study, Salthouse, Mitchell, Skovronek, & Babcock (1989) noted reductions in working-memory capacity for older adults in increased task complexity.

Another problem is that technology may amplify these differences. With the implementation of a new technology which is suppose to improve performance, any of the following three cases may occur: 1) most users improve, but users at the lower end of the distribution get worse, 2) most improve, but users at the lower end of the distribution improve only slightly, or stay the same, or 3) all improve, but the distribution becomes more disperse. Generally, if there is an improvement in performance it seems to differentially benefit those individuals who are least in need of assistance. It would seem to be beneficial to give some extra support those users who are not as comfortable with the system.

The third problem has to do with how the underlying structure and the contingent relationships of the interface are represented. Often, the underlying structure (e.g. a tree structure, or a loop) is hidden from the user's view, as are the contingencies (which actions need to be taken before others can be performed.) This may not be a major problem for high SVA individuals as they are often able to figure out the structure and the contingencies on their own. Users with low SVA, though, may get terribly lost in the system and this may lead to frustration and possibly even computerphobia (fear of computers) and technostress (anxiety toward technology).

Some solutions to these problems have been suggested and their implementation has improved navigation for all users. Metaphors and analogies have been used to aid the user by providing a mental model of the system, and have been shown to be quite effective (Carroll & Mack, 1985). Another successful interface solution has been the use of graphical user interfaces (GUIs) which utilize the spatial metaphor to create physical representations of these metaphors. However, these solutions tend to differentially benefit high SVA users, so that low SVA users (users who are "SVA Challenged") still lag behind users with high SVA (Butler, 1990). That is, they provide the greatest improvement for those individuals who need it least. This is not a bad thing, but what is needed is something to target those individuals who are "at risk", so that they do not get left too far behind, technologically, only due to their decreased visualization ability. Therefore, we should design with options to make navigation easier for those users, so as to accommodate for large variances in SVA. Two options which we suggest to target this problem are: interface apparency and interface manipulability. Interface apparency involves revealing hidden relationships --making these relationships apparent, and thus "off-loading the spatial processing of images from the user to the interface" (Norman, 1994, p.201). Likewise, interface manipulability involves understanding the underlying contingencies and rearranging the image, thus allowing the user to manipulate the image externally. Studies by Norman & Butler (1989a, also reported in Norman, 1991, p. 312-313) used graphical information to reveal hidden relationships with dramatic benefits. They looked at four conditions: (1) buttons only (no apparency), (2) buttons plus all links (non contingent apparency), (3) buttons plus all links to the goal (goal apparency) and (4) buttons with links from the start to the goal (start/goal apparency). Their findings showed that the first two conditions required the users to employ trial and error in order to reach the goal, since they were given no external information on hidden contingencies. However, subjects in the last two conditions reached optimal performance level quickly, thus "nullifying any differences in SVA" (Norman, 1994, p. 201).

The present study attempts to continue the research started with the Norman and Butler (1989a) study. We will be looking at four different methods of representing the underlying structures and contingencies for use in pull-down menus. These four conditions are: (1) Non Apparent (similar to the Non Apparent condition in the Norman and Butler, 1989a study) (2) Color Apparent (buttons leading to the goal button will be highlighted in the appropriate color), (3) Blinking Apparent (buttons leading to the goal button will blink), and (4) Path Apparent (path lines leading to the goal button will be displayed as the user selects each button). We hope these results will suggest methods for using interface apparency to help "SVA Challenged" individuals navigate through the system.

In addition, we propose a model describing the characteristics of (a) the user, in terms of SVA and the tendency to internalize the path to the goal, and (b) the apparency of the interface, and relate them to how the users approach the task of "discovering" the underlying structure and relationships which describe the correct paths (see Figure 1). In the model the user is defined on a continuum from high SVA to low SVA, and the interface is defined on a continuum from high interface apparency to low interface apparency. These characteristics will determine how the user progresses through the task.

Figure 1. Model of Path Processing

For this experiment we will be looking at the dependent measures of mean time to solution and mean number of moves. We hypothesize that the difference between high and low SVA subjects will be less in the Path Apparency Condition, the Color Apparency Condition and in the Blinking Apparency Condition than in the Non Apparent Condition. In addition, practice effects will be investigated over trials. In the Non Apparent condition, practice over trials should result in the learning of underlying contingent relationships and should depend on SVA. In the other conditions, performance is predicted to jump quickly to an asymptotic level shared by subjects of both high and low SVA.

Method

Participants

One hundred and sixty undergraduate students enrolled in introductory psychology courses at The University of Maryland participated in this experiment. Subjects were randomly assigned to one of four conditions and run in groups of up to 20 subjects at a time. The proportions of participants among the groups was as follows: (1) the Non Apparent condition (N = 43), (2) the Color Apparent condition (N=42), (3) the Blinking Apparent condition (N=39) or the Path Apparent Condition (N = 36). The distribution of High and Low SVA scores (determined by a median split with High SVA = 13-20 and Low SVA = 0 -12) was: (1) High SVA (N=86), (2) Low SVA (N = 74) and the distribution of High and Low Reading Comprehension scores (determined by a median split, after removing empty data points with High SVA = 31-38 and Low SVA = 0-30) was: (1) High Comprehension (N=76), (2) Low Comprehension (N = 70), with 20 subjects not completing this portion of the experiment. The 72 female and 88 male participants ranged in age from 17 to 46 years, with the mean age of 19.7, and took part in the experiment in order to gain extra credit as part of their course requirements.

Design

A 2 x 4 design was used to investigate the interaction and main effects of Spatial Visualization Ability (SVA) and Apparency. The independent variable, Apparency has 4 levels (Non Apparent, Color Apparent, Blinking Apparent, or Path Apparent). In the Non Apparent condition, the subjects were not given any information to help them find the goal. In the Color Apparent condition the subjects were given information, in the form of colored buttons, which would help them find the goal button (see Figure 2). In the Blinking Apparent condition the subjects were given information, in the form of blinking buttons (blinking on and off at 1/2 second intervals, which would help them find the goal button. Finally, in the Path Apparent condition the subjects were given information, in the form of lines between the chosen buttons, which would help them find the goal button (see Figure 3). Prior to the experiment, subjects took a 20 question pre-experimental test of Spatial Visualization Ability as measured by the VZ-2 cognitive test (Ekstrom, French, & Harmon, 1976) as well as a Reading Comprehension Test (The Nelson Denny Reading Test) on the computer. A median split for the VZ-2 score for number correct was used to determine level of SVA. Subjects scoring at or above 13 (out of 20) were assigned to the High SVA group and those at or below 12 were assigned to the low SVA group.

Materials

This experiment was run in the AT&T Teaching Theater, located at the University of Maryland in College Park (Shneiderman, Alavi, Norman, & Borkowski, 1994). The room contains 20 workstations all networked together and with two instructor's computers. Each workstation is an AT&T Globalyst 620 unit with a Pentium-based 75 MHz processor with 16MB of RAM and 570MB hard disk. All units are linked together using an AT&T Starlan™ network and through a Novell™ server, which in turn, are linked to the Internet. A workstation is composed of a keyboard, a mouse, and a high resolution color monitor recessed into the desk to conserve space. Designed for noise reduction and comfort, the room has wall-to-wall carpeting and the computer units are stored in an adjacent room.

The software used for this experiment was Object Plus™ that runs under Windows™ in the Teaching Theater. Both the experimental program and the VZ-2 program were created using Object Plus™.

Procedure

Up to twenty subjects at a time participated in this experiment. Each was seated at a computer running the experimental program and set up to run one of the 4 conditions. At the beginning of the session, the experimenter gave some basic instructions pertinent to all subjects, however, once the experiment began, it was completely driven by the computer.

The subjects were first asked to fill in a brief background information questionnaire (Age, Sex, High School G.P.A., SAT scores, years of computer use). Next they had up to 6 minutes to complete a 20 question multiple choice paper folding test of Spatial Visualization Ability (the VZ-2). Next they had 20 (?) minutes to complete a 38 question multiple choice test of Reading Comprehension. Then each subject went through the 20 trials of the experiment that consisted of four different screens, each with a different goal --> Red, Yellow, Blue or Green. These four screens were randomly displayed five times each, for a total of 20 trials. The object of the experiment was to select a pull-down menu, select buttons from that menu, moving from the leftmost column of buttons to the rightmost, which would bring the subject to the goal button (see Figures 2 and 3). The subjects had to pull down the menus to access the buttons, and were only able to see the buttons under each menu when that menu was selected (except for the set of selected buttons in the Path Apparent condition) (see Figure 3). Not all of the buttons would lead to the goal, only certain paths were defined to reach it. If the subject chose a correct path, then a "CORRECT" message was displayed and the subject was taken to the next screen. Otherwise, the subject received an incorrect path message, and by clicking on the reset button was given another chance to find the goal. The subject could not move on to the next screen until a correct path had been chosen. At the end of the 20 trials the subject was debriefed on the experiment and on the particular condition.

Figure 2. Sample screen from experiment -- Color Apparent condition.

(the Green button is displayed in the color Green, whereas all other buttons are either white or grayed out)

Figure 3. Sample screen from experiment -- Path Apparent condition.

Results

The times to solution from the 20 trials were averaged to determine the mean time to solution for the four conditions (Apparency Type) (1) Non Apparent, Color Apparent, Blinking Apparent, and Path Apparent and (2) SVA Level (High versus Low), and their interaction. The 2 factor ANOVA revealed a significant difference for the main effect of Apparency Type (F(3,152)=12.683, p<.01), and for the main effect of SVA Level (F(1,152)=6.189, p<.05) but not for the interaction effect of Apparency Type and SVA Level. A comparison of means gave the following average times for the different Apparency Types: M(Non Apparent)=17.319, M(Color Apparent)=13.270, M(Blinking Apparent)=22.528, and M(Path Non Apparent)=13.607, and for SVA Level: M(High SVA)=15.247, M(Low SVA)=18.369. In addition, the following table (Table 1) and figure (Figure 4) describe the means comparison for each group within the interaction. Post-hoc comparisons using a Fisher's LSD showed the significant differences to be between: (1) Path Apparent and Blinking Apparent (p<.05), (2) Path Apparent and Non Apparent (p<.01), (3) Color, Apparent and Blinking Apparent (p<.05), and (4) Color Apparent and Non Apparent (p<.05).










Table 1. A means comparison of the interaction of Apparency Type and SVA Level for Mean Time to Solution.

Figure 4. Bar graph of the mean contrast between Apparency Type and SVA Level for Mean Time to Solution with 95% confidence error bars

Next, the number of moves from the 32 trials were averaged to determine the mean number of moves for the two conditions and their interaction. The 2 factor ANOVA revealed a significant difference for the main effect of Apparency Type (F(3,157)=4.032, p<.01), but not for the main effect of SVA Level, nor for the interaction effect of Apparency Type and SVA Level (High versus Low SVA). A comparison of means gave the following average times for Apparency Type: M(Non Apparent)=7.076, M(Color Apparent)=5.353, M(Blinking Apparent)=6.778, and M(Path Non Apparent)=5.128. In addition, the following table (Table 2) and figure (Figure 5) describe the means comparison for each group within the interaction.  Post-hoc comparisons using a Fisher's LSD showed the significant differences to be between: (1) Path Apparent and Blinking Apparent (p<.05), (2) Path Apparent and Non Apparent (p<.01), (3) Color Apparent and Blinking Apparent (p<.01), (4) Color, Non Apparent (p<.05), and Non Apparent and Blinking Apparent (p<.05).

Table 2. A means comparison of the interaction of Apparency Type and SVA Level for Mean Number of Moves.


Figure 5. Bar graph of the mean contrasts between Apparency and SVA Level for Mean Number of Moves with 95% confidence error bars

In comparing the four Apparency levels for practice effects over trials, we found that the subjects in the Non Apparent condition took the longest to learn the task and, in fact, did not reach the same level of speed as the subjects in the other three conditions, as described by Figure 6, which shows trials 1-4 and the final trial (20) for the four different conditions. For both the Color Apparent and the Path Apparent condition, the first 4 points are all near optimal, and for the final trial of the Color Apparent condition the performance is also near optimal. For the Path Apparent condition, the performance on the last trial is actually at the optimal level (4 moves). In the Blinking Apparent condition, there is a large jump from the first trial to the second, and then to the near optimal level, however, for the Non Apparent condition, all 4 points are much farther away from optimal performance. It is also important to note, the degree of variability in the Non Apparent and Blinking Apparent conditions as compared to the Color Apparent and Path Apparent conditions.


Figure 6 Graph of the Means for the first 4 trials and the final trial (trial 20) for the Non Apparent, Color Apparent, Blinking Apparent, and Path Apparent Conditions

This same process was repeated for Mean Time to Solution, which showed a similar trend. The graph for these values is shown below (Figure 6):


Figure 7. Graph of the Means for the first 4 trials and the final trial (Trial 20) for the Non Apparent, Color Apparent, Blinking Apparent, and Path Apparent Conditions

Again, performance in the Color Apparent and Path Apparent conditions moved more quickly toward the optimal level (approximately 8 seconds), and the Blinking Apparent condition displayed the worst overall performance, with time to solution never breaking the 10 second mark).

In addition to looking at how SVA affected performance, we were curious to see whether Reading Comprehension would have any relation. Our findings did not show an interaction effect for either Mean Number of Moves, nor for Mean Time to Solution. In addition, there was no main effect for Mean number of Moves. However, there was a significant difference for the main effect of Mean Time to Solution, (F(1,138) = 8.206, p<.01). A means comparison showed that M(High Comprehension)=15.240, and M(Low Comprehension)=17.994, and the breakdown of the means comparison for interaction effects is shown in table 3 and figure 8.

Table 3. A means comparison of the interaction of Apparency Type and Reading Comprehension Level for Mean Number of Moves.


Figure 8. Bar graph of the mean contrasts between Apparency and Reading Comprehension Level for Mean Number of Moves with 95% confidence error bars

It should be noted here that in a previous, related experiment an additional test was performed which involved the test-retest validity of the VZ-2 on-line test. In this experiment, the VZ-2 was given once, prior to the experiment, and once after completion of the experiment. The test-retest reliability was found to be good with the Pearson Product Moment Correlation Coefficient of r=.76.

Discussion

This experiment examined the use of revealing underlying contingent relationships in pull down menus in order to assist users with navigation. By making these contingencies and relationships apparent it will decrease the number of moves and the amount of time it takes to reach the desired goals and to avoid following erroneous paths. This particular research was targeted at users with low SVA since many users with high SVA tend to feel more comfortable exploring and discovering the paths to the goal. Users with low SVA would probably find this inconvenient or even frightening and might therefore avoid using the technology for fear of getting lost.

Results of this experiment indicate that the subjects who were able to utilize the underlying contingent relationships revealed by the Path Apparent and the Color Apparent condition, performed significantly better than the subjects in the Non Apparent and the Blinking Apparent conditions. In fact the users in the Non Apparent condition performed significantly faster than the users in the Blinking condition (It seemed that the blinking buttons was really more of a distraction than a help!). The fact that there was no significant difference between High and Low SVA individuals between conditions shows that the Path Apparency and Color Apparency do assist all users, however, even though this was not a significant result the Low SVA users performed better -- fewer average moves and less average time -- than the High SVA users in the Color Apparent condition, whereas the High SVA users performed better -- fewer average moves and less average time -- than the Low SVA users in the Path Apparent condition.

An additional supporting finding, in terms of practice effects measured over trials, was that in the Color Apparent and Path Apparent conditions, users performed at near optimal level (in terms of number of moves) from the first trial! This implies that they really didn't have to spend much time learning the underlying structure and contingent relationships, they were able to just follow the path that they were given. Performance (specifically in terms of number of moves) of the users in the Non Apparent condition was far worse for the first four trials, although optimal performance was achieved by the 20th trial.

Another interesting finding in terms of Reading Comprehension was that there was a significant difference between users who scored high (High Comp) on the Reading Comprehension test and those who score Low (Low Comp) for Time to Solution. All users, though, did show an improvement in the Path Apparent and Color Apparent conditions over the Blinking Apparent and the Non Apparent conditions.

In general, these results indicate that revealing hidden contingencies does provide a benefit users. By providing information about the underlying path structure users are able to quickly follow these paths to achieve their goal states. This particular experiment suggests that while both Path Apparency and Color Apparency will assist all users, the use of color cues may be even more beneficial to the Low SVA users. These findings will become even more critical as the networks and databases of information become larger and more difficult to navigate and manage. The use of apparency will provide important guideposts for users with Low SVA and will help ensure that these people do not get lost along the way.

This work was supported by a grant from NSF (IRI - Interactive Systems 01-5-23124). We would also like to thank Project Director, Walt Gilbert, Tara Stachura, and Stephanie ---- for the use and scheduling of the AT&T Teaching Theater and, our undergraduate assistants Brad Bebee and Randy Pagulayan for their help in the data collection process.

References

Butler, S. A. (1990) . The effect of method of instruction and spatial visualization ability on the subsequent navigation of a hierarchical data base. (CAR-TR-488 and CS-TR-2398) Department of Psychology and the Human/Computer Interaction Laboratory, University of Maryland, College Park, MD

Carroll, J. M., & Mack, R. L. (1985). Metaphor, computing systems , and active learning. International Journal of Man-Machine Studies, 22, 39-57.

Ekstrom, R. B., French, J. W., & Harmon, H. H. (1976). Manual for kit of factor-referenced cognitive tests. Princeton, NJ: Educational Testing Service.

Lohman, D. F. (1989). Human intelligence: An introduction to advances in theory and research. Review of Educational Research, 59 , 333-373.

Nelson-Denny Reading Test, Forms G & H. (1993). Boston: Houghton Mifflin.

Norman, K. L. (1994). Spatial Visualization -- A Gateway to Computer-Based Technology. Journal of Special Education Technology, 12, 195-205.

Norman, K. L . (1991). The psychology of menu selection: Designing cognitive control at the human/computer interface. Norwood, NJ: Ablex.

Norman, K. L., & Butler, S. (1989a). Apparency: Guiding sequential decision making by revealing inherent contingencies. Paper presented at the Society for Judgment and Decision Making, Atlanta, GA.

Norman, K. L., & Butler, S. (1989b). Search by uncertainty: Menu selection by target probability. (CAR-TR-432 and CS-TR-2230). University of Maryland, Center for Automation Research and the Department of Computer Science, College Park, MD.

Salthouse, T. A., Mitchell, D. R., Skovronek, E., & Babcock, R. L. (1989). Effects of Adult Age and Working Memory on Reasoning and Spatial Abilities. Journal of Experimental Psychology: Learning, Memory and Cognition, 1989, 15, 507-516.

Salthouse, T. A. , Babcock, R. L., Mitchell, D. R., Palmon, R., and Skovronek, E. (1990). Sources of individual differences in spatial visualization ability. Intelligence, 14, 187-230.

Shneiderman, B., Alavi, M., Norman, K., & Borkowski, E. Y. (1994). Windows of Opportunity in Electronic Classrooms. Communications of the ACM, 38, 19-24.

Vincente, K. J., Hayes, B. C., & Williges, R. C. (1987). Assaying and isolating individual differences in searching a hierarchical file system. Human Factors, 29, 349-359.