Effectively using selection frequency to organize menus
Human-Computer Interaction Lab &
Computer Science Department
Institute for Systems Research†
University of Maryland
College Park, MD 20742
June 7, 1993
CR Categories: H.5.2
General Terms: Human Factors, Design
Keywords: Menus, user interface, human-computer interaction, selection frequency, experiment, split menus
Abstract: When some items in a menu are selected more frequently than
others, as is often the case, designers or individual users may be able
to speed performance and improve preference ratings by placing several
high-frequency items at the top of the menu. Design guidelines for split
menus were developed and applied. Split menus were implemented and
tested in two in situ usability studies and a controlled experiment. In
the usability studies performance times were reduced by 17 to 58% depending
on the site and menus. In the controlled experiment split menus were significantly
faster than alphabetic menus and yielded significantly higher subjective
preferences. A possible resolution to the continuing debate among cognitive
theorists about predicting menu selection times is offered. We conjecture
and offer evidence that, at least when selecting items from pull-down menus,
a logarithmic model applies to familiar (high-frequency) items and a linear
model applies to unfamiliar (low-frequency) items.
* This author’s current address is: DePaul University, Department of Computer Science,
243 South Wabash Avenue, Chicago, IL 60604.
Menus are an increasingly popular method of interacting with computers and therefore, designers are paying greater attention to menu organization so as to speed learning and performance. Designers must not only decide the overall organization of the menus for an application, they must also decide how to organize the items within each menu. Typical choices include alphabetical, logical, or categorical organizations.
When menus are relatively short, as they are in many commercial products, traditional organizations work well. However, situations exist where menus are long (e.g. control panels, font menus, b-board lists, and many custom applications) and alternative organizations may prove useful. When menus get longer and a small subset of items are selected more frequently then the remaining items alternate methods for selecting these high-frequency items may prove useful. One alternative is to assign special key combinations that can be used to select these frequently used items. This works well in some situations, but it can quickly become overwhelming. For instance, forty-one command key combinations are defined for the editor being used to write this paper. A small subset of these key combinations prove very useful, but if users do not remember the necessary combination all benefits are lost. Reorganizing menu items will allow faster selections while maintaining the standard selection mechanism and eliminating the need for users to remember additional commands.
In every day life, people often consider how frequently they use items when organizing them. Phone books often have an easily accessed section for frequently dialed numbers and bookcases are often organized with a section for frequently used books. Several researchers have suggested organizing menu items by how frequently they are selected (Brow88, Norm91, PaRo88, Shne92, SmMo86). Often the advice is simply to place the most frequently selected items at the top of the menu. When more detailed advice is given, designers are instructed to place the most frequent item at the top and the least frequent item at the bottom, implying that all items should be ordered by how frequently they are selected. This can lead to faster selections, but users may feel that the menu has no obvious organization.
When giving guidelines for formatting data, Smith and Mosier (SmMo86) suggested: "Where some data items are used more frequently than others, consider grouping those items at the top of the display." They do not indicate how to select these "high-frequency" items, how to organize them at the top of the display, or what to do with the remaining items. Refining this suggestion, and applying it to menus results in split menus (Sear93a).
Split menus are created by splitting a menu into two sections. Designers or individual users may place frequently selected items in the top section and infrequently selected items in the bottom section. Split menus should prove useful when a small subset of the menu items represent the majority of selections. By moving these frequently used items to the top of the menu, users should be able to locate and select them more rapidly. As the length of the menu increases, the potential benefits of split menus also increase.
This research investigates the efficacy of split menus and offers a
theoretical model for predicting performance times. We suggest a resolution
to the continuing debate among cognitive theorists about predicting menu
selection times. We conjecture and provide evidence that a logarithmic
model applies to familiar (high-frequency) items and a linear model applies
to unfamiliar (low-frequency) items.
This paper discusses details of split menus including a model that predicts the benefits. Two in situ usability studies are described which demonstrate the potential of split menus in normal working conditions followed by a controlled experiment which provides evidence supporting the efficacy of split menus and the accuracy of our cognitive model. Finally, guidelines are proposed to assist designers in creating split menus.
2. Related research
People often apply frequency of use to organize objects to make frequent tasks easier. Similarly, using the frequency of actions to organize user interfaces should result in faster performance as well as improved user preference ratings (RuHe86, Sear93b).
Many menu organizations have been suggested such as alphabetical, logical, categorical, or frequency of use. Alphabetical organizations order the items based on the lexical order of item names and are one of the orderings referred to as ‘traditional’ throughout this paper. Traditional organizations include alphabetical, numerical, and chronological (e.g. Monday, Tuesday, Wednesday,...). Logical organizations order menu items by the logical relationships between the items (e.g. Inches, Feet, Yards,...). Determining what is a logical organization can be a highly subjective task. A categorical organization can often be imposed on a group of items (e.g. ways of opening files, ways of closing/saving files, ways of exiting,...). Categorizing items is also highly subjective. Frequency based orderings typically refer to placing the most frequently used item at the top of the menu, followed by the next most frequent item. This continues until all items are placed in the menu. Many guidelines documents suggest using one or more of these organizations depending on the items being displayed (Appl87, MIL91, OSF90, SmMo86, Sun90). Additional methods of organizing items include: sequential (listed by sequence of use), functional (similar to logical), and importance (place critical items first) (Brow88, Norm91).
Several studies have investigated the effects of menu organizations on user performance. Somberg compared four menu organizations: alphabetic, probability of selection, random, and positionally constant (Somb87). However, it is important to remember that for the alphabetic, probability, and random organizations menu items changed positions between each selection making it impossible to learn the location of an item. Initially, alphabetic or probability ordering were fastest, but after practice menus that maintained a constant position for each item proved fastest. Random organizations were the slowest throughout the study. These results indicate that keeping words in fixed locations is better than allowing the words to move within a menu. However, it does not provide a comparison between various methods of organizing items in a positionally constant menu. Card compared positionally constant alphabetical, categorical (called functional by Card), and random organizations (Card82). Alphabetically ordered menus were the fastest and randomly ordered menus were the slowest. These results indicate that in addition to keeping menu items in fixed locations, a meaningful organization should also be used. While there is no simple answer to the question of which organization to use, it is clear that providing users with a stable menu that uses a known organization results in significant benefits.
Another alternative is to dynamically organize the menu based on the current frequency of selection. This could lead to a menu that changes automatically after users make selections, or to a system that is under user control and only changes when the user decides that a change would be beneficial. Mitchell and Shneiderman compared static menus and menus that were automatically reorganized based on the users’ current pattern of selections and found that users preferred static menus (MiSh89). In addition, when comparing the first exposure to the system, users were faster and made fewer errors with static menus. After practice there was no difference in performance, but users still preferred static menus. Greenberg and Whitten investigated the benefits of organizing menus based on an a priori set of frequencies and updating the menu to reflect recent usage as users make selections (GrWi85). The results suggests that organizing items by frequency and recency of use may prove useful. These two studies indicate that automatically updating menus to reflect current usage patterns may be useful, but can also lead to problems.
Other attempts at speeding menu selection have used non-linear menus. Callahan, Hopkins, Weiser, and Shneiderman investigated the benefits of circular (pie) menus which make the distance to each item equal (CHWS88) while Walker and Smelcer explored the benefits of making menu items larger the farther down they were in a menu (WaSm90). Both of these research efforts focused on making the movement to a menu item easier and both demonstrated that this can save users time.
3. Split menus
The benefits of using split menus depend on two factors: how often each item is selected and where the frequently selected items are located in the traditional and split menus. If all items are selected with near equal frequency, minimal benefits would be expected for split menus. However, if a few items represent the majority of selections, as is often the case with computer commands (Gree88), it is likely that split menus will improve performance. If the frequently selected items are located at the top of the traditional menu minimal benefits would be expected, but if they are at the bottom of the traditional menu, split menus should prove beneficial. However, even in situations where users do not save time, they may prefer a menu organization that emphasizes frequently selected items.
Organizing split menus
To facilitate rapid scanning of both the low- and high-frequency sections, particularly if the menu is relatively large, both sections should be presented in the ‘traditional’ order the entire menu would have been presented in. For instance, if the menu contains names, it would be organized alphabetically. If it contains dates, chronological ordering would be used. Organizing both sections in a traditional order allows users to search each section, as they would normally search the entire menu, quickly locating the item of interest.
We developed preliminary guidelines which help decide which items should be placed in the high-frequency section of a split menu. These guidelines demonstrate the potential of split menus and are revised later in this paper based on the results of the field studies and the controlled experiment.
Preliminary Guideline 1: Limit the number of items in the high-frequency section to four or less in most situations. This will allow users to quickly scan and remember the high-frequency items. A few more items may be placed in the high-frequency section to preserve some meaningful organization or categorical relationship.
Preliminary Guideline 2: Sort all items by selection frequency. Starting with the least frequently selected item, scan until the increase in frequency between two successive items is greater than the mean of the frequencies. Once this point is located, all items on the high-frequency side of this point are placed in the high-frequency section. If there are more than four items, only the four most frequently selected items are placed in the high-frequency section.
Applying these preliminary guidelines to the menu in Figure 1, using hypothetical selection frequencies, results in Figure 2.
Figure 1: Traditional menu. Figure 2: Split menu.
Predicting the benefits of split menus
If the benefits of using a new menu organization are uncertain, users may be hesitant to risk changing their system. However, if a reasonably accurate prediction can be made, users may be more willing to switch to a new organization. To predict the amount of time that will be saved we must understand what happens when users select items from a menu. First, there are at least two different types of selections that users make. When users know the name of the item they want, they must locate and select it. When users know the action they want to perform, but do not know the name of the menu item that performs that action, they must search for an item that they feel performs the desired action and select it. Our discussions will be limited to interactions where users know the name of the item they want to select. Split menus may also be beneficial when users are unsure of which item to choose because likely choices are displayed at the top of the menu.
Several models have been developed to predict the amount of time necessary to select an item from a menu. These models typically fall into one of two categories: linear or logarithmic (LaNa85, LeMa85, Norm91, PaRo86). Linear models predict that the amount of time to select an item is a linear function of the position of the item in the menu (LeMa85). These models typically assume that the majority of the selection time is spent searching the menu in a linear fashion for the desired item. Logarithmic models predict that the amount of time to select an item is a logarithmic function of the position of the item in the menu. Logarithmic models assume that users do not scan the menu for the item linearly, but use the order of the menu items to search more efficiently. For instance, users may move to the center of the menu and decide if the desired item is above or below the center. Using this information users move to the middle of the upper or lower half of the menu and repeat the process until the desired item is located. Fisher, Yungkurth, and Moss developed a model that accounted for different menu items being selected with different frequencies and extended previous work with the linear model to a larger variety of menus (FYM90).
The first limitation of these models is that they assume that users select all menu items using the same strategy. We believe that different strategies are used depending on how familiar users are with the item being selected. Second, each of these models assumes menu selection is performed by pressing a key on a keyboard. This is not accurate for many mouse based computer menus which are widely available. Our model must deal with menu selections which are performed using a mouse and a pull-down menu. When selecting items from a pull-down menu users must move the cursor to the top of the menu, press a mouse button to select the menu, move the cursor to the desired item while holding the mouse button down, and release the mouse button to select the highlighted item. During the process of searching for the correct item, the mouse cursor acts as a visual anchor changing the way the search proceeds.
We suggest a cognitive model in which users are not equally familiar with all items in a menu. Users’ familiarity with an item is assumed to be a function of how frequently they select the item. Items that are selected frequently will be more familiar than items selected infrequently.
Users quickly learn the location of frequently selected items allowing them to move the cursor directly to the item of interest. This indicates that the time to select a frequently selected menu item should be dominated by the time necessary to move the cursor to the correct location. Therefore, selection times for high-frequency items should be a logarithmic function of the location of the item in the menu as would be predicted by Fitts’ Law. Infrequently selected items do not become familiar to users, therefore users cannot automatically move to the correct location in the menu. This lack of familiarity forces users to scan the menu for the desired item. Since users must move the cursor down the menu to the item of interest, the cursor acts as a visual anchor guiding their search. This results in users scanning the first item, then the second, third, etc. until the desired item is located. Therefore, the selection time for an infrequently selected item should be a linear function of the position of the item in the menu. Selection times for items that are selected occasionally would be predicted by some intermediate model, but for the purpose of predicting the benefits of split menus, either a linear or logarithmic model will be used for each item in the menu.
Split menus divide menu items into two categories: high- and low-frequency items. Assuming users are familiar with the high-frequency items, a logarithmic model will be used to predict times for these items. Similarly, assuming users are not familiar with the low-frequency items, a linear model will be used to predict times for these items. Of course, users will be somewhat familiar with many of the low-frequency items so predictions are expected to be less accurate for the low-frequency items than for the high-frequency items.
To predict the amount of time that will be saved, or lost, by switching from a traditional organization (alphabetical) to split menus the following values must be known:
• LT(x) The location of item x in the Traditional Menu (often organized alphabetically),
in Figure 1 LT(Courier) = 6, LT(Helvetica) =9, and LT(New York) = 20,
• LS(x) The location of item x in the Split Menu, in Figure 2 LS(Courier) = 1, LS(Helvetica) = 2, and LS(New York) = 22, L(x) is equal to one plus the number of items above item x in the menu (including the dividing line for split menus),
• tT(x) The average amount of time to select item
x in the Traditional menu,
• SHF The slope of the regression equation for the high-frequency (HF) items. The regression equation for high-frequency items is based on log2 (LT(x)) and tT(x).
• SLF The slope of the regression equation for the low-frequency (LF) items. The regression equation for low-frequency items is based on LT(x) and tT(x).
Expected Benefits = SHF * _ f(x)*[log2 (LT(x)) - log2 (LS(x))]+ SLF * _ f(x)*[LT(x) - LS(x)]
HF items LF items
The first part of this equation provides an estimate of the amount of time users will save when selecting the high-frequency items. The second part provides an estimate of the amount of time users will save (or lose) when selecting low-frequency items. The result, the Expected Benefits, is the average amount of time (in seconds) that users can expect to save when selecting items from a split menu as compared to a traditional (often alphabetical) menu.
Of course, if the menu has not been implemented, it is impossible to compute the necessary regression equations. Using the experience of the usability studies and the controlled experiment, reasonable values for the slopes of the regression equations will be presented. These results are presented in the section that discusses the accuracy of the Expected Benefits formula.
4. Two in situ usability studies
The first step in evaluating split menus was to install them in normal working environments. The goal of these in situ usability studies was to demonstrate the potential of split menus when real users were performing the tasks they normally do. We use the term in situ usability studies to describe data collection that takes place in the users’ own environment rather than in a usability lab with artificial tasks. If split menus are successful, additional studies can be conducted to provide a better understanding of exactly what happens when users make selections from these pull-down menus.
A program that created split menus for the font menus in MacWrite and Microsoft Word was installed on Macintosh computers at two sites: the Computer Science Department at the University of Maryland and NASA - Goddard Space Flight Center. Data collection software ran for four weeks while users continued using the standard menus provided by their system (Phase One). At the end of four weeks, selection frequencies were analyzed to determine how the menus should be reorganized. This information was used to create split menus for both MacWrite and MicrosoftWord. Data collection continued for an additional five weeks (Phase Two). Split menus were used for five weeks to allow users to adjust to the new menu organization. After the split menus were used for five weeks, the software was removed and the data were analyzed.
There were six users of the four Macintosh computers at the University of Maryland. All four computers had the same font menus, allowing them to be analyzed as one distributed system with multiple users. The font menus contained 28 items.
Eleven users at NASA, each with their own Macintosh and their own font menu, participated in the usability study. NASA font menus had between 6 and 18 items each with an average of 11. Each Macintosh at the NASA site was analyzed separately.
Once the data were collected it was filtered. Any menu item with three or fewer selections was eliminated due to the potentially large variability with such a small sample size. Next, any selection times which were more than twice as long as the next longest selection time for the same menu item were eliminated. A total of 5% of the selections were filtered before the data were analyzed. For Phase Two, only selections of items for which data were available from Phase One were used and the first 20% of the selections were considered practice and were not used in the analysis.
University of Maryland
All data from Phase One were used to reorganize the menus into split menus. The data were then divided into two parts: MacWrite and Microsoft Word selections. A total of 277 menu selections were recorded at the UMD site. Twelve fonts which were selected an average of 1.2 times each were filtered before the data were analyzed.
NASA - Goddard Space Flight Center
All data from Phase One were used to reorganize the menus into split menus. Since each system at NASA had a unique font menu, each was reorganized based on selections on that system alone. A total of 232 menu selections were recorded at the NASA site. Seven fonts which were selected an average of 1.8 times each were filtered before the data were analyzed.
Table 1 contains the average selection time for individual fonts, with the exception of the NASA data, as well as the mean for UMD - Word, UMD - MacWrite, and NASA - Word.
Courier 1.7 1.4 18
Helvetica 3.3 2.6 21
Times 3.7 1.7 54
UMD - MacWrite Mean 3.6 1.5 58
Courier 3.0 1.3 57
Times 3.8 1.6 58
NASA Mean 2.9 2.4 17
Table 1: Time necessary to select individual fonts and the average time for all fonts (in seconds).
At the end of the usability studies, participants were asked which of the two styles of menus they preferred. Of the thirteen participants that answered, one preferred the alphabetical menus, nine preferred the split menus, and three expressed no preference.
As expected, the split menus reduced the average selection time for all menus. The results of the usability studies illustrate the practical potential of split menus. Even with users who were familiar with the traditional alphabetical ordering of font menus, split menus resulted in faster selections after limited practice. In addition, 92% of the participants either had no preference or preferred the split menus.
Individual differences are important and must be accommodated whenever possible. At the University of Maryland site, multiple users shared multiple computers. Users were treated as a group and average selection frequencies for the group were used to create split menus. This strategy worked well, and selection times were reduced by 27 to 58% depending on the editor being used. At the NASA site, each user had their own Macintosh. In this situation each system was customized based on the individual user. Once again, this strategy proved effective reducing selection times by an average of 17%. In situations where groups of users work with a standardized subset of the available options, using average selection frequencies for the group may prove effective. However, when individuals working on one system have distinctly different usage patterns, it may be more effective to develop a method of providing a customized split menu for each user or to continue using traditional menus. If users must identify themselves to the system before using it, customizing the menus for each individual is simple. Otherwise traditional menus may remain the most effective interface.
5. Controlled Experiment
With the success of the usability studies, we proceeded to run a controlled experiment to develop a more precise understanding of how users select items from pull-down menus. The controlled experiment had several purposes. The first was to validate that split menus can result in faster selections. To demonstrate this, Alphabetical, Frequency-ordered, and Split menus were all evaluated in the experiment. Frequency-ordered menus were included to demonstrate that using frequency information alone is not sufficient. The second purpose was to evaluate our cognitive model for pull-down menu selection and the expected benefits formula. Since the benefits of split menus depend on how frequently each item is selected and where each item is located in the original (traditional) menu organization, three frequency distributions were explored. This resulted in a total of nine menu organization-frequency distribution combinations.
Thirty-eight frequent computer users were recruited as subjects from the University of Maryland campus at College Park. Two subjects were not included in the analysis due to failure to follow instructions. Subjects were primarily graduate students in the Computer Science and Electrical Engineering Departments. Subjects were offered $10.00 to participate in the experiment.
Design and Procedure
The experiment used a within subject design. Three menu organizations
were explored: Alphabetic, Frequency, and Split. Alphabetic menus were
organized based on the name of each menu item. Frequency menus were organized
with the most frequently selected item at the top and the least frequently
selected item at the bottom. Split menus were arranged based on the Preliminary
Guidelines. Three selection frequency distributions were explored. These
distributions assigned 60% of selections to the three high-frequency items
while in the usability studies the two or three high-frequency items represent
70-90% of selections. Distribution One had the frequently selected items
near the bottom of the alphabetic menu (Table 2). Distribution Two had
the frequently selected items near the middle of the alphabetic menu (Table
3). Distribution Three had the frequently selected items near the top of
the alphabetic menu (Table 4). Tables 2, 3, and 4 describe how frequently
each menu item was selected for each of the three menu organizations for
Distributions One, Two, and Three respectively. Note that the alphabetic
menus have all items in the traditional location, frequency-ordered menus
have all items in order by selection frequency, and split menus place the
frequently selected items at the top of the menu while maintaining the
traditional ordering for both the high- and low-frequency items.
Alphabetic Split Frequency
Item f(x) Item f(x) Item f(x)
1 2 11 24 11 24
2 4 13 20 13 20
3 0 15 16 15 16
4 8 1 2 4 8
5 2 2 4 8 8
6 2 3 0 12 6
7 4 4 8 2 4
8 8 5 2 7 4
9 0 6 2 1 2
10 2 7 4 5 2
11 24 8 8 6 2
12 6 9 0 10 2
13 20 10 2 14 2
14 2 12 6 3 0
15 16 14 2 9 0
Table 2: Frequency of selection for each menu item for Distribution
One in each of the three menu organizations.
Alphabetic Split Frequency
Item f(x) Item f(x) Item f(x)
1 2 6 24 6 24
2 4 8 20 8 20
3 0 10 16 10 16
4 8 1 2 4 8
5 2 2 4 13 8
6 24 3 0 12 6
7 4 4 8 2 4
8 20 5 2 7 4
9 0 7 4 1 2
10 16 9 0 5 2
11 2 11 2 11 2
12 6 12 6 14 2
13 8 13 8 15 2
14 2 14 2 3 0
15 2 15 2 9 0
Table 3: Frequency of selection for each menu item for Distribution
Two in each of the three menu organizations.
Alphabetic Split Frequency
Item f(x) Item f(x) Item f(x)
1 24 1 24 1 24
2 4 3 20 3 20
3 20 5 16 5 16
4 8 2 4 4 8
5 16 4 8 8 8
6 2 6 2 12 6
7 4 7 4 2 4
8 8 8 8 7 4
9 0 9 0 6 2
10 2 10 2 10 2
11 2 11 2 11 2
12 6 12 6 14 2
13 0 13 0 15 2
14 2 14 2 9 0
15 2 15 2 13 0
Table 4: Frequency of selection for each menu item for Distribution Three in each of the three menu organizations.
Every subject used each of the menu styles with each of the frequency distributions. Thirty-six evenly distributed permutations (using a Latin square design) were used to determine the order in which the menu-distribution combinations were used.
All menus contained fifteen items. Menu items were selected from a filtered list of the 1000 most frequently used words in printed English (Thor44). Words that began with capital letters, contractions, and words that were less than four characters or more than eight characters were eliminated from the list of possible menu items. Every subject used nine different menus which contained a randomly selected set of fifteen words from the remaining list. No word appeared in more than one menu for a given subject. The order of the selections was randomized for each subject and each menu. In addition, the order of selections was controlled to create four balanced blocks. The first, second, third, and fourth block of twenty-five selections each accurately represented the overall selection frequencies for the distribution.
Subjects used a Macintosh computer to perform the menu selections. Subjects were instructed that a word would appear on the screen and that they were to select the same word from the single pull-down menu which was available. Subjects were instructed to select the item as rapidly as possible while maintaining high accuracy since they would be required to repeat selections until they were correct. Subjects took a short break between each of the nine menus to prevent fatigue. When a subject began the experiment the type of menu they would be using (Alphabetic, Frequency, or Split) was presented on the screen. The menu item to be selected was then presented on the screen. Subjects had to move the cursor to the pull-down menu at the top of the screen, click and hold the mouse button (this is when we began recording selection times), drag the cursor to the correct item, and release the mouse button to select the highlighted item. If an error was made the subject was instructed to try again. Once the correct item was selected, there was a brief pause and the next item to be selected was presented to the subject. Subjects made 100 selections with each of the nine menus. Selection times and the number of errors were recorded for every selection. Once they completed the 100 selections for a given menu subjects took a short break before continuing with the next menu. When the subject completed all nine menus, they were asked to rank the menu organizations in order of preference (1=most preferred menu, 3=least preferred menu).
We expect different results for each of the three frequency distributions. Distribution One has the frequently selected items near the bottom of the alphabetical menu. Split menus should be faster than both alphabetic and frequency-ordered menus for this frequency distribution. Frequency-ordered menus should provide small benefits when compared to alphabetic menus for Distribution One. Split menus should still be faster than alphabetic menus for Distribution Two which has the frequently selected items near the middle of the alphabetic menu, but the benefits will be smaller. Frequency-ordered menus should provide even smaller benefits when compared to alphabetic menus for Distribution Two. Distribution Three, which has the frequently selected items near the top of the alphabetic menu, should result in no differences between the alphabetic and split menus. Frequency-ordered menus should result in a small negative impact when compared to alphabetic menus for Distribution Three. Overall, we expected users to prefer split menus due to the ease of accessing the frequently selected items and the alphabetical ordering which is useful when selecting low-frequency items. Due to the difficulty of understanding the organization, users are expected to rate the frequency-ordered menu as the worst of the three. Although the frequency-ordered menu may be faster than the alphabetic menu for some frequency distributions, the apparent random organization of the frequency-ordered menu will lead to lower preference rankings.
Mean preference rankings are presented in Table 3. The Freidman test was used to determine the extent to which the subjects ranked the menu organizations in the same order. The results indicate that subjects consistently rated split menus the best, alphabetic second best, and frequency-ordered menus worst (Cr2 = 25.68, p<0.001).
Alphabetic Split Frequency
Mean 2.00 1.40 2.60
SD (0.60) (0.73) (0.41)
Since performance at both the first exposure to the menu and after limited practice are important, mean selection times for the first and last blocks are presented in Table 4. Two 3x3 ANOVAs with repeated measures were performed for the first and last blocks separately. The ANOVA for the first block showed significant main effects for menu organization and frequency distribution (F(2,70)=11.39, p < 0.001; F(2,70)=18.05, p < 0.001 respectively). There was also a significant interaction between menu organization and frequency distribution (F(4,140)=4.17, p < 0.005).
The ANOVA for the last block showed significant main effects for menu
organization and frequency distribution (F(2,70)=20.87, p < 0.001; F(2,70)=17.16,
p < 0.001 respectively). There was also a significant interaction between
menu organization and frequency distribution (F(4,140)=9.57, p < 0.001).
Contrast matrices were used to perform post hoc tests comparing each of
the three menu organizations for the first and last blocks for each distribution.
The significance level was set at p < 0.05 using the Bonferroni technique.
Significant differences are summarized in Table 5.
Menu Organization Menu Organization
Distribution Alphabetic Split Frequency Alphabetic Split Frequency
One 1.80 1.64 1.76 1.67 1.43 1.52
(0.44) (0.41) (0.50) (0.44) (0.34) (0.44)
Two 1.79 1.67 1.84 1.66 1.52 1.55
(0.45) (0.46) (0.55) (0.48) (0.40) (0.44)
Three 1.47 1.56 1.72 1.38 1.39 1.55
(0.36) (0.40) (0.41) (0.32) (0.37) (0.42)
Table 4: Mean selection times in seconds (standard deviations in parentheses).
Distribution Block Significant differences
1 First Split < Alphabetic
1 Last Split < Alphabetic
1 Last Frequency < Alphabetic
3 First Alphabetic < Frequency
3 Last Alphabetic < Frequency
3 Last Split < Frequency
Table 5: Summary of significant difference from post hoc tests.
For Distribution One, the post hoc tests indicated that Split menus were significantly faster than the Alphabetic menu during both the first and last blocks. Frequency menus were also faster than Alphabetic menus during the last block (see Figure 4).
Figure 4: Average selection time vs. Block for three menu organizations for Distribution One.
The post hoc tests indicated that there were no significant differences
for Distribution Two. Although Split menus were faster than Alphabetic
menus during both the first and last blocks the differences were not significant.
Figure 5: Average selection time vs. Block for three menu organizations for Distribution Two.
For Distribution Three, Alphabetic menus were significantly faster than Frequency menus during both the first and last blocks and Split menus were significantly faster than Frequency menus during the last block (see Figure 6). There were no significant differences between Alphabetic and Split menus.
Figure 6: Average selection time vs. Block for three menu organizations for Distribution Three.
Mean error rates for the first and last blocks are presented in Table 6. Two 3x3 ANOVAs with repeated measures were performed for the first and last blocks separately. The ANOVA for the first block showed no significant main effects or interactions. The ANOVA for the last block also showed no significant main effects or interactions. Contrast matrices were used to conduct post hoc tests similar to those performed for the selection time data. No significant differences were found.
Menu Organization Menu Organization
Distribution Alphabetic Split Frequency Alphabetic Split Frequency
One 0.012 0.006 0.006 0.014 0.006 0.006
(0.025) (0.014) (0.014) (0.027) (0.014) (0.017)
Two 0.012 0.010 0.011 0.016 0.014 0.010
(0.023) (0.020) (0.025) (0.032) (0.032) (0.020)
Three 0.007 0.009 0.003 0.007 0.011 0.007
(0.018) (0.024) (0.011) (0.018) (0.021) (0.015)
Assuming the menu would normally be organized alphabetically, the benefits that can be expected by switching from an alphabetical menu to a split menu were calculated. The following computations used the average selection time for each menu item and combined the three alphabetic menus to generate the two necessary regression equations. Expected benefits were then computed for each item in each of the three Split menus as well as for the three distributions. These values were compared to the actual time savings.
Four other models, different from our log-linear model, for predicting the savings users could expect were evaluated. Each model used different combinations of linear and logarithmic equations to predict user performance:
2) One linear equation modeled the low-frequency menu items, and
a second linear equation modeled the high-frequency menu items.
3) A single logarithmic equation modeled all menu items.
4) One logarithmic equation modeled the low-frequency menu items, and
a second logarithmic equation modeled the high-frequency menu items.
The regression equation, based on the logarithm of the position in the menu, for the high-frequency items from the three Alphabetic menus is: T=0.199*log2 (LT(x))+0.948 with r2=0.96, p<0.001. This supports our conjecture that the logarithmic model predicts performance for the high-frequency items.
The regression equation, based on the position in the menu, for the low-frequency items from the three Alphabetic menus is: T=0.046*LT(x)+1.294 with r2=0.79, p<0.001. Although the correlation is lower than for the high-frequency items, as was expected, it is still relatively high. This supports our second conjecture that the linear model predicts performance for the low-frequency items. Although our initial conjectures are confirmed by these results, the correlation between the predicted savings and the actual savings is far more important. An analysis of these results follows.
Using the slopes from the two regression equations (SHF =0.199 and SLF =0.046) the expected benefits can be computed for each individual menu item as well as the three distributions as a whole. The expected benefit for an individual menu item is weighted by how frequently the item is selected. Figure 7 presents the expected vs. actual time saved for individual menu items. The regression equation for individual items is Actual = 0.97*Predicted + 0.00 and r2=0.98, p < 0.001. This is very close to the perfect equation of Actual = 1.00*Predicted + 0.00. Similarly, Figure 8 presents the overall results for the three distributions. The regression equation is Actual=0.88*Predicted - 0.03 and r2=0.97, p<0.09. This divergence from the ‘perfect’ equation is probably due to the small but consistent overestimate of the benefits for the high-frequency items and small but consistent underestimate of the negative effects of Split menus on low-frequency items.
Overall, our combined log-linear model predicts the performance for the low- and high-frequency menu items more accurately than any of the four simpler models. Of course, it is difficult to validate or invalidate any of these models based on a single experiment.
Figure 7: Predicted vs. Actual time saved (in seconds) for individual items.
Figure 8: Predicted vs. Actual time saved (in seconds) for three distributions.
Predicting benefits when a menu does not exist
If a menu has not been implemented, but expected frequency data is available, it may still be possible to estimate the benefits of split menus. Based on the limited data from the usability studies and the controlled experiment the slope of the high-frequency items can be estimated to be 0.20, and the slope of the low-frequency items can be estimated as 0.06. Using these values for SHF and SLF respectively should provide conservative estimates of the benefits that can be expected by using split menus. Additional data must be collected before more accurate default values can be provided.
As expected, there were no significant differences in the error rates. Subjects made between 0.3 and 1.6% errors depending on the menu organization, frequency distribution, and block.
Although users do become familiar with menu organizations, split menus would likely represent a small percentage of all menus on a computer system. In addition, different users may have different split menus depending on their particular usage patterns. Users who occasionally use another person’s computer, or rarely use a split menu, must still be able to make selections rapidly. Of course, performance after practice is important for regular users. Therefore, the results for both the first and last blocks were analyzed.
The results for Distribution One indicated that Split menus were significantly faster than Alphabetic menus in both the first and last blocks. Frequency-ordered menus were significantly faster than Alphabetic menus by the last block. These results indicate that using the frequency of selection resulted in a significant improvement over a purely alphabetical organization for this distribution. Combining the traditional organization with selection frequencies, as is done in split menus, appears to result in even larger benefits. On average, subjects took approximately 0.25 seconds, or 17%, longer per selection when using the Alphabetic menus than they did when using Split menus.
Distribution Two was an intermediate distribution. It was selected to provide a middle point between Distribution One (most favorable for Split menus) and Distribution Three (least favorable for Split menus). There were no significant differences between the three menu organizations for this distribution. However, if you compare the results with those of Distributions One and Three it becomes clear that the results are what should be expected. Alphabetic menus took an average of 0.14 seconds, or 9%, longer than Split menus for this distribution.
Distribution Three was chosen to represent one of the least favorable situations for Split menus. Although distributions where all items are selected with near equal frequencies would be less favorable for Split menus, the preliminary guidelines would not have recommended Split menus in that situation. Alphabetic menus were significantly faster than Frequency menus in both the first and last blocks. Split menus were significantly faster than Frequency menus by the last block. There were no significant differences between Alphabetic and Split menus. These results indicate that considering only the selection frequency results in slower selections for this distribution. However, if selection frequencies and the traditional organization are considered there is no additional cost over using the traditional organization alone. Interestingly, the proposed guideline presented in the next section would not have suggested using split menus for this frequency distribution.
Users preferred the Split menus over both the Alphabetic and Frequency menus. Even though Alphabetic menus were slower than Frequency menus for two of the three frequency distributions, users did not like the ‘random’ appearance of the frequency-ordered menus. These results provide support for the use of Split menus. Regardless of the distribution used, Split menus were always as fast or faster than Alphabetic menus. Even in the least favorable situation, where Split menus may be expected to be slower than Alphabetic menus, selection times were equal. In addition, users preferred Split menus, suggesting that even in situations where they are not very beneficial, users may prefer menu organizations that place the frequently selected items where they are easier to access.
Although the advantage was just a few tenths of a second in our experiment, this speed-up could make a substantial difference in high volume (e.g. airline reservation) or stressful (e.g. air traffic control) applications. This advantage will increase with menu length and with more skewed distributions.
6. Proposed Guidelines
Preliminary guidelines have been presented to assist designers and individual users in deciding if split menus should be used and how to organize items within the menu. Given the traditional menu organization and how frequently each item is selected, these guidelines could be automated and a split menu could be proposed if it would be beneficial. Users could then view the traditional and split menus to decide which they prefer to use. This section presents revised guidelines, based on the Expected Benefits formula, which should provide more helpful advice to designers. The following guideline uses the Expected Benefits formula to determine the number of items to be placed in the high-frequency section. This guideline maintains the requirement that the items in the high-frequency section be selected more frequently than those in the low frequency section.
• EB(x) is the expected benefit from moving the x most frequently selected items to the top of the split menu, EB(0) = 0. If multiple items are selected with the same frequency, the item closest to the bottom of the menu should be moved to the top of the split menu first. The requirement that all items in the high-frequency section be selected more frequently than items in the low-frequency section is easily maintained using this process.
• _EB1-2 is the difference between EB(1) and EB(2). In other words, _EB1-2 is the additional improvement that can be expected if two items are moved to the top of the menu instead of one.
• e is the minimum improvement that is necessary before additional items will be moved to the top of the split menu. Therefore, _EB1-2 must be greater than or equal to e (_EB1-2 _ e) before two items will be moved to the top of the menu.
• To determine the number of items to move to the high-frequency section of the split menu begin with _EB0-1. If _EB0-1 _ e then move at least one item to the high-frequency section. If _EB0-1 _ e and _EB1-2 _ e then move the two most frequently selected items to the top of the split menu. This process repeats until _EBx-y < e. This indicates that X items would be moved to the top of the split menu.
Of course, to calculate EB(x) the slope of regression equations for both the high- and low-frequency items must be known. Since the high-frequency items have not been specified, and multiple items must be specified for each section of the menu before the regression equations can be computed, we must estimate both slopes. As discussed near the end of Section 5, the slope for the high-frequency items can be estimated to be 0.20 and the slope for the low-frequency items can be estimated to be 0.06. Using these values, and an e of 0.05 seconds, the three frequency distributions from the controlled experiment were analyzed. The results appear in Table 7. The three most frequently selected items would be moved to the top of the split menu for Distributions One and Two. Split menus would not be recommended for Distribution Three. These recommendations match the results of the experiments well. Of course, additional research may refine the values for the slopes for both the high- and low-frequency items as well as e(a lower value for e , such as 0.03 would probably work well for creating split menus).
# HF items (x) Distribution One Distribution Two Distribution Three
1 0.10 0.07 -0.05
2 0.19 0.15 ----
3 0.25 0.20 ----
4 0.25 0.21 ----
5 ---- ---- ----
Table 7: Proposed guidelines applied to the three frequency distributions from the controlled experiment. The bold items in each column represent items that would be placed in the top of the split menu.
7. Future Directions
Several important questions must still be answered. When a system is first being used selection frequencies can be estimated until sufficient data can be collected. The question is: How much data is necessary before it is sufficient? A variant of this question becomes important once split menus are being used. Usage patterns may change over time, so the question becomes: How do we identify changing selection patterns which lead to suggesting changes to the split menus? These two questions are essentially the same: How do we identify when usage patterns have stabilized? Since selection frequencies will change with time, it is clear that we must place more emphasis on recent selections. This could be done by weighing recent selections more heavily (GrWi85), or by using only the most recent selections. Strategies must be developed that can monitor selection frequencies and visually alert users to consider reordering a menu.
An interesting alternative to the current version of split menus would be to leave the high-frequency items in the bottom of the split menu and duplicate them in the high-frequency section. This would ensure that users would locate the high-frequency items regardless of the section of the menu they searched, while maintaining the benefits of placing the items at the top of the menu. However, the results of the controlled experiment indicate that users scan the high-frequency list regardless of the item they are searching for, indicating that this organization would not prove beneficial. Another option would be to collapse the low-frequency items into a hierarchical menu. This would result in the high-frequency items being immediately accessible and the low-frequency items requiring the traversal of a two level hierarchical menu. This reduces the visual clutter and number of options users see, possibly speeding performance for high-frequency items. Extending this idea could result in a menu with three sections: high-frequency, medium-frequency, and low-frequency items hidden in a hierarchical sub-menu.
The usability studies demonstrated the potential of split menus in normal work conditions. Even users who had used the alphabetic menus for years saved time when using split menus (between 17 and 58% depending on the usability study site and menu). Ninety-two percent of users either expressed no preference or preferred the split menus.
The controlled experiment provided valuable support for the use of split menus and the refined cognitive model for pull-down menu selection. The controlled experiment also confirmed the potential of the Expected Benefits formula. The predictions for individual items closely approximated actual values, while the predictions for the three distributions provided valuable information concerning the expected impact of split menus. Actual benefits for split menus may be larger than in these studies if menus are longer or the skewness of the frequency distribution is larger as appears to be the case in many realistic situations.
The results of the controlled experiment not only demonstrated the time savings and higher preference ratings split menus create, but also demonstrated the value of the proposed guideline for creating split menus. The proposed guideline would have suggested the three most frequently selected items be move to the top of the split menus for Distributions One and Two, and that split menus were not appropriate for Distribution Three. These recommendations match the results of the experiment well.
Split menus provide the benefits of both frequency-ordered and alphabetical menus. Frequently selected items are moved to the top of the menu making them easy to locate and select, while both sections of the menu maintain a traditional (alphabetical) organization allowing users to quickly scan the menu to locate the item of interest. We encourage interface designers to consider incorporating the necessary software to support split menus. Monitoring selection frequencies, and providing a dialog box which allows users to view the menu both alphabetically and as a split menu and to choose which one to use can result in faster selections and higher user preference ratings.
We would like to thank the editor, David Kieras, and the anonymous reviewers for their thoughtful comments. Their detailed reviews led to many improvements to this paper. We appreciate Richard Potter’s efforts in creating the data collection and menu organization software used in the usability studies, and NASA’s financial support through grant #NGT-50762. We would also like to thank all of the participants in both the usability studies and the controlled experiment for participating.
Brow88 C. Marlin Brown, Human-Computer Interface Design Guidelines, Norwood, NJ: Ablex, 1988.
Card82 S. K. Card, User perceptual mechanisms in the search of computer command menus, Human Factors in Computer Systems Proceedings (1982), New York: ACM, 190-196.
CHWS88 J. Callahan, D. Hopkins, M. Weiser, & B. Shneiderman, An empirical comparison of pie versus linear menus, Proceedings of CHI ‘88 (1988), New York: ACM, 95-100.
FYM90 D. Fisher, E. Yungkurth, & S. Moss, Optimal menu hierarchy design: Syntax and semantics, Human Factors 32, 6, 665-683, 1990.
GrWi85 S. Greenberg & I. H. Witten, Adaptive personalized interfaces: A question of viability, Behaviour and Information Technology 4, 1, 1985, 31-45.
Gree88 S. Greenberg, Tool use, reuse, and organization in command-driven interfaces, Ph.D. Dissertation, Calgary, Alberta, Canada: The University of Calgary Department of Computer Science, 1988, 28-33.
LaNa85 T. K. Landauer & D. W. Nachbar, Selection from alphabetic and numeric menu trees using a touch screen: Breadth, depth, and width, Proc. Human Factors in Computing Systems (April 1985), New York: ACM SIGCHI, 73-78.
LeMa85 E. Lee & J. MacGregor, Minimizing User Search Time in Menu Retrieval Systems, Human Factors 27, 2, 157-162, 1985.
MIL91 Department of Defense, Military Standard - Human engineering design criteria for military systems, equipment and facilities, MIL-STD-1472D, March 20, 1991.
MiSh89 J. Mitchell & B. Shneiderman, Dynamic versus Static Menus: An exploratory comparison, SIGCHI Bulletin 20, 4, 1989, 33-37.
Norm91 K. Norman, The Psychology of Menu Selection. Norwood, NJ: Ablex, 1991.
OSF90 Open Software Foundation, OSF/Motif Style Guide, Englewood Cliffs, NJ: Prentice Hall, 1990.
PaRo86 K. R. Paap & R. J. Roske-Hofstrand, The optimal number of menu options per panel, Human Factors 28, 4, 377-385, 1986.
PaRo88 K. R. Paap & R. J. Roske-Hofstrand, Design of Menus, In M. Helander (Ed.), Handbook of Human-Computer Interaction (pp. 205-235). Amsterdam: Elsevier Science Publishers, 1988.
RuHe86 R. Rubinstein & H. Hersh, The Human Factor: Designing Computer Systems for People. Burlington, MA: Digital Press, 1986.
Sear93a A. Sears, Layout Appropriateness: Guiding user interface design with simple task descriptions, Ph.D. Dissertation, College Park, MD: The University of Maryland Computer Science Department, 1993.
Sear93b A. Sears, Layout Appropriateness: A metric for evaluating user interface widget layout, 1992. To appear in: IEEE Transactions on Software Engineering.
Shne92 B. Shneiderman, Designing the User Interface. Reading, Second Edition, MA: Addison-Wesley, 1992.
SmMo86 S. L. Smith & J. N. Mosier, Guidelines for designing user interface software, Report 7 MTR-10090, Bedford, MA: Mitre Corporation, 1986.
Somb87 B. L. Somberg, A comparison of rule-based and positionally constant arrangements of computer menu items, Proceedings of CHI & GI ‘87 (1987), New York: ACM, 255-260.
Sun90 Sun Microsystems, Inc., OPEN LOOK Graphical User Interface Application Style Guidelines, Reading, MA: Addison-Wesley, 1990.
Thor44 E. L. Thondike & I. Lorge, The teachers’ wood book of 30,000 words. New York: Columbia University Press, 1944.
WaSm90 N. Walker & J. Smelcer, A comparison of selection times from
walking and pull-down menus, Proceedings of CHI ‘90 (1990), New
York: ACM, 221-225.