Visualization Viewpoints

Editor: Theresa-Marie Rhyne

Why Not Make Interfaces Better than 3D Reality? ______

Ben Shneiderman

University of Maryland

ome designers dream about building interfaces that approach the richness of 3D reality. They believe that the closer an interface resembles the real world, the easier the usage. They strive for resolution that matches film—with rapid camera movement and lively animated objects. This is a dubious proposition since user studies show that disorienting navigation, complex user actions, and annoying occlusions can slow performance in the real world as well as 3D interfaces.1-3

Many constrained interfaces are designed to be simpler than the real world by restricting movement, limiting interface actions, and keeping interface objects in a plane. However, the strong utility of pure 3D interfaces for medical, architectural, product design, and scientific visualization means that interface design for pure 3D remains an important challenge.

An intriguing possibility is that enhanced 3D interfaces might offer simpler navigation, more compelling functionality, safer movements, and less occlusion, than 3D reality, especially for information exploration and visualization tasks. Such features can enable superhuman capabilities such as faster-than-light teleportation, flying through objects, and x-ray vision. Enhanced 3D interfaces might have super-natural tools such as magic wands for instantly shrinking, enlarging, duplicating, or sending objects and enchanted environments that provide error prevention, history keeping, and programming-by-demonstration.

Playful game designers and creative application developers have already pushed the technology further than those who seek merely to mimic reality. Advanced designs are marked by their support of

This article reviews these clever enhanced 3D-design features and encourages approaches that facilitate user tasks rather than mimic reality.

Enhanced 3D interfaces

For some computer-based tasks, pure 3D representations are clearly helpful and have become major industries: medical imagery, architectural drawing, computer-assisted design, and scientific simulations.

Designers of these successful applications start with the goals of meeting user needs for high resolution and maintaining faithfulness to reality. However, in these cases, the success is often due to design features that make the interface even better than reality. Users can change colors or shapes, group and ungroup components, send objects by email, and attach floating labels. Users can also carry out enhanced actions such as going back in time by undoing recent actions or playing animations in reverse. They can also collaborate with distant partners and use dynamic query sliders to support exploration.

Among the many innovations, there have been questionable 3D prototypes such as air-traffic control systems that show altitude by perspective drawing, thereby only adding clutter to the plan view display.4 Other dubious 3D prototypes include digital libraries (showing books on shelves might be nice for browsing, but can inhibit searching and linking) and file directories (showing files as towers and trees as node-link diagrams can increase occlusion and navigation problems). Further questionable applications include ill-considered 3D features for situations in which simple 2D representations would do a better job. For example, adding a third dimension to bar charts might slow users and mislead them,5,6 but some users find these so attractive that designers include them in most business graphics packages.

The controversy over 3D versus 2D interfaces is especially lively in information visualization circles. For scientific visualization, 3D is necessary because typical user tasks involve continuous variables (for example, temperature, density, pressure, velocity) and volumes, surfaces, inside and outside, left and right, and above and below. However, for information visualization, typical user tasks involve more categorical variables and the discovery of patterns, trends, clusters, outliers, and gaps. Users who analyze gene micro arrays, stock markets, or manufacturing quality control might work with hundreds of variables. Their preferred strategy is to explore relationships a few at a time through dynamic queries over coordinated 2D scattergrams with color and size coding, and occasionally using 3D (for example, Spotfire, DataDesk, SPSS/SigmaPlot, or SAS/GRAPH). To move past 3D, some users appreciate parallel coordinates, which map points in a multidimensional space into polylines across a 2D display.

The lessons from information visualization are that

12 November/December 2003 Published by the IEEE Computer Society 0272-1716/03/$17.00 © 2003 IEEE

  • interface controls are as important as the graphics display,
  • metrics help guide design, and
  • usability testing is essential.

Since navigation complexity is a key determinant of user success, it might be more important to fight for two versus three clicks than to debate 2D versus 3D.

Another source of lessons are the intriguing, successful applications of 3D representations in game environments. These include first-person shooter games, such as Doom and Quake, in which users patrol city streets or race down castle corridors while shooting at opponents. Other successes are role-playing fantasy games with beautifully illustrated island havens or mountain strongholds—for example Myst, RealMyst, or Riven. Many games are socially enriched, letting users choose 3D avatars to represent themselves. Users can choose avatars that resemble themselves, but often the theatrical nature of these environments encourages them to pick bizarre characters or fantasy personas with desirable characteristics such as unusual strength or beauty.

Some Web-based 3D environments—such as ActiveWorlds (http://www.activeworlds.com)—involve millions of users and thousands of user-constructed worlds such as Yellowstone National Park, shopping malls, or urban neighborhoods (see Figure 1). Game devotees might spend dozens of hours per week immersed in their virtual worlds, chatting with collaborators or negotiating with opponents. Sony’s Everquest attracts users with this ambitious description: “Welcome to the world of EverQuest, a real 3D massively multi-player fantasy role-playing game. Prepare to enter an enormous virtual environment—an entire world with its own diverse species, economic systems, alliances, and politics.” Game consoles, such as Microsoft Xbox and Sony PlayStation 2, offer users the chance to play livelier 3D games—including NASCAR Thunder, Halo, or War of the Monsters.

Sims Online has 3D characters who live in a limited 3D home environment with social behaviors that users control; meanwhile There.com (http://www.there.com) emphasizes real people meeting to discuss substantive issues (see Figure 2). These environments might prove successful because of their increasingly rich social contexts based on spatial cognition. That is, users might come to appreciate the importance of the setting, and value participants who choose to stand close to them. Such environments might come to support effective business meetings (as promoters of Adobe’s Atmosphere, There.com, and Blaxxun envision), community discussion groups, and even contentious political forums. Atmosphere promoters invite users to “Imagine walking down the aisles of a virtual store and inspecting merchandise before you purchase it. Or imagine taking a virtual tour of the Great Pyramid of Giza, where you explore its internal corridors and view details down to the chisel marks on the stone.”

Three-dimensional art and entertainment experiences, often delivered by Web applications, provide another opportunity for innovative applications. Early 3D front ends for Windows offer rooms for shopping, games, Internet, and office applications and will likely remain attractive to games, entertainment, and sports enthusiasts. Another recent product is Browse3D’s browser that provides a limited 3D Web browsing experience based on perspective. This product’s main benefit is appropriate screen management of up to 16 Web pages. A skeptic might wonder if a 2D version would produce faster performance and better use of screen space.

A modest use of 3D techniques is to add highlights to 2D interfaces, such as buttons that appear raised or depressed, windows that overlap and leave shadows, or

3 Task Gallery from Microsoft enables users to perform Windows operations, although it restricts movement to in/out. Applications are on the sides, floor, and ceiling. (Courtesy of Microsoft Corporation.)

4 Win3D from Clockwise3d enables users to perform Windows operations, provides multiple rooms with planar movement, and permits teleportation among rooms. (Courtesy of ClockWise Technologies.)

November/December 2003

icons that resemble real-world objects. Users might find these interfaces enjoyable, recognizable, and memorable because they improve spatial memory use,8but they can also distract and confuse because of increased visual complexity. Users have not been enthusiastic about 3D malls and product demos, but they appreciate real estate walkthroughs. Attempts to build realistic devices—such as telephones, books, or CD players—produce pleasant smiles from first-time users, but these designs have not caught on, probably because the compromises needed to produce 3D effects undermine usability.

Guidelines for 3D designers

Discussion of 3D design guidelines could lead to improved recommendations for interfaces that require pure 3D or for users who might benefit from constrained or enhanced 3D experiences. Of course, these guidelines are just a starting point and they need refinement and testing before designers can ensure that they improve the user experience. For the moment, this enumeration of features for effective 3D interfaces might serve as a checklist for designers, researchers, and educators:

  • use occlusion, shadows, perspective, and other 3D techniques carefully;
  • minimize the number of navigation steps for users to accomplish their tasks;
  • keep text readable (better rendering, good contrast with background, and no more than 30-degree tilt),
  • avoid unnecessary visual clutter, distractions, contrast-shifts, and reflections;
  • simplify user movement (keep movements planar, avoid surprises such as going through walls),
  • prevent errors (surgical tools that cut only where needed, chemistry kits that produce only realistic molecules and safe compounds);
    • simplify object movement (facilitate docking, follow predictable paths, limit rotation);
    • organize groups of items in aligned structures to allow rapid visual search; and
    • enable users to construct visual groups to support spatial recall (placing items in corners or tinted areas).
    • Breakthroughs based on clever ideas seem possible. Enriching interfaces with stereo displays,9 haptic feedback, and 3D sound might yet prove beneficial in applications other than specialized ones. Bigger payoffs are more likely to come sooner by following guidelines for inclusion of enhanced 3D features:
  • provide overviews so users can see the big picture (plan view display, aggregated views);10
    • allow teleportation (rapid context shifts by selecting destination in an overview);
    • offer x-ray vision so users can see into or beyond objects;
  • provide history keeping (recording, undoing, replaying, editing);
  • permit rich user actions on objects (save, copy, annotate, share, send);
  • enable remote collaboration (synchronous, asynchronous);
    • give users control over explanatory text (popup, floating, or excentric labels and screen tips) and let users select for details on demand;
    • offer tools to select, mark, and measure;
  • implement dynamic queries to rapidly filter out unneeded items;
  • support semantic zooming and movement (simple action brings object front and center);
  • enable landmarks to show themselves even at a distance;11
  • allow multiple coordinated views (users can be in more than one place at a time and see data in more than one representation at a time); and
  • develop novel 3D icons to represent concepts that are more recognizable and memorable.12

Since enhanced 3D interfaces are such a rich topic, it seems likely that readers could easily add further guidelines.

Conclusions

Three-dimensional environments are greatly appreciated by some users and are helpful for some tasks. They have the potential for novel social, scientific, and commercial applications if designers go beyond the goal of mimicking 3D reality.

Enhanced 3D interfaces could be the key to finally making some kinds of 3D teleconferencing, collaboration, and teleoperation popular. Of course, this will require a good 3D interface design (pure, constrained, or enhanced) and more research on finding the payoffs beyond the entertaining features that appeal to first-time users.

Success will come to designers who provide compelling content, relevant features, appropriate entertainment, and novel social structures. Then by studying user performance and measuring satisfaction, they can polish their designs and refine guidelines for others to follow.

Acknowledgments

This essay was stimulated by the invitation from Sandy Ressler to keynote the February 2002 Web3D conference. Ben Bederson, Andy Cockburn, Mary Czerwinski, Eric Dementhon, Jean-Daniel Fekete, Theresa-Marie Rhyne, George Robertson, Alistair Sutcliffe, Colin Ware, and anonymous reviewers provided thoughtful comments. A revised version of this essay will be included in the 4th edition of Designing the User Interface (Addison Wesley), to appear in April 2004.

References

  1. A. Sutcliffe and U. Patel, “3D or Not 3D: Is It Nobler in the Mind?” People and Computers: Proc. 1996 British Human–Computer Interaction Conf., Cambridge Univ. Press, 1996, pp. 79-94.
  2. K. Risden et al., “An Initial Examination of Ease of Use for 2D and 3D Information Visualizations of Web Content,” Int’l J. Human–Computer Studies, vol. 53, no. 5, Nov. 2000, pp. 695-714.
  3. A. Cockburn and B. McKenzie, “Evaluating the Effectiveness of Spatial Memory in 2D and 3D Physical and Virtual Environments,” Proc. ACM Computer–Human Interaction Conf. Human Factors in Computing Systems, ACM Press, 2002, pp. 203-210.
  4. H. Smallman et al., “Information Availability in 2D and 3D Displays,” IEEE Computer Graphics and Applications, vol. 21, no. 5, Sept-Oct. 2001, pp. 51-57.
  5. E. Levy et al., “Gratuitous Graphics? Putting Preferences in Perspective: Empirical Studies of Graphics and Visual Design, Proc. Computer–Human Interaction Conf. Human Factors in Computing Systems, ACM Press, 1996, pp. 42-49.
  6. M. Hicks et al., “Comparison of 2D and 3D Representations for Visualising Telecommunication Usage,” Behaviour & Information Technology, vol. 22, no. 3, 2003, pp. 185-201.
  7. S. Card, J. Mackinlay, and B. Shneiderman, Readings in Information Visualization: Using Vision to Think, Morgan Kaufmann, 1999.
  8. W. Ark et al., “Representation Matters: The Effect of 3D Objects and a Spatial Metaphor in a Graphical User Interface,” Proc. Human–Computer Interaction Conf. People and Computers XIII, Springer-Verlag, 1998, pp. 209-219.
  9. C. Ware and G. Franck, “Evaluating Stereo and Motion Cues for Visualizing Information Nets in Three Dimensions, ACM Trans. Graphics, vol. 15, no. 2, 1996, pp. 121-139.
  10. U. Wiss, D. Carr, and H. Jonsson, “Evaluating Three-Dimensional Information Visualization Designs: A Case Study of Three Designs,” Proc. Int’l Conf. Information Visualization, IEEE Press, 1998, pp. 137-144.
  11. R.P. Darken and J.L. Sibert, “Navigating Large Virtual Spaces,” Int’l J. Human–Computer Interaction, vol. 8, no. 1, 1996, pp. 49-71.
  12. P. Irani and C. Ware, “Diagramming Information Structures Using 3D Perceptual Primitives,” ACM Trans. Com-puter–Human Interaction, vol. 10, no. 1, ACM Press, 2003, pp. 1-19.

Readers may contact Ben Shneiderman at Univ. of Maryland, Dept. of Computer Science, College Park, MD 20742; ben@cs.umd.edu.

Readers may contact Theresa-Marie Rhyne by email at tmrhyne@ncsu.edu.

IEEE Computer Graphics and Applications