ABSTRACT

User interface technology has advanced rapidly in recent years. Incorporating new developments in existing systems could result in substantial improvements in usability, thereby improving performance and user satisfaction, while shortening training and reducing error rates. We describe low-effort, high-payoff strategies that focus attention on improvements to data display and entry, consistency, messages, documentation, system access and additional functionality. We report on experience from six projects, describing observations, recommendations and outcomes. We close with guidance for managers and designers who are responsible for user interface reengineering.

1. INTRODUCTION

Substantial user interface research and design experience has led to an increasingly deep understanding of design principles (Hix & Hartson, 1993; Nielsen, 1993; Shneiderman, 1997). Incorporating the new developments in user interface technology could greatly enhance the usability of existing systems (Landauer, 1995).

We use the term "user interface reengineering" to describe the process of redesigning an existing interface, to improve user performance and satisfaction, while shortening learning time and reducing error rates. The challenge for developers is to understand how the old user interface was designed and conceived (Moran and Carroll, 1995), and what constraints the new user interface must respect (Merlo, Gagne, Girard, Kontogiannis, Hendren, Panangaden & Mori, 1995). Case studies of user interface redesign have shown that benefits for users or companies can be sizable (Tullis, 1981; Burns et al., 1986; Egan et al., 1989). The strategy presented here emphasizes minimal changes to the hardware and software requirements of the current system and emphasizes low-effort high-pay-off improvements.

Reengineering can be defined as the examination and alteration of a system to reconstitute it in a new form, and the subsequent implementation of the new form (Chikofsky & Cross, 1990). Process reengineering requires looking at fundamental processes of the business from a cross-functional perspective (Hammer, 1990). Any mode of reengineering would involve reengineering the business process as well, but this paper, focuses only on redesigning the user interface of existing systems. Although business processes may be affected, extensive revamping of the business process is beyond the scope of user interface reengineering.

This paper provides guidance for managers and designers responsible for user interface reengineering, based on our experience from six projects. The diagnostic strategy used in the process, the improvements we identified, and outcomes for each project are presented. The approach to each project was unique, depending on factors such as the size and complexity of the system, the number and type of users, customer requirements and the customers' commitment in terms of time and capital. We hope to encourage others to apply and refine our strategy as they improve existing systems. This seems especially important since many successful systems are somewhat dated and there is a grand opportunity to improve performance and job satisfaction.

Six projects

In each project we were approached to assist in reengineering the user interface, and initial intentions towards making changes were positive. The interventions were highly varied but we learned valuable lessons from each experience. In all cases we conducted a review using the diagnostic strategies described in section 2 and proposed short term recommendations falling within the categories described in section 3.

Some projects had clear time limits by which we had to complete our recommendations. This forced us to set our priorities and quickly assess what the client would be willing to implement. Other limitations are the financial resources, software tools, staff availability, and staff expertise. Our reports included comments about these limitations and our respect for them. We sometimes provided estimates of the effort required, in terms of the number of hours and level of expertise, to implement our recommendations. They appreciated our sensitivity to their constraints.

Our work on short-term recommendations was usually followed by more extensive long-term explorations of novel interfaces but those explorations fall outside the scope of this paper. These brief descriptions set the context for our work:

Juvenile Justice (DJJ)

In the large Maryland Department of Juvenile Justice project, we evaluated the Information System for Youth Services used by over 600 case workers. Short term recommendations included changes requiring only a low level of effort to implement and for which advantages were visible to all the users. DJJ initiated action on some of the issues we raised while others were postponed to the more complete reengineering which is currently taking place.

Micro-anatomy (NLM)

At the National Library of Medicine, we critiqued and redesigned the interface of the MicroAnatomy Visual Library System, an interactive computer system that allows library patrons or students to view videodisk images of human cell structures (Chimera & Shneiderman, 1993).

Network management (HNS)

For Hughes Network Systems, we evaluated a complex satellite network configuration system based on numerous overlapping forms. This was a complex system that we could not entirely master within the scope of our project but several problems were identified and our short terms recommendations have been found useful.

Library card catalog (LC)

We worked with the Library of Congress staff to improve access to the library online catalog for first-time users and eliminated training classes (Marchionini, Ashley & Korzendorfer, 1993). The existing command driven on-line interface to the catalog, SCORPIO, was given a colorful touchscreen interface called ACCESS, designed to serve first time users. ACCESS has reduced the workload of the reference staff at the help-desk, allowing them to help advanced users with complex searches.

Telepathology (Corabi)

Our analysis of a remotely-controlled microscope developed by a small firm, Corabi Telemetrics, for pathologists, identified key issues like time delays, incomplete feedback and interference (Carr, Hasegawa, Lemmon & Plaisant, 1992).

Home Automation (CCS)

At Custom Command Systems, we evaluated a home automation system for security, lighting, entertainment, and climate control (Plaisant, Shneiderman and Battaglia, 1990). Home owners used touchscreens mounted in the walls or cabinetry to control all the equipment in their house. The redesigned interface was successful used as a front end for several home automation systems.

2. DIAGNOSTIC STRATEGIES

Our diagnostic strategies can be used by designers to learn about and evaluate most user interfaces. The objective is to understand the functioning of the system, and to identify the key areas where substantial improvements can be achieved by minor reengineering of the interface.

Table 1 shows that three strategies were used for all projects: documentation study, observe users and expert review. We took the formal training whenever possible and studied the documentation as a way to understand the background of users. The other techniques consist of discussions that were conducted depending on the availability of managers, designers or users. Printed surveys or questionnaires complemented the discussions or interviews.

Documentation: Compile, and go through all available documentation.

One of the easier ways to learn about any system is to peruse the available documentation: system specifications, design documents, user manuals, training videos, on-line help, etc. An organizationís annual report, or such other material which defines the goals and mission helps understand the organizationís objectives and future direction. The documentation is useful as a reference during the interface reengineering process, and itself may become the object of redesign. All projects had some sort of documentation available.

Formal training: Undergo training and attend demonstrations.

Another diagnostic strategy is to attend the formal training programs and demonstrations, with the users. In addition to having the same type of introduction to the system as the users, valuable insights can be gained about the training process itself. Also, some of the deficiencies in the interface design become apparent at this stage. At DJJ the youth information system was presented to groups of about 50 new users in one-hour training sessions; and as warnings and tips were given to users we learned the major problems of the interface. At HNS the one week-long training was summarized for us into a one-day session which taught enough be able to observe users and follow their work.

Discuss with managers: Identify goals, commitment and resources.

Creating rapport with the top management and gaining their support is essential for the successful implementation of the new design. Discussions with management can help identify the goals of the organization, their commitment to the redesign process, the metrics they use to measure success, the resources available, and the time frame for implementing the reengineered design. In addition, their requirements for executive summary reports and other statistical information can be identified at this stage. These discussions are also useful for laying out and prioritizing the benefits, like improved productivity and user satisfaction, that could be achieved by redesigning the interface. At LC the main priority was to relieve librarians of having to help patrons coming with simple queries. At DJJ, the priority was to improve accuracy. Knowing this allowed us to focus attention on the most effective short-term improvements that satisfied the expressed goal.

Discuss with designers: Identify resources and constraints.

Discussions with the design and maintenance staff helps identify their goals, the system constraints, and the alternatives available. In some cases we were not able to contact the designers of the original system (e. g . the contractors were no longer available to discuss the existing design) but we interviewed the team that would potentially implement the recommendations. Building a working relationship with the technical staff is essential for the successful implementation of the new interface. If the reengineering work is being done in-house, these discussions can aid in identifying the strengths and weaknesses of the design staff, and will influence the rating of the effort level required for each recommendation. Short term improvements that require unavailable funds or staff are not appropriate.

Discuss with users: Learn about their frustrations and expectations.

The methods and amount of time spent varies greatly across projects. At DJJ, case workers used the system in many different ways and we had to spend a long time with users to understand the different practices. At HNS or Corabi the number of users was limited and their time precious, so only a few users were interviewed. This can be compensated by spending more time observing users doing their work (see next section). Discussions with users have to be carefully planned. During our informal discussions with DJJ users, identifying ourselves and gaining their confidence was of utmost importance (Rose, Shneiderman & Plaisant, 1995). Learning the work culture, and adapting to it, goes a long way in winning support. Soliciting comments on overall system performance, and asking open-ended questions, will help in identifying problems users face in daily use of the system. Some of these problems might have simple solutions. The spontaneous first response from users can be extremely useful to spot needed short term improvements. For example we heard from users of the Corabi telepathology system users "Itís great but I donít use it much because itís too slow", pointing us immediatly to an important problem of the system.

Observe users: Watch users perform their routine tasks.

In addition to discussions with users, and especially when users cannot be interviewed easily, observing them using the system gives feedback on the manner in which experts and novices react to different system responses. It helps identify specific bottlenecks that might be overlooked. Other information like the condition of the hardware, the physical work environment, system response time and so on, can be gathered in this process. At DJJ we observed the effect of inoperational equipment and identified the most common handwritten "cheat-notes" used to remember cryptic codes. At HNS we observed how difficult it was to manage numerous similar-looking overlapping windows while standing up and talking on the phone, or how often some of the error messages appeared (like: too many windows opened). Those problems surfaced very rapidly but would have been ignored without direct user observations. In some cases we conducted usability tests: for the two "zero-training" systems (NLM micro-anatomy and CCS home automation) users were brought to the lab and given representative tasks to perform.

Expert reviews: Use the system to gain first-hand experience.

Expert reviews can be obtained by hands-on experience with the system and are invaluable for gaining an in-depth knowledge on the process flow and system procedures. Where possible, the review team should have access to the actual system (but in the case of HNS' network management system a training system had to be used.) This diagnostic strategy was used for all projects and generated the largest number of suggestions for short-term improvements, in particular for consistency and screen layout..

Questionnaire: Administer questionnaires to get user feedback.

Another valuable tool we used for evaluating existing user interfaces was QUIS, the Questionnaire for User Interaction Satisfaction (Chin, Diehl & Norman, 1988). It was developed by the Human-Computer Interaction Laboratory at University of Maryland, and is used widely in industry and academia, for evaluating 71 interface features. It can be customized to suit the specific system being evaluated. QUIS uses 1-to-9 scales, and is useful in identifying the major problem areas, as perceived by the user. Major advantages of a questionnaire, over personal interviews, are that it can be administered to a larger population, and it maintains anonymity. The system can be compared to industry standards, or similar systems that are currently in use. QUIS also provides for open-ended questions, where users can express their comments and suggestions. Questionnaires can be re-administered after the reengineered interface has been installed, in order to measure the relative gains of the new interface. This was the case for the micro-anatomy project: QUIS was administered before and after the changes were made, and the results showed that the revised version generated higher ratings that were statistically significant for 19 features. A separate test of performance speed confirmed these benefits.

3. OPPORTUNITIES FOR IMPROVEMENT

For short term recommendations to be effective the reengineering team has to understand exactly how the organization functions, and should have a good working relationship at all levels of the organization. The goal is that when the recommendations are presented, they will be well received and serve as a basis for action. We repeatedly stressed that we would only make recommendations and that the responsibility for action was with management and their staff. This enabled them, managers and technical staff, to maintain a sense of ownership of the project, and reduce fear that we were taking away some of their control.

We believe that the success rate for our intervention was higher if the new design was implemented in phases. Our reports were usually separated into short-term and long-term recommendations for items such as data display, so that our clients could decide how far to go in implementing them. For example, counters can be used to identify the most frequently accessed screens, and the layouts of these screens can be redesigned first. The users would be able to see the benefits of the new design, thereby encouraging them to contribute even more to the success of the project. Phased implementation also helps the design team set a schedule of specific targets for each team member, and to meet the deadlines for the deliverables.

The six projects showed a clear pattern in the problems we identified, and the resulting low-effort high-payoff opportunities for improvement. The identified areas were documentation, system access, data display, data entry, consistency, system/error messages, and additional functions (Table 2).

Documentation: Three systems did not have proper system documents. Providing a comprehensive userís manual and a quick reference guide help the users to perform their functions in a more competent manner. The first step is to provide better documentation for the parts of the system that have the most problems. Having sections of the documentation on-line, like help, a mini tutorial and frequently asked questions make it easier to complete tasks. This was found applicable in all non-public-access projects. For example, DJJ users could not easily remember the cryptic codes for offenses: discussions with managers and designers had eliminated a short term move to graphical menus so we recommended the addition of a rudimentary online help for the offense codes.

System Access: We regularly found that system access could be improved by bringing distant equipment closer to the work site, opening frequently locked work rooms, repairing damaged equipment, increasing system speed and reliability, and also simplifying access procedures. At DJJ we recommended reduction in the number of steps to login the system. The revised interface now has five steps instead of ten, and one password instead of two. Better warnings are also issued when the password is about to expire, making access to the system much easier. This change was highly appreciated by all users.

Data Display: The screen layout could be improved in all the projects using color, sorting and grouping of fields, and improvements to the type and amount of highlighting. For example screens that used only upper case characters were made more readable by using mixed characters and bolding of important data, a simple but very effective improvement. Many of the screens we analyzed contained obsolete information and obscure codes that were not useful. At the same time, new elements had been added to existing screens to meet changing needs. This contributed to screen clutter, and made information retrieval difficult. We chose to limit recommended changes to a few critical screens.

Data Entry: Improving data entry procedures can reduce the number of errors and speed performance, thereby improving user productivity. We looked for instances where the same data was being entered in different locations, resulting in duplication of work. One way to eliminate redundant data entry is by displaying default information whenever possible. Limiting cursor movement to only editable fields, using a high-precision strategy for touch screen selection, making the cursor more visible, or having consistent key sequences can speed up data entry and reduce errors.

Consistency: Consistency refers to common action sequences, terms, units, layouts, abbreviations, spelling, capitalization, color and so on within an application program; it is naturally extended to include compatibility across application programs and compatibility with paper or non-computer-based systems (Shneiderman, 1992). Making the interface more consistent results in faster learning, higher performance speed, lower error rate and better retention over time system (Mahajan & Shneiderman, 1996). In all six projects, we suggested improvements in the consistency of the terminology used, the sequence of operations, the screen headers, the field labels and the screen colors. Even the most developed interfaces we reviewed had large numbers of obvious inconsistencies, for example the HNS network management interface we reviewed used up to four different terms for the same object in the same window.

Error/System Messages: Improving error and system messages is one of the easiest and most efficient ways to improve an existing system (Shneiderman, 1992). More information can be conveyed by making messages more specific and by providing constructive guidance to the user. Using a positive tone, and a user-centered style for the messages makes the user more comfortable with the system, especially in difficult situations. Displaying information that is not relevant, like the error code, only adds to screen clutter. Using a consistent format, terminology, color, abbreviation and placement, increases acceptance of the system. User satisfaction is further enhanced by providing feedback for every user actions to indicate changes and status.

Additional Functionality: During our discussions with the users and using the feedback from the questionnaire, we were often able to identify additional functions that could easily be integrated, and would greatly enhance user performance and satisfaction. The management and maintenance staff were consulted, and modified versions of these features were incorporated in the revised design. Often, graphical representation and information visualization techniques can be used to present more information in the same screen area, in a clearer manner. This helps in analyzing the information and makes information retrieval much easier. In the Corabi telepathology system, we recommended the addition of an overview of the slide which provided context for the zoomed view and greatly facilitated navigation. In the home automation system we identified the scheduling of devices as the most challenging component of the interface and suggested a direct manipulation interface using a timeline and flags. For the online catalog, additional features were suggested to cross-reference subject headings, in order to simplify navigation.

Summary

In table 3 we show the strategies we found more effective to:

- generate recommendations in the 7 areas of improvements and

- prioritize the recommendations according to payoff and cost.

We also map the sections of the QUIS onto the improvement areas. This allows practitioners to select or prioritize strategies to use.

Good communications among the reengineering team, the designers of the existing system, managers, and users are essential. A clear schedule and explicit statement about the level of effort for the review and re-implementation is vital to ensure common understanding of the level and duration of effort, plus the time for expected payoffs to appear.

Our reengineering review process took several days to several weeks, and the outcome was a written report, usually ranking recommendations by anticipated level of effort and payoff. The recommendations were specific: proposed rewording of messages, new layouts for screens, or mockups of the revised navigation. We encourage others to be specific in their recommendations.

5. CONCLUSION

While there are increasingly sophisticated design methodologies for development of new user interfaces (Hix and Hartson, 1993; Nielsen, 1993), user interface reengineering is a relatively new direction. Reengineering a working system is complex because of the potential disruption for users and managers, the justifiable fear of change, and the lack of guarantees that changes would be for the better.

Our largely positive experiences encourage us to believe that user interface reengineering is a viable and important process. Low-effort and high-payoff recommendations for improvements can probably be made for most existing systems.

The proposed diagnostic strategies and opportunities for improvement do not deal with the much larger task of business reengineering, but they provide practical approaches for short-term action. As more systems are candidates for reengineering, greater attention to reengineering processes could have important impact for human-computer interaction practitioners. Reengineering interfaces for existing systems would open up new areas for research, leading to more innovative designs. Controlled studies of reengineering processes would help confirm the benefits. Practitioner reports are useful in assessing and refining these strategies in commercial practice.

ACKNOWLEDGMENTS

Our sincere thanks to HCIL members who were involved directly or indirectly in the projects listed here, and contributed to their successful completion. We also thank users who provided valuable feedback during the reengineering process, and the subjects in the usability testing of our prototypes. The preparation of this document was supported in part by the Maryland Department of Juvenile Services.

REFERENCES

Burns, Michael J., Warren, Dianne L., and Rudisill, Marianne, "Formatting space-related displays to optimize expert and nonexpert user performance", Proc. ACM SIGCHI '86 Human Factors in Computing Systems, ACM, New York (1986), 274-280.

Carr, D., Hasegawa, H., Lemmon, D. and Plaisant, C., "The effects of time delays on a telepathology workstation," Proc. of the 16th Symposium on Computer applications in Medical Care, Baltimore, MD (1992), 256-260.

Chimera, R. and Shneiderman, B., "User interface consistency: An evaluation of original and revised interfaces for a videodisk library," in Shneiderman, B. (Editor), Sparks of Innovation in Human-Computer Interaction, Ablex Publishers, Norwood, NJ (1993), 259-271.

Chikofsky, E. J. and Cross II, J. H., "Reverse Engineering and Design Recovery: A Taxonomy," IEEE Software 9, 1 (January 1990), 13-17.

Chin, J. P., Diehl, V. A. and Norman, K. L., "Development of an Instrument Measuring User Satisfaction of the Human-Computer Interface," Proceedings of CHI '88: Human Factors in Computing Systems, ACM, New York, NY (1988), 213-218.

Egan, Dennis E., Remde, Joel R., Gomez, Louis M., Landauer, Thomas K., Eberhardt, Jennifer, and Lochbum, Carol C., "Formative design-evaluation of SuperBook," ACM Transactions on Information Systems 7, 1 (January 1989), 30-57.

Hammer, M., "Reengineering Work: Donít Automate, Obliterate," Harvard Business Review (July-August 1990), 104-112.

Hix, D. and Hartson, H., Developing User Interfaces: Ensuring Usability through Product & Process, John Wiley & Sons, New York, NY (1993).

Landauer, Thomas K., The Trouble with Computers: Usefulness, Usability, and Productivity, MIT Press, Cambridge, MA (1995).

Mahajan, R. and Shneiderman, B., "Visual & textual consistency checking tools for graphical user interfaces", University of Maryland Technical Report CS-TR-3639 (1996), submitted for publication.

Marchionini, G., Ashley, M. and Korzendorfer, L., "ACCESS at the Library of Congress," in Shneiderman, B. (Editor), Sparks of Innovation in Human-Computer Interaction, Ablex Publishers, Norwood, NJ (1993), 251-258.

Merlo, E., Gagne, P., Girard, J., Kontogiannis, K., Hendren, L., Panangaden, P. and Mori, R., "Reengineering User Interfaces," IEEE Software 12, 1 (January 1995), 64-73.

Moran, T. and Carroll, J. (editors), Design Rationale: Concepts, Techniques, and Use, Lawrence Erlbaum Assoc (1995).

Nielsen, J., Usability Engineering, Academic Press, New York, (1993).

Plaisant, C., Shneiderman, B. and Battaglia, J., "Scheduling home-control devices: A case study of the transition from the research project to a product," Human Factors in Practice, Human Factors Society Computer Systems Technical Group (December 1990), 7-12. Reprinted in Shneiderman, B. (Editor), Sparks of Innovation in Human-Computer Interaction, Ablex Publishers (1993), 205-215.

Rose, A., Plaisant, C., and Shneiderman, B., "Using ethnographic methods in user interface re-engineering", Proc. DISí95 : Symposium on Designing Interactive Systems, ACM Press, New York, NY (August 1995), 115-122.

Shneiderman, B., Designing the User Interface: Strategies for Effective Human-Computer Interaction (3nd Edition), Addison Wesley, Reading, MA (1997).

Tullis, T. S., "An evaluation of alphanumeric, graphic and color information displays", Human Factors 23 (1981), 541-550.