FUTURE DIRECTIONS FOR HUMAN-COMPUTER INTERACTION
 
 

BEN SHNEIDERMAN

Department of Computer Science, University of Maryland, College Park, MD 20742, USA
 
 

ABSTRACT

This paper offers a set of goals for user interface development and then scenarios of future developments. The applications include home control, hypermedia, office automation, digital photography, collaborative meeting/classrooms, public access, professional workstations, and medical record-keeping. Also, predictions are made for some of the underlying technologies such as User Interface Management Systems, remote control, flexible search, display devices, and touchscreens.
 
 

INTRODUCTION

It is dangerous, but necessary, to dream about the future. Dangerous because misguided dreams mislead designers, necessary because without vision navigation is difficult. Without dreams we risk stagnation, and lose the chance to make a better world. This biased and partial portrait of the future is offered to guide designers of future interactive systems. It is organized top-down, from goals to applications to software/hardware. Planning for the future is never easy, but when innovation is the propelling force, prediction is especially difficult. This portrait is mainly an extrapolation of current trends, shaped by the high-level goals, and colored with a bit of wishful thinking.

GOALS

The goals of interactive system designers are varied: productivity increases, reduced error rates, easier learning, and more consistency in performance are often cited and are relatively easily measured. But larger goals are often implicit in the broader work environment:

increased production of high quality goods and services at low cost,

improved user or customer satisfaction,

increased safety or health,

improved communication or cooperation among people,

better educational tools,

reduction in disease or famine, and

even global goals such as world peace.

These work-related and societal goals might be expanded to include personal goals for users:

increased sense of self-worth,

empowerment to deal with large institutions,

means to communicate facts, ideas, or feelings,

capacity to innovate,

reduced stress,

opportunity to explore, and

relaxation through entertainment.

Designers may debate the relative importance of these goals, and explicit discussion in public forums is strongly encouraged. As a professional community, I believe that we will be more productive and more appreciated if we engage in such discussion. However, the longer debate about underlying values is not the focus of this paper; specific applications and directions are.

APPLICATIONS

In the past decade improved user interfaces have opened up the door to the widespread use of word processors for writing, spreadsheet software in accounting, computer assisted design for engineering, and desktop publishing plus graphics in organizational communications (Shneiderman, 1987). Here are some predictions about which applications will be similarly expanded in the next decade. The first group of applications have personal impact such as home control, public access information resources, and medical record keeping. The second group contains home and work related applications such as hypertext and digital photography. The last group of applications focus on work related aspects of electronic mail, collaborative meetings, and professional workstations.

Home controls & household automation

Internationally, many companies have logically concluded that the next big market will be the inclusion of richer controls in homes (Time Magazine, January, 1989). Simple ideas such as turning off all the lights with a single button or remote control of devices (either from one part of the home to another, from outside, or by programmed delays) are being extended in elaborate systems that channel sound and audio throughout the house, schedule lawn watering as a function of ground moisture, offer video surveillance and burgular alarms, and provide multiple-zone environmental controls plus detailed maintenance records. Demonstrations such as the Smart House project and installations such as those by Custom Command Systems are a testing ground for the next generation.

Some futurists and marketing types promote voice controls and home robots, but the practical reality is more tied to traditional pushbuttons, remote controllers, telephone keypads, and especially touchscreens, with the latter proving to be the most popular. Providing users with rich feedback and a clear sense of control is vital in these and most other applications.

In our own studies (Plaisant & Shneiderman, 1989) we explored four touchscreen designs for scheduling operations such as VCR recording or light switching:

1) digital clock that is set by pressing step keys (similar to "onscreen programming" in

current video cassette players),

2) a 24-hour circular clock whose hands could be dragged with the fingers,

3) a 12-hour circular clock (plus AM/PM toggle) whose hands could be dragged with the fingers, and

4) a 24-hour time line where ON/OFF flags could be placed to indicate start/stop times.




Figure 1:This scheduler was most successful in our usability studies. The users select a date by pointing on the calendar and then drag ON and OFF flags to the 24-hour time lines. The feedback is a red line on the claendar and the time lines. (Copyright 1988 University of Maryland)
 

Our results indicate that (4) a 24-hour time line was easiest to understand and use (Figure 1). We continue to make further trials with more complex tasks such as editing schedules, repeated events (lights on every Friday night at 7PM), and long duration events. Many interesting product

suggestions have emerged during our trials, for example an alarm clock that would ring only on weekdays, thus avoiding the oversight that leads to a ruined Saturday morning when the alarm rings at 6 AM.

Controlling complex home equipment from a touchscreen reshapes how we think of homes and their residents. New questions arise, such as whether residents will feel safer, be happier, save money, or experience more relaxation. Are there new notations such as petri net variants or role/task diagrams for describing home automation and the social relations among residents? The benefits to handicapped users and the aged were often on our minds as we designed these systems, since they may be substantial beneficiaries of this technology even though initial implementations are for the healthy and wealthy.

Public access information resources

From the early days of computing, there have been innovators who have put computers in public access situations such as airports, hotels, libraries, banks, museum exhibits, or stores. Many of the early design were difficult to use, slow, poorly organized (too many or confusing menus), hard to read (small displays, poor fonts, garrish colors), error prone (imprecise touchscreens), and unrealiable. Much has changed and there is the possibility of a new generation of public access applications, if users can be convinced to forget the bad impressions that have already been left.

Automatic teller machines are remarkably more successful than they were a decade ago because of thorough attention to the user interface by some leading banks. Improved designs have been proven to lead to increased usage and greater customer satisfaction. A similar movement is in progress with respect to online public access card catalogs at libraries where the awkward first generation of designs are being challenged by designers who believe they can do better. Museum and exhibit designers are developing attractive computer and interactive videodisc projects that have the potential to revolutionize museum going by making the vast knowledge and resources of curators more accessible to patrons (Shneiderman, Brethauer, Plaisant and Potter, 1989).

Commercial projects such as interactive sales information on sports equipment, tourist destinations, vacation resorts, real estate, shoes, home redecorating, and clothing are now succeeding more regularly because the user interface designers have been able to build on a growing body of knowledge and experience. We will undoubtedly see many more such projects.

Medical records

It is disturbing that technology has progressed rapidly in many areas, but that medical record keeping is quite similar to what it was a hundred or more years ago. While I am a great devotee of paper and pencil approaches, there are substantial benefits to having records in machine readable form. First let me make the case for machine readable medical records and then suggest some of the mechanisms for creating and manipulating the information.

If all my medical history were kept in a standardized electronic database form, I would be able to more easily transfer my records to a new physician or to a specialist during a consultation. My wife recently needed surgery and had difficulty getting her current records delivered on time for each consultation, and never could succeed in getting vital records about two previous surgeries. While computers are no guarantee that 20 year old records would be available, there does seem to be a better chance of success. Certainly, if physicians could see records of recent lab tests, examinations, or consultations, there is a chance that better medical advice could be given.

But now, let's assume a standard electronic medical history, and explore the benefits and dangers. Having good records of weight or blood pressure would enable monitoring of abnormalities or sudden changes. Within a community, a doctor might be able to spot a pattern of influenza or food poisoning that might be otherwise undetected. Early warning approaches might alert neighboring communties of potential problems so that vaccination or health emergencies might be called for. On a societal level, researchers could analyze data from large numbers of people to study the effects of smoking, obesity, exercise, or diet on a scale that was not imaginable till now. The pioneering Framingham, MA study would be dwarfed by longitudinal studies of millions of individuals rather than thousands. Of course, there are dangers of loss of privacy and violation of the doctor/patient relationship, but these have been dealt with effectively in the banking, credit, and telephone industries and there is every reason to believe that adequate protection is possible. In fact, considering the lax security in many doctors offices that I have visited, I might prefer electronic records to increase privacy protection.

Once the idea of electronic medical histories becomes acceptable, then other possibilities emerge. Each citizen might carry a magnetic card with a brief version of their medical history and links to their full record. In an emergency, physicians would immediately have the relevant medical information (pharmacological information, recent electrocardiagram, reports on recent illnesses, etc.) and be able to contact the patient's physician.

Data collection could become much more thorough. For example, records of workouts (pulse rates, duration), medication (frequency and time of day), minor illnesses (headaches, colds), even bathroom scales might be recorded regularly. While not everyone might want such a complete record of body weight, blood pressure, or temperature there are many situations (hypertension monitoring, fertility programs) that currently require more detailed histories over long periods. Research on the linkage between diet, exercise, or sleep habits on mental or physical health would be greatly facilitated by larger clinical databases. This seems like a grand opportunity.

Hypertext and hypermedia

Hypertext is rapidly emerging, but the idea of reading fragments of text on computer screens and following links still has a long way to go to gain acceptance (Conklin, 1987; Marchionini and Shneiderman, 1988; Shneiderman, 1989; Shneiderman and Kearsley, 1989). Hypertext is most appropriate when there are a large number of short information fragments that cross reference each other, and when the users need to view only a small slice at a given time. Catalogs, diagnostic problem-solving guides, business procedures manuals, organizational guides, museum exhibits, encyclopedic sources, reference books, and cookbooks seem likely candidates for practical commercial applications, but there is already lively activity among interactive fiction writers and visionary hypermedia designers who will tap the affective and emotional side (Slatin, 1988). Hypertext authors are only beginning to come to grips with effective organizing principles that permit users to recognize the structure and contents.

There is still much work to be done in improving the user interface for hypertext systems. Users need to be able to conveniently find facts they seek, browse for exploratory purposes, and rapidly discover when the database does not contain relevant information. Higher speed hardware, especially for graphics, database search, and network transactions are needed to avoid delays that distract the user from their exploration. Software improvements are necessary to:

simplify and speed the authoring process,

allow for automatic or semi-automatic loading of existing databases,

permit more powerful search strategies over large databases,

enable network linkage across databases,

support aggregation mechanisms for clustering related nodes,

provide the capacity to extract fragments or print an entire database, and

offer annotation and bookmarking.

Graphical browsing methods with a visual representation of the networks of nodes and links have been advocated by many hypertext proponents. While prototypes with a few dozen nodes seem appealing, strategies for dealing with larger networks are needed. Hardware to permit rapid and continuous zooming/panning of large networks would help, but aggregation methods may be more effective or at least a useful complement (Conklin and Begeman, 1988).

The addition of still frame and full motion video creates a compeling hypermedia environment that designers are only beginning to exploit. Imagine the potential for:

selectively viewing sports events, entertainment, or documentary footage of historical events, exploring the human body by examining microscopic slides of diseased and healthy cells, discovering foreign destinations by surrogate travel,

revisiting historical moments such as the signing of the Declaration of Independence or the coronation of Queen Elizabeth, or

studying electronic biographies of key personalities.

Our efforts during the past six years have been directed at creating an information presentation version of hypertext that has been used in museums exhibits, hyperbooks (Shneiderman and Kearsley, 1989) for corporate information distribution, training abd education, information retrieval, scientific conference programs, and organization orientation guides. Our system, Hyperties, was developed in an IBM PC version (available from Cognetics Corp., Princeton Jct., NJ) and a SUN workstation version (see Figure 2) for research purposes.
 


 
 
 

Digital photography and image handling

The attraction to images in office communications is an instance of the general appeal of visuals. The next decade should see dramatic growth in digital photography and elaborate methods for capturing, storing, editing, sending, printing, and searching image databases. Scanners are making it increasingly easy, cheap, and convenient to digitize images, but still-frame digital cameras should speed the process. Existing systems have modest resolution (one million pixels compared to 18 million pixels in typical snapshot) and are too expensive for the consumer market, but change should come quickly here. The attraction is instant viewing, reusable media, editing, titling, and electronic dissemination.

The news services regularly send color digital images for newspaper and magazine usage, allowing the photographer's output to appear anywhere in the world within minutes by cellular phone or satellite connection. Reduced cost and improved quality will attract a wide following of users who will eagerly edit images, mix with text, and send pictures of the new baby to grandparents electronically. Receiving stations could be FAX machines or personal computers that would allow further manipulations or storage. With proper indexing and hypertext linking a family album or visual genealogy might be copied or sent electronically to neighbors, distant relations, or friends.

Electronic mail and office procedures

While text processing is already widespread and has reached a certain maturity, image processing is still rare, costly, and difficult. The dramatic emergence of FAX machines is an indictment of our failure to provide good user interfaces for image manipulation. If system developers had developed convenient mechanisms for creating, sending, browsing, storing, and searching images, electronic mail might have grown much more rapidly. While there are certainly numerous appropriate applications of FAX technology, many offices create documents on word processors, print them, and send them by FAX only to have the recipient re-enter the text because it is easier than dealing with electronic mail.

It seems clear that electronic mail systems will improve and spread. Part of that improvement will be the more common inclusion of graphics, even color graphics. Further refinements will be to include spreadsheets, databases, hyper-documents, animations, and sound. It took more than 20 years for standardization of ASCII text, so it may take that long for standardization of these more complex structures, unless there is a strong consumer movement or compelling forces (government pressure or international competition) for industry cooperation. Voice mail will continue to expand and provide an appealing, more personal, easy-to-use, and accessible alternative.

Electronic filing of correspondence and enhanced search capabilities seem within reach (Malone et al., 1986). There are numerous Personal Information Managers that are suitable bases for such development (PC Magazine reviews, December 13, 1988) and the relevant hardware/software is improving steadily. Advanced facilities such as coordinated commenting on documents by multiple users in parallel, even simultaneously, should become available for use when appropriate.

Active routing slips, which provide for sequential review and approval would make a nice additional function. When the first recipient had seen the document and affixed an approval it could be automatically routed to the next recipient, with a confirmation sent to the orignator. Confirmations might be hidden or made visible to successive recipients. These pre-specified procedures might be called routing slippers, suggesting that the documents are gently walked through the sequence of recipients.

Collaborative meeting rooms and classrooms

Thoughtful networking of multiple personal workstations can lead to exciting social environments for business meetings or education (Kraemer and King, 1988; Mantei, 1989). Many businesses and universities have training rooms with multiple personal computers but each person works separately or watches as the instructor demonstrates using the large screen projector. Networking to permit shared printers, file servers, or electronic mail is a step in the right direction, but the restructuring of social roles is a necessary preliminary to the development of collaborative meeting rooms or classrooms. Could we call them collaborooms? or maybe co-rooms?

For example, if each participant could send their screen to the large shared projector, then there is the opportunity for more equal participation. In business meetings, participants could each show their sales projections or could add items to the emerging set of concerns or meeting minutes that are communally created and critiqued. In teaching, students could show clever or poor solutions for an exercise to the entire class or could add suggestions for a reading list that was being created. When the meeting or class was over each participant could copy the results to their files or print them.

If the presenter in a collaboroom has a set of notes, they could be sent electronically to each attendee for copying, annotation, or printing. No more waiting for paper copies to be made, or worries about updating each paper copy when a change is made. Online votes are another nice feature of a collaboroom: the choices are shown on the screen, each participant simply selects with a mouse click, and the results are immediately displayed for all to see. Straw votes can be taken quickly to assess the mood and then binding votes can be taken for the record.

Parallel human processing by having forty or more people work together on a problem is an attractive possibility. Imagine if a team of managers working in parallel could individually study a fraction of the reports from hundreds of franchise stores to spot high performing locations and try to identify the determinants of success. Imagine if students working in parallel could individually review a fraction of the 800 abstracts on "touchscreen applications in medical care" retrieved from a bibliographic database and select the 20-30 relevant to the class discussion.

Could programmers collaborate more effectively in design reviews or code inspections if they could easily show pieces of code to the entire group? Could students write poems collaboratively or engineers create Computer Assisted Designs for VLSI circuits cooperatively. Early successes at sites such as Electronic Data Systems and the University of Arizona are encouraging exploration in these directions.

Professional workstations & visualization

Professional users are likely to see rapid changes in the next few years, with increased screen sizes, higher resolution, multiple window facilities, faster response time, and improved use of color. Underlying technologies of database systems, networks, multi-media, sound input/output, and multiple concurrent processors will support exciting possibilities in engineering and design workstations, scientific and scholarly research tools, and exploratory simulations. As Ted Nelson has said: "If computers are the wave of the future, displays are the surfboards." We will become more aware of the truth of this aphorism as designers demonstrate remarkable visualizations of many phenomenon and domains of discourse. The bandwidth of the visual media is far greater than sound or other media and the perceptual skills of the human brain are keenly tuned to the visual (Foley and vand Dam, 1982; DeFanti and Brown, 1989). I conjecture that as screens get larger and higher in resolution, we will give up the Turing test as a meaningful idea and recognize that the goal of computer design is to offer users' a clearer vision rather than a mediocre conversation.

Natural goals for visualization are scientific phenomenon that are microscopic (cell processes, molecular layouts, or crystal lattices) or macroscopic (geologic formations, planetary motions, or galactic collisions). But other goals are to show events that are not normally visible (clear air turbulence, computer program organization (see Figure 3), tectonic plates movements, or subatomic
 
 


 
 

particle interactions). Helping physicists develop intuition about relativistic phenomenon at the speed of light or quantum theory within sub-atomic particles would be other goals. But novel possibilities of exploring emotional spaces (can we convey the depths of sadness or the heat of passion?) or knowledge spaces (how would we show the gulfs between humanities and sciences or the overlap between Eastern religious mythologies and Western psychologies?) might really expand our consciousness.

UNDERLYING TECHNOLOGY: HARDWARE/SOFTWARE

User Interface Management Systems

One of the most important emerging technologies is User Interface Management Systems and its variants such as toolkits, rapid prototyping tools, and user interface extensions (procedure libraries or language additions) to existing tools (Sibert, Hurley, and Bleser, 1988; Hartson and Hix, 1989). There is a longer history of forms building and menu management software, but the newer approaches handle graphic designs, pointing devices, and even animations. These tools will have a profound effect on the pace of user interface developments because they provide approximately an order of magnitude increase in productivity where they can be applied.

UIMSs will play a role quite similar to the Data Base Management Systems (DBMS) that emerged 15 years ago (Carey, 1988). In the same way that data independence allowed information architects to deal with the logical structure of data (entity-relationship model, relational model, etc.) and avoid concerns about disk allocation or index management, now dialog or user interface independence will allow user interface architects to deal with the interface design and avoid concerns about touchscreen drivers or cursor positioning algorithms. This separation of function can lead to much more rapid development and higher quality systems.

Along the way, we are seeing the emergence of new notations for describing interactions, methods for ensuring consistency, improved review processes, better project management, earlier and more effective usability testing, and simplifed maintenance procedures. This field is still new, but already it shows promise of becoming one of the major software classes such as operating systems, compilers, or database management systems. Progress is needed in dealing more effectively with visually-oriented direct manipulation interfaces (Shneiderman, 1987)

Remote control - do it there, do it then

An exciting opportunity exists in designing remote control mechanisms. The remoteness may be in space, that is, controlling equipment at different locations, or it may be remoteness in time, controlling equipment operations at future times. The popular hand-held remote control units for televisions, videotape players, and stereo systems are first steps towards more complex remote control devices that will allow operation of devices across telephones or high speed networks. Simple devices such as being able to operate slide projectors or computer displays in other cities will enhance the possibilities of "reaching out to touch someone." There will be useful home control products that will permit remote operation of air-conditioners, burgular alarms, or other appliances. Remote access to information will continue to grow rapidly so that users will come to expect it and even not be aware nor care that automatic bank teller machines, phones, and credit card machines are accessing databases that are geographically remote.

On a more complex plane, NASA is pursuing telescience to allow ground-based scientists to operate experiments or telescopes on orbiting laboratories (Sheridan, 1987). In general, I expect tele-operations will become more productive and exciting than autonomously operating robots. Tele-operations offer the attraction of greater human control, more flexibility and the rewards and satisfaction of accomplishment. Micro-tele-operations, will flourish with doctors controlling small surgical devices inside the human body or engineers making micro-miniaturized circuits or devices. Macro-tele-operations will allow huge signs to be painted or remote-controlled aircraft to do skywriting or reconnaissance. Tele-operations will expand underwater and space exploration. Tele-medicine will allow specialists to provide consultations to remote sites by viewing X-rays, sonograms, or tissue cultures.

Remote time control will expand with preprogrammed operation of simple household applicances such as turning on the dishwasher while the family is asleep or setting lights on while on vacation. More elaborate home controls will enable programming of week-long cycles of alarm clocks, heating/cooling, videotaping, or burgular alarm setting. Seasonal adjustments and maintenance to heating/cooling systems will be accomplished more reliably. Automatic resetting of clocks for daylight savings time will be a minor benefit. With the addition of sensors, sprinkler systems can be programmed to turn on only when the rainfall has been insufficient or heating systems can be adjusted to match patterns of home usage.

Remote operations will have an increasingly large effect on manufacturing plants where complex procedures for assembly of electronic, optical, or mechanical components, spray painting, sewing, printing, or welding can be specified and then executed repeatedly or when needed at the local plant or remote sites. Benefits to offices, hospitals, or stores can also be great.

Flexible search

One of the next great advances in computer algorithms will enable greater flexibility in locating information. Current search paradigms are quite structured and rigid. Many users wish for a perfectly flexible search paradigm that finds all and only all of the desired items, but for the moment we can expand search paradigms to be a bit more flexible.

Search algorithms currently favor two paradigms: free text search within a document to locate the next occurrence of a specified string (may include wild card characters) and index search in bibliographic databases to locate a collection of documents that match the search pattern (may include AND, OR, NOT, ADJACENT, or other operators). The flexibility of these search paradigms increases with each generation of system, in part in response to the increasing appetite of users. Still there are many situations in which search algorithms could be helpful but for which there is rare or awkward support. There is the potential for incremental but important improvements and some exciting products.

Rainbow search: First, for searches within a document, there is the chance to enable rainbow searching. Most search facilities treat text as if it had only a single color, but most word processors already support a range of colorful features such as multiple fonts, font sizes, styles (italic, bold, underscore, etc.), and text attributes (footnotes, references, titles, headers, footers, etc.). It might be useful to allow users to locate the next bold-faced occurrence of George Washington or search only through the footnotes. This is fairly easy to implement, but some re-design of the user interface is necessary. In addition to searches, global changes such as replacement of all bold-faced items with italic might be useful.

Search expansion: For searches across documents, increased ability to specify search within a component, such as the title, abstract, or conclusion might be helpful in some situations. Search expansions would also facilitate solution of some problems, for example, if the user desires to locate documents dealing with "New England", the system thesaurus, would inform the user that the search could be expanded to include "Connecticut", "Rhode Island", "Massachusetts", "Vermont", "New Hampshire", and "Maine". Search expansions might provide more specific terms (as in this example), more general terms, synonyms, or related terms.

Sound search: Search concepts are increasingly important for non-text databases. For example, imagine a music database that would enable the user to hum a few notes and would produce a list of symphonies that contain that string of notes. Then with a single touch, the user could bring up the full symphony. Doing this in the unstructured world of analog or even digitally encoded music is very difficult, but imagine that the score sheets of the symphonies were stored with the music on the CD-ROM or CD-I disks and that string search over the score sheets were possible. Then the application becomes easier to conceive of. Identifying the user's hummed input may not be reliable, but if visual feedback were provided or if the user entered the notes on a staff, as in the Deluxe Music Construction Kit on the Macintosh, then the fantasy becomes feasible.

Picture search: Other non-text applications are also appealing. Imagine a database of MacDraw pictures. It would be nice to be able to find all the pictures with a circle inside a square or maybe a green square to the left of a yellow oval. If the data representation were searchable, then these tasks would be relatively simple, although there are some interesting user interface problems to be solved. Imagine CAD/CAM database that could be searched to find electrical circuits that used components in a certain relationship or automobile engine database that could be searched to find designs in which valves were within 3 cm of each other. Imagine weather map databases that could be searched to find regions with more than 30 mi/hr gusts at the boundary of a low pressure system. In all these cases, there is some representation of the contents that is more easily searched than the bit patterns. When computer image analysis programs can be effectively applied, for example, identifying defective red blood cells or light bulbs, then the results of such analyses could also be used in search algorithms.

Photo Libraries: Current photo libraries are organized according to date, photographer, geographic regions, or topic, but indexing is sometimes inaccurate and cross referencing incomplete. A powerful "faceted" index that dealt with all these topics and still others would be a great benefit. If digitally photographed images could contain encoded information on the date, photographer, place, personalities, and some topical references then some of the burden of indexing would be lightened and an economical and successful photo or video library could be built. I think we are still a long way from systems that could automatically identify all television images that contained a handshake, ribbon-cutting, or other topic, but when there is some representation of the contents then a flexible search paradigm can enhance the user's power.

Display technology - screens / processors

Better resolution & larger displays: Early computer displays showed 16 rows of 40 characters each and no graphics. Display technology and manufacturing has rapidly improved so that 25 rows by 80 characters are common and graphics displays have improved from 640 x 200 pixels to 640 x 480 pixels on standard personal computer displays. For many professionals far higher resolution is common, 1024 x 768 or 1270 x 1024 on a large color display. It seems clear that the resolution of screens will continue to improve (4096 x 4096 pixels or the 1125 scan lines of high definition television are the next major destinations) and that users will increasingly desire larger and more readable displays. New fonts, better use of color, improved contrast, less flicker, and other improvements will help improve the readability of displays over time (Mills and Weldon, 1989). While current displays take 30% longer to read than typewritten text, new displays with anti-aliased fonts or higher resolution have been shown to match the reading speed from paper.

Window macros: Major gains will also come from more effective ways of using the existing display space. Current muliple window strategies that depend on the user to manage the size, position, and contents of each window will surely give way in many applications to more carefully conceived strategies that bring up windows, arrange them, fill them, and close them as a function of the user's tasks. For example in a medical application, if the physician is consulting a treatment database and for the specified sympton there are six possible treatments, then the system should bring up six windows that cover the screen in a tiled manner. If the physician scrolls any window to see the contra-indications, then all six windows should scroll synchronously. Finally, when a treatment is selected, all six windows disappear and the display returns to its previous state. Such window macros should become common and easily specified.

Zoom & Pan: There are still many applications where even the largest display is inadequate to show the full context. Node/link diagrams for browsing hypertext, weather maps, city maps, transportation networks, and computer program structure charts are a few examples. While more thoughtful methods for abstraction and clustering are the preferred direction, there is still a use for detailed visual representations. However, current hardware technology makes zooming/unzooming and panning (left/right and up/down) unacceptably slow and disturbingly discontinuous. It would be quite exciting and useful to be able to see the whole world on the display and then be able to zoom in on a continent, then country, then city and finally see familiar local streets. With only a bit more effort we should be able to zoom in on or pan over to apartment houses, check the list of residents, and pick up phone numbers or other publicly available information. How about zooming in on the Library of Congress and continuing on through the card catalog to zoom in on the full text of a book. There are severe hardware restraints that prevent smooth zooming and the algorithms are still lacking. Vast increases in parallel procesing power are needed to enable such zooming scenarios, but why not dream. Parallel processor have been built for computation (inverting matrices or solving differential equations) and for database search, but great gains could come from parallel processing for screen displays. Maybe we should dream about making one processor per pixel to enable rapid display of information.

Improved touchscreens

Touchscreens are appealing as input devices because:

- the user points directly at the object of interest

- their potential for simplicity and low cognitive load

- they require little or no training

- they have been shown to be the fastest of pointing devices for many tasks

- there are no moving parts or extra equipment beyond the monitor

- no additional desk space is needed.

The prime advantages of simplicity, speed, and durability make touch screens attractive for public access information kiosks, banking terminals, marketing situations, learning environments, automobiles, factory environments, home automation, and many other situations.

New strategies for touchscreen usage are dramatically improving the potential for much wider usage of these devices (Pickering, 1986; Potter, Weldon and Shneiderman, 1988; Potter, Berman and Shneiderman, 1989). While many textbooks and previous implementations limited touchscreen resolution to the size of the user's finger, newer strategies enable pointing to single pixels (Sears and Shneiderman, 1989). Instead of simply pointing to targets, users can now touch the screen surface to produce a small cursor above the center of their fingers. Then they can drag the cursor and when the placement is as desired, they merely lift-off their fingers to initiate action. Touchscreens have consistently been shown to be the fastest of pointing devices and now problems with precision can be overcome. Of course, there are still problems of hand obscuring the screen, possible smudging, and possible arm fatigue, but each of these problems can be responded to in practical ways. Some touchscreens permit multiple simultaneous touches and others offer a third dimension, that is, the forcefulness (pressure sensitivity) of the touch.

Of course there are still some potential problems with touchscreens:

- arm fatigue if the touch surface is at an uncomfortable location

- callouses on finger tips with heavy use

- insufficient tactile feedback to support touch typing

- fingers can obscure important parts of the screen

- smudges can accumulate

- optical interference or increased glare.

The problems can often be overcome or reduced, but they should be considered during the design process.

In our hypertext research the touchscreen has proven to be an excellent device for selecting links in text and graphics. The low cognitive load of touching highlighted terms or objects is extremely important since it enables users to maintain concentration on the information contents.

Dragging an icon with a touch mouse is a useful possibility that parallels the mouse. Actually it is often more dramatic to drag an icon with the finger than with a mouse. We have had great success and fun in dragging paint icons (to create a finger painting package), clock hands to set the time of day, and flags to mark begin and end points on time lines. Dragging musical notes onto a score sheet or ketchup, mustard, and onions onto a hotdog/hamburger are engaging construction kit ideas that will surely grow in popularity among designers.

Summary

There is excitement in the user interface research and development community. New ideas are emerging daily and the reduction to practice is rapid. Perfection is not anticipated, but there is clear progress in many areas beyond the ones described in this paper. Designers and researchers are invited to propose other directions, disagree with my assessments, or seek more ambitious visions. Beyond the technology, I strongly encourage vigorous discussion of the high-level goals and the dangers. Discussing the future is an important part of the process of creating it.
 
 

References

Carey, Tom, 1988. The gift of good design tools, In: Hartson, H. R. and Hix, D. (Editors), Advances in Human-Computer Interaction: Volume II, Ablex Publishers, Norwood, NJ, pp. 175-213.

Conklin, J., 1987. Hypertext: A survey and introduction, IEEE Computer 20, 9, (Sept., 1987), pp. 465-472.

Conklin, J. and Begeman, M., 1988. gIBIS: A hypertext tool for exploratory policy discussion, ACM Transactions on Office Information Systems 6, 4, (October 1988), pp. 303-331.

DeFanti, Thomas A. and Brown, Maxine D., 1989. Scientific animation workstations: Creating an environment for remote research, education, and communication,Academic Computing, (February 1989), pp. 10-12, 55-57.

Foley, J. D. and van Dam, A., 1982. Fundamental of Interactive Computer Graphics, Addison-Wesley Publ., Reading, MA.

Hartson, H. Rex, and Hix, Deborah, 1989. Human-computer interface development: Concepts and systems, ACM Computing Surveys 21, 1, (March 1989), pp. 5-92.

Kraemer, K. L., and King, J., 1988. Computer-based systems for cooperative work and group decision making, ACM Computing Surveys 20, 2, (June 1988), pp. 115-146.

Malone, T., Grant, K., Lai, K.-Y., Rao, R., and Rosenblitt, D., 1986. Semi-structured messages are surprisingly useful for computer supported coordination. In: Proc. of CSCW'86 ACM Conference on Computer Supported Collaborative Work, Austin, TX , pp. 102-114.

Mantei, Marilyn, 1989. A study of executives using a computer supported meeting environment, Decision Support Systems, (to appear).

Marchionini, Gary and Shneiderman, Ben, 1988. Finding facts vs. browsing knowledge in hypertext systems, IEEE Computer 21, (1), pp. 70-80.

Mills, Carol and Weldon, Linda, 1987. Reading text from computer screens, ACM Computing Surveys, 19, (4), pp. 329-358.

Pickering, J. A., 1986. Touch-sensitive screens: The technologies and their applications, International Journal of Man-Machine Studies 25, pp. 249-269.

Plaisant, Catherine & Shneiderman, Ben, 1989. Scheduling ON-OFF home control devices: Design issues and usability evaluation of four touchscreen interfaces , Human-Computer Interaction Laboratory, University of Maryland, (submitted for publication).

Potter, R., Berman, M., and Shneiderman, B., 1989. An experimental evaluation of three touchscreen strategies within a hypertext database, International Journal of Human-Computer Interaction, 1, (1).

Potter, R., Weldon, L., and Shneiderman, B., 1988. Making touchscreens effective: An experimental evaluation of three strategies, Proc.ACM CHI'88: Human Factors in Computer Systems, pp. 27-32.

Sears, Andrew and Shneiderman, Ben, 1989. High precision touchscreen: A comparison of touchscreens and a mouse, International Journal of Man-Machine Studies (to appear).

Sheridan, Thomas B., 1987. Teleoperation, telepresence, and telerobotics: Research needs for space, In: Sheridan, Thomas, B., Kruser, Dana S., and Deutsch, Stanley (Editors), Human Factors in Automated and Robotics Space Systems: Proceedings of a Symposium, Committee on Human Factors, National Research Council, Washington, DC, pp. 279-291.

Shneiderman, Ben, 1987. Designing the User Interface: Strategies for Effective Human-Coomputer Interaction, Addison-Wesley Publ., Reading, MA.

Shneiderman, Ben and Kearsley, Greg, 1989. Hypertext Hands-On! A new way of organizing and accessing information, Addison-Wesley Publ., Reading, MA.

Sibert, J. L., Hurley, W. D., and Bleser, T. W., 1988. Design and implementation of an object-oriented user interface management system, In: Hartson, H. R. and Hix, D. (Editors), Advances in Human-Computer Interaction: Volume II, Ablex Publishers, Norwood, NJ, pp. 175-213.

Slatin, John M., 1988. Hypertext and the teaching of writing. In: Barrett, Edward (Editor), Text, Context, and Hypertext, MIT Press, Cambridge, MA, pp. 111-129.