Show Me! Guidelines for Producing Recorded Demonstrations

Catherine Plaisant and Ben Shneiderman

Department of Computer Science & Human Computer Interaction Laboratory

University of Maryland, College Park, MD 20742

Abstract.  Although recorded demonstrations (screen capture animations with narration) have become a popular form of instruction for user interfaces, little work has been done to describe guidelines for their design. Based on our experience in several projects, we offer a starting set of guidelines for the design of recorded demonstrations. Technical guidelines encourage users to keep file sizes small, strive for universal usability, and ensure user control etc. and provide tips to achieve those goals. Content guidelines include: create short demonstrations that focus on tasks, highlight each step with auditory and visual cues, synchronize narration and animation carefully, and create demonstrations with a clear beginning, middle, and end.

1. Introduction

Standardization and improved designs have made user interfaces easier to learn and use, however getting started with unfamiliar interfaces, learning advanced features and understanding application domain concepts remains a challenge for many users.  As the goal of universal usability has become more prominent, especially in digital government applications, the need for dramatic improvements in tutorial and help methods has grown. While government agencies seek to make their information resources  and services available to an ever-growing set of users, guidelines for web design proliferate and improve by building on research results (Koyani et al., 2003), but there is little research about guidelines for helping users get started with new web applications.

Designers of any user interface documentation must make a variety of design decisions, including choosing an appropriate technology that matches the content. Paper documentation is valuable in presenting concepts, goals, and terminology, while graphics are helpful to convey site maps, state transitions, and data structures. Animations have long been promoted as an even more compelling way to educate users by showing the dynamics of user interface actions. However, the research results have often surprised and frustrated the promoters of animations, since it is rare to find measurable benefits for users of animations (Palmiter & Elkerton, 1991) (Morrison & Tversky, 2000). User satisfaction with animations is usually high, but animations appear to distract users from concentrating on key issues (Weiss, Knowlton & Morrison, 2002).

One theory is that animations lead mainly to mimicry and superficial learning (Atlas, Cornett, Lane & Napier, 1997). Rieber, et al. (1991) did a comparison study to examine the effects of different levels of visual elaborations (animated graphic, static graphic, and no graphic) on adult application learning in a computer-based science lesson. No significant results were found in this study when the static and animated visuals were added to the text. Palmiter et al. (1991) compared the efficacy of learning computer-based tasks by using animated demonstrations, procedural textual instructions and combination of demonstrations and spoken text. They found that the demonstration groups were faster and more accurate for learning the procedural tasks, but, interestingly, seven days later the text group was faster and as accurate when performing identical and similar tasks. It seems that the retention issue needs to be addressed more fully. The results in Harrison’s (1995) comparison study suggested that using visual instruction in on-line help, either still graphic or animated, enabled the users to perform more tasks in less time and with fewer errors than those users who didn’t have visual instruction, but there was no significant difference between the performance of subjects in the still graphic conditions and the animated conditions.

A review of instructional uses of animation suggested that the effectiveness of animation appears to be confined to younger learners (Wetzel, Radtke, & Stern, 1994).  This review also reported that using animation to present content is less effective than other uses of animation, such as the four were identified by Rieber (1994): directing attention, presenting information, enhancing practice, and cosmetic appeal.

A more recent review covered research on computer animation in education, human-computer interaction and psychology (Bétrancourt & Tversky, 2000).  These authors concluded that computer animation is not a panacea, but it can improve users’ performance and attitude under certain circumstances. They summarized five factors that could influence the effectiveness of animation: the content to be animated, the level of interactivity, objective of animation, design of animated interface, and individual differences.

Rieber (1990) presented three design recommendations for the use of animated visuals in instruction: (1) Animation should be incorporated only when its attributes are congruent to the learning task. Visualization, motion, and trajectory were three attributes considered to be brought by animation. (2) When learners are novices in the content area, they may not know how to attend to relevant cues or details provided by animation. (3) Animation’s greatest contributions to computer based instruction (CBI) may lie in interactive graphic applications.

A study of students showed that the group that was given both animated and auditory descriptions about the operation of a bicycle tire pump performed better on the tests of creative problem solving than those only given animated description and those only given the auditory description (Mayer and Anderson, 1991).

Haas and al. (2005) compared text, graphic and animation to provide meanings of statistical terms.  There was little difference among formats in the effectiveness of presentations.  User preferences varied but results suggest that the type of terms being explained made a difference, animations being more useful when there is an action or process.  The lack of control of the timing of animations was seen as a problem.

These studies provided important guidance for our project to develop recorded demonstrations for web-based statistical software.  We also studied the growing variety of animated demonstrations on the web to form and test our emerging hypotheses about guidelines for design.  This paper presents a set of guidelines and tips to help designers prepare successful recorded demonstrations, which we playfully call “Show Me!” demonstrations.  This term seems more active and conveys user control more strongly than the traditional term “Help”, which has a desperate tone to it that may undermine user eagerness.  We also reflect on the need and difficulty to quantify the relative benefit of design elements of recorded demonstrations, and summarize our experience and feedback from users.

2. Exploration of recorded demonstrations

A review of recorded demonstrations on the WWW that introduce users to a new interface revealed three major types:

 

1)       Video movies show a person or the hands of a person using an interface.  They are shot with a video camera and are effective at showing hardware manipulation or providing the nice touch of a human welcome.  They are usually limited to a small window size for performance reasons.   An example is the demonstration of body movement sensors of Ascension Technology (http://www.ascension-tech.com).

 

2)       Composed animations include screen shots of the interface augmented with hand-made animations.  An example is the demonstration of the interface of Endeca (Fig. 1) which combines an overview of the interface and explanation of the algorithms underlying the search interface (Fig 1).  Those animations are usually created by multimedia professionals using tools such as Macromedia Director and Flash.

   

Fig. 1. Screens from a recorded demonstration of Endeca search. On the left, cropped views of real screens are annotated and animated.  On the right, animation is used to explain the search process.  (www.endeca.com/demos)

3)       Recorded demonstrations demonstrate how a user interface can be used by replaying and narrating the recording of interface use. A good example can be seen at Amazon.com (Fig. 2).  Recorded demonstrations are simple to produce and narrate with recording tools such as Camtasia and generate faithful demonstrations of the interface.

 

Fig. 2. A good example of recorded demonstration can be found on Amazon.com (select “New Customer: start here” at the top of the homepage). The demonstration shows the top portion of a full size view of the interface and is streamed using Siteseer technology which does not require any plug-in. An audio transcript is available on demand (see bottom left of the screen). (www.amazon.com,

Of course hybrid techniques exist as well and a recorded demonstration can be included in a composed animation. This paper focuses on the third type – recorded demonstrations, because we believe they are a very effective method to provide help, and because they can be produced easily and therefore likely to be used widely. We also focus on the demonstrations of interfaces as opposed to explanations of concepts (Haas et al. 2005).

Our guidelines are based on several threads of experience.  First we have a long history of developing video demonstrations of our own work.  Since 1991 the Human-Computer Information Lab has produced video reports at the occasion of its annual Symposium and Open House, and several segments have also become part of the CHI video proceedings.  The videos served as archival record of our work and have been used by educators and researchers to illustrate interaction techniques to students and colleagues. Over the years we refined our techniques to write scripts and record short and effective videos in the space of a few hours with the help of a video professional pointing a camera at the screen.  

More recently, we have worked on developing new methods for providing help to users of government websites offering statistical information. In particular we looked at how to help users get started with a tool allowing them to get information about US states using a map, scatterplots, table and sliders for filtering. This work led to a series of interfaces using multi-layered design, a new help method called Integrated Initial Guidance, and recorded demonstrations (Plaisant et al, 2003; Kang et al. 2003).  Usability testing with 12 participants suggested that recorded demonstrations were quite effective and satisfying to users. As part of this project we generated recorded demonstrations as help for our own tools such as Treemap 4.0 (Fig. 3). In presenting these results to online help developers we received enthusiastic approval of these rapid production techniques, thereby encouraging us further to recommend the technique widely.

 

     

Fig. 3: Treemap 4.0 online documentation page offers both textual descriptions and recorded demonstration for each section of the user manual. On the right, a sample screen from a recorded demonstration.  (www.cs.umd.edu/hcil/treemap)

 

Finally, we reviewed web-based recorded demonstrations in order to assess current practices. In connection with our research on digital government, we emphasized US and state government sources (Appendix 1), so our list is far from comprehensive, but is meant to be representative of commonly used strategies.  The features we collected were related to the usability aspects of these recorded demos.

 

Size:                  Size in kilobytes of the downloaded presentation

Buffering:          Can the presentation start before it has finished downloading?

Length:              Length in minutes and seconds

Pause:               Can the user pause the presentation?

Seek:                 Can the user seek to a specific location in the presentation?

Time Display:  Is there a control that indicates the position in the presentation?

Voice:                Is there voice narration?

Text Boxes:       Are there additional text boxes that are used in the presentation?

Click Cues:       Are there audio/video cues to let the user know the mouse was clicked?

Highlighting:    Was the mouse cursor or interface highlighted to help guide the user?

 

While our Treemap 4.0 recorded demonstrations were designed to have all these features, none of the others had the full set of features. Some of the recorded demos had excellent production values, such as professional narration and musical accompaniment, but the range was large.  We found that one major improvement to content would be to remove long pauses and edit the demonstration more tightly.  The recorded demonstrations without voice instructions were plainly less appealing and more difficult to follow, and smaller file sizes were a strong asset.

3. Guidelines for recorded demonstrations

 

We propose a set of ten guidelines and numerous tips to help designers create effective recorded demonstrations. We should start with the importance of making access to these Show Me!’s visible and attractive, but not intrusive. The guidelines attempt to remain general and their applicability should be reviewed in the context of the interface being demonstrated and the target user population.  

1) Provide procedural or instructional information rather than conceptual information

Procedural information describes the steps required to complete a specific task, and the goal for this type of information is to walk users through the successful and immediate completion of the task. Conceptual information provides important theory and background information, but users may not need it to complete a task. Instructional information is a hybrid of procedural and conceptual information that encourages more comprehensive learning experiences for users. Keep the goal of showing procedure prominent in your thinking and scripting.

Tips:   - Base your script on a live demonstration (don’t start from a paper)

            - Pretend to be a user thinking-aloud while performing a real task

            - Do not advertise

2) Keep segments short

Each segment should be as short as possible (15 to 60 seconds) in order to keep users engaged while exploring the documentation and minimize how much users need to remember at once. Keeping each segment autonomous, while planning a linear sequencing between segments will allow users to see them individually as needed, or all at once as a tutorial.  Avoid showing multiple tasks in one segment.  If you decide to combine tasks, consider using short titles to separate sections of the recording (example: “Printing”, then later “Exporting”). 

Tips:    - Use the same example data or follow related examples across all segments.  

-  Starting the first segment with a tour of main screen components orients and introduces the vocabulary of the interface

- Starting each segment from a consistent screen also orients users.

3) Ensure that tasks are clear and simple

Choose understandable simple example tasks, and use familiar terms. Speak simply, directly, and accurately based on the following criteria; use active sentences, make positive assertions, keep sentence simple, speak directly to users, and state quantities exactly.  Avoid abbreviations.

Tips:    - Prepare a script

            - Use the same example data or follow related examples across all segments.

- Act out the interaction then describe changes seen on the screen

- Use concrete examples, e.g. do not say “selecting a color attribute with apply this attribute to all the objects presented on the screen”.  Instead say – and act out - “if you select “Gender” as the color attribute,  all men are now colored blue and women red”, and point at the effect as you talk. 

- Edit, edit, edit, to cut all unnecessary words: e.g. do not say: “at the top left of the screen users will find the list of attributes, at the bottom right users can find the control panel”. Instead say: “here is the list of attributes… and the control panel” while pointing or highlighting the corresponding areas, in perfect synchronization with the narration.

4) Coordinate demonstrations with text documentation

Often, recorded demonstrations complement text documentation to provide users alternate ways of learning the materials.  Text documentation can provide more details and should be preferred to explain concepts that are not interactive. A common mistake is to develop them separately so they do not use the same structures or even vocabulary.  Integrate text documentation and demonstrations and let users choose what is most appropriate for them. 

Tips:    - Make recorded demonstrations available right at the start of the application or where ever they are most likely to be needed by first-time users. 

            - Make the online documentation table of contents a table, clearly showing alternative formats available.

5) Use spoken narration

We highly recommend the use of voice narration (opposed to textual explanations). Users can easily attend to voice while watching the demonstration, and even possibly interacting with the interface. Narration keeps users engaged and paying attention. However, if narration and demonstration are not compatible or properly synchronized, it can confuse users.  A common mistake is to have the voice give generic “advertisement-like” information about the interface while users are trying to understand what operation is actually being demonstrated on the screen.  Another mistake is for the narration to be out-of-synch with the action.

Tips: - Look for a narrator that has a lively voice and speaks clearly 

 - Favor a narrator who is likely to be around when time comes to update

 - Ideally the same person will be able to control the interface and narrate at the same time to ensure good synchronization, but practically most people have difficulties doing this, and it often results in lengthy recordings.  Our favored alternative is to work as a team, where one person uses the interface and the other narrates.  Another alternative is to record demonstration and narration separately, then combine them with an editor. Several trials may be needed to adapt the narration speed and the speaker intonation to the rhythm of the interaction.

6) Be faithful to the actual user interface

Create a recorded demonstration that gives viewers the impression that they are watching the real interface. Showing the full size user interface helps users remember the location of interface widgets. Do not shrink the recorded demonstration if at all possible.  

Tips:    - Zoom in on portions of the display when readability is a severe problem.

            - If you cannot create a rapid and zoom transition to the detail area, first highlight the area where zooming will take place, then jump to it.

7) Use highlighting to guide attention

Narration can help understand what the user actions are but providing additional highlighting of the mouse cursor by adding circles, blinking features, or spotlights will help focus users’ attention on where the action is. Sound effects can represent mouse clicks or sliding motion.

 

There are many reasons to use sound in online documentation: to support visuals, to attract attention and interest, to improve perceptions of quality, to represent things not shown, to provide feedback, to aid memory, to preserve continuity, and so on. Likewise, in video documentation, sound plays an important role in attracting users’ attention, providing feedback, and most importantly representing interactions between the interface and input device. Combining sound with the visual effect of an input device (mouse clicks or keyboarding sounds) often makes it easier for users to understand and recognize what is happening within the interface, and sometimes it allows representing more diverse input interactions. For example, combining a mouse click sound and an animated circle around the cursor help users easily recognize when and where the cursor is clicked over the interface. Moreover, changing the color of animated circle around the cursor makes it possible to distinguish left mouse click from right mouse click.

Tips: - Don’t use too many highlights, and avoid highlighting effects that can be confused with the interface itself.

            - Wiggling the cursor is a simple way to attract attention and highlight a part of the screen, but stop moving the cursor afterward. Incessant and meaningless cursor movement will be dizzying, and confusing.

8) Ensure user control

If the recorded segment is longer that 15 seconds, give users controls so that they can skip the familiar parts, find the part they want or stop and replay. Speed control might be useful in some cases.  Adding time length information and progress feedback helps users recognize their position in the recorded demonstration.        Providing jumps to the next segment in a logical sequence is also helpful.

9) Keep file sizes small

If the recorded demonstrations are designed to be displayed on the internet for online help, the response time has to be good since users will not have the patience to wait several minutes to see only a few seconds of demonstration.  For example, suppose the recorded demonstration need to be accessible through a 56 Kbps dial-up modem connection, then the maximum size of a recorded demonstration that can be downloaded per minute is 420 KB (56Kbps (bit per second) * 60 seconds = 420 KB (bytes)). Unless the created video file is less than 420KB/min, video quality and response time should be adjusted and quality reduction may be needed. If resolution, color depth, and video frame rate are essential, then the response time should be delayed to buffer a certain amount of file before playing or completely download it. On the contrary, if the response time is more important factor, then the video should be reduced and compressed to meet the rate of 420KB/min at the cost of video quality.

Tips:    - Avoid actual video recording, instead use only on-screen recording of the interface which will generate much smaller files.

- Experiment with different output formats, e.g. saving in Flash format might save a lot of space.

- Provide duration and file size for each individual segment

- Provide both streamed and downloadable versions.

10) Strive for universal usability

As mentioned in the previous guideline, the size of Show Me! files is an essential factor to make the recorded demonstrations practical for use in online documents. Nevertheless providing universal access will requires a careful review of technology choices.  For example video codec, audio codec, plug-ins or dedicated players are often considered to reduce the size of files or enhance the response time, but designers should consider carefully if their users will have access to those technologies or the patience to install them. 

Another consideration is to address the needs of users with disabilities.  Providing transcripts of the narration for deaf users and textual descriptions for blind users will benefit others that have poor hearing or limited English competency.  It also makes the demonstrations visible to search engines, and allow users to capture and print snapshots of the demonstrations. Control of the replay speed will be appreciated by everyone. 

Tips:    - Indicate that your demo has audio

            - Keep looking for new technology and formats that do not require users to install software.

4. Conclusions                                                    

Although the benefits of recorded demonstration for learners of user interfaces have not been quantitatively measured, there have been a few investigations that suggest that recorded demonstrations help users learn procedural tasks (see introduction). Our experience using Show Me!’s with our own products has been positive. For example the release of the recorded demonstration of Treemap has clearly changed the nature of the questions submitted to us.  Before the release we would receive many questions about procedures e.g. “How do I change the colors?” or “Is there a way to do X” where X was possible and described in the text manual.  Now we have NO questions of this nature, while questions about features that should be added, or licensing questions remain. We also receive nice feedback from users such as:

 

“First of all, thanks very much for giving me the opportunity to explore your Treemap application with great ease. Through the self-explanatory demo, it allows me to truly appreciate the immense potential of such visual analytical application”                                                                                      A new treemap user from Singapore

 

We also observe a growing pattern of adoption (e.g. Amazon, Endeca) which suggests that web service providers find them effective, or at least easy enough to produce and keep up-to-date.  A useful next step would be to gather usage data from those websites (e.g. are first comers that use them more likely to purchase?).  A set of controlled experiments could measure the benefits of individual features (but the diversity of types of interface and users will make an exhaustive evaluation particularly challenging.)

Instead of jumping to lengthy tutorials or manuals new users first intuition is to ask a colleague to give them a quick tour when they encounter a new interface. We believe that recorded demonstrations are a distant surrogate for such active and practical demonstrations and we hope that our guidelines will help designers produce  more effective recorded demonstrations.

Acknowledgements

We greatly appreciate the implementation efforts of Hyunmo Kang in developing our Treemap 4.0 recorded demonstrations. We also thank the students who helped us review current practices (Michael Anderson, John Booze, Pratik Mathur, Vikram Mittal and Dariush Samari) and all the members of the Govstat project (see http://ils.unc.edu/govstat/).  This material is based upon work supported in part by the National Science Foundation under Grant No. EIA 0129978, and the US Census Bureau.  

References

1.     Baecker, Ronald, Small, Ian, and Mander, Richard (1991). Bringing icons to life, Proc. CHI ’91 Human Factors in Computer Systems, ACM, New York, 1–6.

2.     Bétrancourt, M. & Tversky, B. (2000). Effect of computer animation on users’ performance: a review. Le travail Humain, 63(4), 311-330.

3.     Carroll, J.M. (1997). Minimalism beyond the Nurnberg Funnel, MIT Press, Cambridge.

4.     Dominick, J., Hughes, A., Marchionini, G., Shearer, T., Su, Chang, and Zhang, J. (2003). Portal help: Helping people help themselves through animated demos, Univ. of North Carolina Information & Library School Technical Report, available at http://ils.unc.edu/ils/research/reports/TR-2003-01.pdf

5.     Hackos, J. T. and Stevens, D.M. (1997). Standards for Online Communication, John Wiley & Sons, New York.

6.     Harrison, S.M. (1995). A comparison of still, animated, or nonillustrated on-line help with written or spoken instructions in a graphical user interface, Proceeding of ACM CHI'95 Conference on Human Factors in Computing Systems, ACM, New York, 82-89.

7.     Haas, S., Brown, R., Cao, L., and Wilbur, J. (2005). Evaluation of the GovStat Statistical Interactive Glossary:  Implications for Just-in-Time Help.  U. of North Carolina Technical Report, http://ils.unc.edu/govstat

8.     Horton, William K. (1994). Designing and Writing Online Documentation: Hypermedia for Self-Supporting Products, John Wiley & Sons, New York.

9.     Kang, H., Plaisant, C., and Shneiderman, B. (2003). New approaches to help users get started with visual interfaces: Multi-layered interfaces and integrated initial guidance, Proc. 2003 National Conference on Digital Government Research (2003), http://www.dgrc.org/dgo2003/.

10.  Mayer, E., and Anderson, R. B. (1991). Animations need narrations: An experimental test of a dual-coding hypothesis, Journal of Educational Psychology 83, 484-490.

11.  McGrenere, J., Baecker, R., and Booth, K. (2002). An evaluation of a multiple interface design solution for bloated software, Proc. CHI’02 Human Factors in Computing Systems, ACM, New York 163-170.

12.  Morrison, Julie B. and Tversky, Barbara (2001). The (in)effectiveness of animation in instruction, Proc. CHI 2001 Human Factors in Computer Systems: Extended Abstracts, ACM, New York, 377-378.

13.  Morrison, Julie B., Tversky, Barbara, and Betrancourt, M. (2000). Animation: Does it facilitate learning, Proc. Workshop on Smart Graphics, AAAI Press, Menlo Park, CA.

14.  Palmiter, S., Elkerton, J. and Baggett, P. (1991). Animated demonstrations vs. written instructions for learning procedural tasks: A preliminary investigation, International Journal of Man-Machine Studies, 34, 687-701.

15.  Payne, S. J., Chesworth, L., and Hill, E. (1992). Animated demonstrations for exploratory learners, Interacting with Computers, 4, 3–22.

16.  Plaisant, C., Kang, H., and Shneiderman, B. (2003). Helping users get started with visual interfaces: multi-layered interfaces, integrated initial guidance and video demonstrations, Proc. Human-Computer Interaction International 2003: Volume 4 Universal Access in HCI, Lawrence Erlbaum Associates, Mahwah, NJ (June 2003), 790-794.

17.  Rieber, L. P. (1991). Animation, incidental learning, and continuing motivation. Journal of Educational Psychology, 83(3), 318-328.

18.  Rieber, L. P. (1994). Computers, Graphics and Learning, Brown & Benchmark Publishers, Madison, WI.

19.  Shneiderman, B. and Plaisant, C. (2004).  Designing the User Interface: Strategies for Effective Human-Computer Interaction (fourth edition), Addison-Wesley Publishers, Boston, MA.

20.  Shneiderman, B. (2000). Universal Usability: Pushing human-computer interaction research to empower every citizen, Communications of the ACM 43, 5 (May 2000), 84-91.

21.  Sukaviriya, Piyawadee “Noi” and Foley, James D. (1990). Coupling a UI framework with automatic generation of context-sensitive animated help, Proc. User Interface Software and Technology, ACM, New York, 152–166.

22.  Weiss, Renee E., Knowlton, Dave S., and Morrison, Gary R. (2002). Principles for using animation in computer based instruction: Theoretical heuristics for effective design, Computers in Human Behavior 18, 465-477.

23.  Wiedenbeck, S. & Zila, P. L. (1997).  Hands-on practice in learning to use software: a comparison of exercise, exploration, and combined formats, ACM Transactions on Computer-Human Interaction (TOCHI), 4(2), 169-196.

 

 


Appendix 1:

 

A sample of recorded demonstration examples, collected during our exploration of current practices. We collected features related to the usability aspects of recorded demos from state government web sites and other sources.