ABSTRACT

 

Title of Dissertation:            USER INTERFACE TECHNOLOGIES AND GUIDELINES TO SUPPORT CHILDREN’S CREATIVITY, COLLABORATION, AND LEARNING

Juan Pablo Hourcade, Doctor of Philosophy, 2003

Dissertation directed by:      Professor Benjamin B. Bederson

                                          Department of Computer Science

 

 

Computers are failing children. They are taking time away from meaningful interactions with people, and are often providing children with inappropriate experiences. In particular, they are failing to support children collaborating, being creative, using their imagination, and accessing appropriate content. To address these issues, I have created developmentally appropriate technologies that support children collaborating, creating, and learning. To support collaboration, I developed MID (Multiple Input Devices), a Java toolkit that supports advanced events, including those from multiple input devices. I used MID to develop KidPad, a collaborative storytelling tool that supports groups of children in the creation of drawings and stories. To support collaboration in a concrete, developmentally appropriate manner, KidPad uses the local tools user interface metaphor in which I implemented several improvements to make efficient use of screen space and to encourage collaboration. SearchKids is an application that also supports collaboration and gives children the ability to search and browse a multimedia animal library. The International Children’s Digital Library uses a similar user interface to enable children to search and browse an international collection of digitized children’s books. Both applications offer children access to curated collections, shielding them from inappropriate content while keeping them in control of what to experience. While building these technologies I observed that young children had greater difficulty using input devices. This affected their ability to collaborate, be creative and access valuable content. Motivated by such observations, I conducted a study to gain a better understanding of how young children use mice as compared to adults. The results provide guidelines for the sizing of visual targets in young children’s software and insight into how children use mice.

 

 

 

 

 

 

 


 

 

 

 

 

USER INTERFACE TECHNOLOGIES AND GUIDELINES TO SUPPORT CHILDREN’S CREATIVITY, COLLABORATION, AND LEARNING

 

by

 

Juan Pablo Hourcade

 

 

 

 

Dissertation submitted to the Faculty of the Graduate School of the

University of Maryland, College Park in partial fulfillment

of the requirements for the degree of

Doctor of Philosophy

2003

 

 

 

 

 

 

 

 

Advisory Committee:

      Professor Benjamin B. Bederson, Chair

      Professor Allison Druin

      Professor François Guimbretière

      Professor Atif Memon

      Professor Saundra Murray Nettles

 


 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 


 

 

 

 

 

 

 

 

DEDICATION

 

To my wife Silvia, and my parents: Alicia and Milton.

ACKNOWLEDGEMENTS

 

I would like to thank my advisor, Ben Bederson, for all his guidance, and for teaching me how to always take one extra step to make my work as good as it can be. I would like to thank Allison Druin for her transfers of energy to me and for showing me how to collaborate with children and people of different disciplines. I would also like to thank François Guimbretière, Atif Memon, and Sandra Murray Nettles for accepting to be part of my committee and for providing me with very helpful advice on writing and presenting my work.

I would like to thank the Human-Computer Interaction Lab’s graduate and undergraduate students I have worked with as well as the faculty and staff. A special thanks to Ben Shneiderman for taking the time to listen to my ideas and for giving me great advice and access to his library. And thanks to the children I have worked with at the HCIL, without whom many of the contributions described in this dissertation would not be a reality.

I would also like to acknowledge colleagues from other institutions who have collaborated with me in the past. In particular, I would like to thank my KidStory colleagues from the Royal Institute of Technology in Stockholm, Sweden, the Swedish Institute of Computer Science, and the University of Nottingham; and my colleagues at the Internet Archive.


 

TABLE OF CONTENTS

 

List of Tables. ix

List of Figures. xi

1.       Introduction. 1

1.1.        Potential Benefits of Children’s Use of Computers. 1

1.2.        Shortcomings of Children’s Use of Computers. 2

1.2.1.     Lack of Interaction with People. 3

1.2.2.     Lack of Support for Creative, Imaginative Experiences. 4

1.2.3.     Lack of Support for Large Curated Collections. 4

1.3.        Addressing These Shortcomings. 5

1.3.1.     MID.. 6

1.3.2.     KidPad. 8

1.3.3.     Digital Library User Interfaces. 10

1.3.4.     Study of Young Children’s Mouse Use. 12

1.4.        Design Process. 13

1.4.1.     Technology Immersion. 14

1.4.2.     Participatory Design. 14

1.4.3.     Contextual Inquiry. 15

1.4.4.     Adult Researcher Tasks. 15

1.4.5.     Disciplines. 17

1.4.6.     Design Roles. 18

1.4.7.     My Participation. 19

1.5.        Dissertation Organization. 21

2.       MID, A Single Display Groupware Architecture. 22

2.1.        Related Work. 23

2.1.1. MMM    23

2.1.2.     Colt 24

2.1.1.     Local Tools. 24

2.1.2.     Pebbles. 25

2.1.3.     ICON.. 26

2.1.4.     SDGToolkit 27

2.1.5.     The Context Toolkit 27

2.1.6.     BEACH.. 28

2.2.        MID Architecture. 28

2.1.1.     Event Manager 29

2.2.1.     Event Sources. 31

2.2.2.     MID Mouse Event Source. 32

2.2.3.     MID Socket Mouse Event Source. 33

2.2.4.     MID Switch Event Source. 33

2.2.5.     MID DiamondTouch Event Source. 34

2.3.        MID Versus Typical Java Event Sources. 35

2.1.1.     Differences in event source implementation. 35

2.3.1.     Differences for listeners. 36

2.4.        Support for Multiple Output Devices. 39

2.5.        Layers of MID.. 40

2.6.        Extending MID.. 41

2.7.        Limitations. 41

2.8.        Performance. 42

2.9.        Use. 43

2.10.      Lessons Learned. 43

2.11.      Recommendations for Operating Systems Designers. 44

3.       KidPad. 45

3.1.        Related Work. 48

3.1.1.     Single-Display Groupware User Interfaces. 48

3.1.2.     Other Systems Supporting Co-Present Collaboration. 53

3.1.3.     Studies of Children’s Use of Collaborative Software. 60

3.2.        Design Roles. 61

3.2.1.     Design Partners. 61

3.2.2.     Informants. 62

3.2.3.     Testers. 62

3.2.4.     Users. 62

3.3.        Local Tools. 63

3.3.1.     Design Evolution. 64

3.3.2.     Trade-offs of Local Tools. 67

3.4.        Toolbox Design. 68

3.5.        Arriving at KidPad’s Design. 70

3.6.        Observations of KidPad’s Use. 72

3.6.1.     Storytelling. 72

3.6.2.     Other Uses of KidPad. 73

3.7.        Collaboration. 74

3.7.1.     Programming Issues. 76

3.7.2.     Testing Issues. 77

3.8.        Creativity. 78

3.9.        Extensions to KidPad. 79

3.10.      Evaluations. 80

4.       Digital Library User Interfaces. 84

4.1.        Related Work. 84

4.1.1.     Designed for Children. 84

4.1.2.     Designed for Adults. 86

4.1.3.     Digital Books. 88

4.2.        SearchKids. 91

4.2.1.     Design Roles. 91

4.2.2.     Design Principles. 93

4.2.3.     Interface Description. 93

4.2.4.     Collaboration. 96

4.2.5.     Learning. 96

4.2.6.     Evaluations. 97

4.3.        International Children’s Digital Library (ICDL) 99

4.3.1.     Design Roles. 100

4.3.2.     Designing the ICDL. 101

4.3.3.     Information Retrieval Architecture. 115

4.3.4.     Informal Evaluation of Book Readers. 116

4.3.5.     Learning. 124

4.3.6.     Use. 125

5.       Study of Young Children’s Mouse Use. 126

5.1.        Introduction. 126

5.2.        Literature Review.. 127

5.2.1.     Information Processing Speed in Children. 127

5.2.2.     Fitts’ Law.. 130

5.2.3.     Fitts’ Law Applied to Children. 131

5.2.4.     Fitts’ Law Applied to Input Devices. 132

5.2.5.     Children and Input Devices. 134

5.3.        Study. 139

5.3.1.     Motivation. 139

5.3.2.     Research Questions. 139

5.3.3.     Participants. 140

5.3.4.     Materials. 141

5.3.4.     Procedure. 143

5.3.5.     Design. 144

5.3.6.     Results. 145

5.4.        Discussion. 163

5.4.1.     Relevance of Results. 163

5.4.2.     Implications of Results. 165

6.       Future Work. 170

6.1.        Supporting Collaboration. 170

6.2.        Digital Library User Interfaces. 170

6.2.1.     Text Searching. 170

6.2.2.     Community Features. 171

6.2.3.     Support for Reading Books over Several Sessions. 171

6.2.4.     Link to Creative Software. 172

6.3.        User Populations and Graphical User Interfaces. 172

7.       Conclusion. 174

References. 177

 


List of Tables

 

Table 1: Summary of adult researcher tasks. 15

Table 2: Disciplines participating in design teams. 17

Table 3: Ages of participating children. 118

Table 4: Preferred readers for all children, children under 10, adults, and reader adults thought helped children the most in understanding the story (two adults did not respond to this last question). 121

Table 5: Book preferences and average rating of readers by book. A sad face was assigned a rating of 0, a neutral face a rating of 1, and a happy face, a rating of 2. 121

Table 6: Number of adult observers that thought each of the readers was confusing, exciting or helpful 122

Table 7: Empirically derived data from four psychology studies of children’s performance in Fitts’ law tasks. 132

Table 8: Empirically derived constant b for six, eight, and ten year-olds from Jones’ study for a continuous task with square targets averaged over all input devices used Jones (1991). 135

Table 9: Comparison of improvement in performance with age between Jones’ Fitts’ law study (with input devices), two psychology studies, and predictions from Kail’s model. 136

Table 10: Result of repeated measures ANOVAs, looking at significant differences based on target size, age level, and distance to target (amplitude). 146

Table 11: Accuracy, target reentry, and target reentry during click for each age group and target size. The significant differences for age were obtained through Dunnett’s T3 test. The significant differences for target size were obtained through pairwise comparisons using Bonferroni’s correction. 147

Table 12: Fitts’ law correlation coefficient and constants a and b for movement time on first entering and last entering the target. 151

Table 13: Fitts’ law correlation coefficient and constants a and b for movement time on pressing the mouse button. 152

Table 14: Fitts’ law correlation coefficient and constants a and b for movement time on releasing the mouse button. 153

Table 15: Fitts’ Index of Performance (IP), in bits per second, based on movement time to first enter the target. Significant differences reported according to pairwise comparisons using Bonferroni’s correction. 162

    


List of Figures

 

Figure 1: Summary of shortcomings of technologies used by children. 5

Figure 2: Children using KidPad on MERL's DiamondTouch. 10

Figure 3: Summary of contributions. 13

Figure 4: Architecture of MID. 30

Figure 5: This code fragment shows Java code that gets standard Java mouse and mouse motion events. 37

Figure 6: This code fragment shows Java code using MID events to access multiple mice. 38

Figure 7: KidPad with all its tools visible. 47

Figure 8: KidPad’s bottommost toolbox. 68

Figure 9: KidPad’s second toolbox from bottom.. 69

Figure 10: KidPad’s Bulletin Board 70

Figure 11: On the left, a scene from a story by a 7 year-old girl and a 9 year-old boy showing a special chair. On the right, a scene from a story by an 11 year-old boy. 73

Figure 12: On the left, Microsoft Reader 2.0 with the riffle control open at the bottom of the window. On the right, Adobe Acrobat eBook Reader 2.2, showing its navigation bar at the bottom of the window. 90

Figure 13: From left to right: SearchKids’ initial screen, the zoo area, the world area, and the search area. 93

Figure 14: Search area when exploring biological taxonomy. 95

Figure 15: ICDL’s initial screen. The buttons on the top right of the screen (from left-to-right) give access to help, take users back one screen, take users back to the initial screen, and exit the application. 102

Figure 16: World area in ICDL with Africa highlighted by user. The arrow on the right side of the screen spins the globe. 103

Figure 17: PhotoMesa showing results of clicking on Africa in ICDL's world area. 104

Figure 18: Sample query in search area looking for bilingual book written in both English and Spanish that have an orange cover. 105

Figure 19: ICDL's search area showing top search categories. 108

Figure 20: ICDL’s search area showing grayed out icons due to query previews. The all Science and Nature icon is highlighted by the user and shows a tooltip with the full name of the icon. 109

Figure 21: ICDL’s results area. 110

Figure 22: ICDL’s book preview screen. 111

Figure 23: ICDL’s standard reader with labels explaining what the buttons do. 112

Figure 24: ICDL’s Comic strip reader. 113

Figure 25: Spiral reader. 114

Figure 26: ICDL's information retrieval architecture. The ICDL software may access metadata on books either from a local or a remote database. Book pages are also accessible locally or remotely. 116

Figure 27: Reader ratings by children. A sad face was assigned a rating of 0, a neutral face a rating of 1, and a happy face, a rating of 2. Error bars are two standard deviations in length. 122

Figure 28: Charts showing log data for time spent (in seconds), number of page changes, and number of changes of direction for each book read under each type of reader. The error bars are two standard deviations in length. 123

Figure 29: Plot of Kail’s model with RTadult = 1, and the values for b and c reported in (Kail, 1991) (b = 5.16, c = 0.21). 129

Figure 30: Screenshot of study software showing the yellow square representing the home area, the crosshair cursor, the target circle, and information on elapsed time and number of hits and misses on the bottom left. 142

Figure 31: Average accuracy for participants clicking on targets by target size and age level. Error bars are two standard deviations long. 148

Figure 32: Average number of times participants reentered target (not counting the first time they entered a target) by target size and age level. Error bars are two standard deviations long. 148

Figure 33: Average number of times participants entered a target while pressing the mouse button during a click by target size and age level. Error bars are two standard deviations long. 149

Figure 34: Summary of findings with respect to accuracy and target reentry. 150

Figure 35: Plot of time to first enter target versus index of difficulty for 4, 5 year olds and adults, including regression lines. 151

Figure 36: Plot of time of last entering target versus index of difficulty for 4, 5 year olds and adults, including regression lines. 152

Figure 37: Plot of time to press mouse button versus index of difficulty for 4, 5 year olds and adults, including regression lines. 153

Figure 38: Plot of time to release mouse button versus index of difficulty for 4, 5 year olds and adults, including regression lines. 154

Figure 39: Average movement time for the three age levels at each target size decomposed by the time it took to first reach the target, the time it took to press the mouse button, and the time it took to release it. 155

Figure 40: Plots of three participants' mouse motion towards a 32 pixel circular target 256 pixels away from the home position. Participant in (1) was a 4 year 6 month old female. Participant in (2) was a 5 year 8 month old female. Participant in (3) was a 21 year-old female. 156

Figure 41: All paths taken by 4 year old participants to click on a 32 pixel target at a distance of 256 pixels. 157

Figure 42: All paths taken by 5 year old participants to click on a 32 pixel target at a distance of 256 pixels. 158

Figure 43: All paths taken by adult participants to click on a 32 pixel target at a distance of 256 pixels. 159

Figure 44: Mouse paths by a 4 year 3 month old male; (1) when clicking on a 16 pixel target 256 pixels way; and (2) when clicking on a 64 pixel target 256 pixels away. 161

Figure 45: Summary of findings with respect to movement time and Fitts’ law. 162

Figure 46: Mouse button use by age. 163

 

 

 


1.       Introduction

1.1.         Potential Benefits of Children’s Use of Computers

Computers have the potential of offering a wealth of opportunities for children* to express themselves and to learn. As a creative medium, computers can help children express their imagination in ways not possible with other materials. For example, drawing on a computer can give children the ability to draw in any color they want, on a canvas as large as they want, where they can use special effects that would never be available on a piece of paper.

Connections to the Internet can give children the ability to access content far beyond what a school library can provide. Just imagine being able to read a one of a kind book that only Kings in a far away country had access to; or being able to see videos of that animal that so intrigued you when you read about it in a book; or having your favorite book read by its author.

Internet connections can also provide children the opportunity to interact with family, friends, or other children and teachers anywhere in the world. Classrooms in the United States and Brazil could connect to learn about each other’s cultures. Children could send the artwork they create in class to a parent in a humanitarian mission halfway around the world.

1.2.         Shortcomings of Children’s Use of Computers

These and other potentials have encouraged parents, school officials, and politicians to give children access to computers at home and in school. Yet, as children have gained access to computers, a group of researchers has raised doubts about the benefits computers can bring to children. They claim computers are not delivering the type of experiences children need while causing concern for children’s physical and mental health.

This group, the Alliance for Childhood, challenged the notion that schools should make computers available to children in preschool and elementary school. They argue that computers are too costly, do not give children the opportunity to interact with other people, do not provide creative outlets, do not seem to have an impact on academic achievement, and are often used in ways that may physically harm children. They conclude by recommending a moratorium on the further introduction of computers in preschools and elementary schools (Alliance for Childhood, 2001).

A closer look at these arguments suggests that the problem lies not with computers, but with the ways they are being used. School officials and parents have often rushed to give children access to computers without first understanding how they could improve children’s learning. Well informed teachers and parents are key to avoiding many of the problems attributed to computers including those related to cost, excessive time spent on computers, poor ergonomics, and accessing of inappropriate material. Equally important is that parents, teachers and school officials understand how to use computers to further children’s educations. Like other tools in the classroom, computers may help children, may harm them, or may provide them with entertainment without educational value. The key is for parents and teachers to consistently observe and evaluate how children are using computers.

However, parents, teachers and school officials cannot address all of the shortcomings of children’s use of computers. It is up to researchers and technology developers to provide children with technologies that take into account their needs and abilities. The Alliance for Childhood identifies three particular needs that have not been well addressed so far. These are computer’s current lack of support for: interaction with people, developmentally appropriate user interfaces to large curated collections, and creative, imaginative experiences (Alliance for Childhood, 2001).

1.2.1.   Lack of Interaction with People

There is a reason why the “P” in PC stands for “personal”. Desktop and laptop computers were designed to accommodate only one user. With one keyboard, and one mouse, only one user at a time may interact with a personal computer. The Alliance for Childhood worries that computers, like televisions, are becoming electronic babysitters (Alliance for Childhood, 2001). They have also complained about robotic baby dolls that may take away from children’s interactions with other children (Alliance for Childhood, 2000).

This organization also cites how research shows that interaction with parents, teachers and other children is critical to children’s development. They also mention that children learn by developing personal bonds, having face-to-face interactions, imitating, and finding their place in society. The most important point advanced is the significance of giving children rich human interactions. This is why the lack of interaction between users is one of the Alliance for Childhood’s chief complaints against computers (Alliance for Childhood 2000, Alliance for Childhood 2001). They are not the only ones to believe that collaboration is beneficial to learning, as many curriculum standards have set it as a priority (Chambers & Abrami, 1991; Cohen, 1994; Fulton, 1997; Johnson & Johnson, 1999; Lou, Abrami & d´Apollonia, 2001; Slavin, 1996). Supporting collaboration should therefore be seriously considered in the development of any technology designed for young children.

1.2.2.   Lack of Support for Creative, Imaginative Experiences

An additional issue, complicating the type of experiences children have with computers, concerns the scarcity of software that supports children’s abilities to be creative and imaginative. Early childhood is the time for storytelling, reading books aloud, and for children generating their own ideas. Currently, most educational software does not provide children with creative outlets. Instead, it provides preprogrammed scenarios and tells children what to do (Alliance for Childhood 2001). The support for creativity and imagination should therefore be a goal in the design of children’s software. Ideally, such software should provide a medium for children to express themselves in ways that would not be possible without a computer.

1.2.3.   Lack of Support for Large Curated Collections

Perhaps more alarming than the lack of support for creativity and imagination is the fact that computers often provide children with inappropriate experiences. Children can access violence and other inappropriate content by using the Internet. While filters may be able to block most violent and pornographic content, they cannot do much to ensure that children access valuable and accurate information, or that they avoid websites that bombard them with advertisements for toys and candy (Alliance for Childhood, 2001). There are many gems to be found on the Internet, but it has the disadvantages of an uncurated collection. On the other hand, curated media collections for children tend to be very limited in scope and content (Druin et al., 2001; Hourcade et al., in press). The lesson when designing systems for young children to access media is to provide them with a curated collection that is accessible through developmentally appropriate interfaces and is large enough to sustain children’s interests.

Lack of Interaction with People

  • Not designed to encourage interactions with people

Lack of Support for Creative, Imaginative Experiences

  • Preprogrammed scenarios and software that tells children what to do

Lack of Support for Large Curated Collections

  • Access to inappropriate, inaccurate or incomplete content over the Internet

Figure 1: Summary of shortcomings of technologies used by children.

1.3.         Addressing These Shortcomings

This dissertation addresses some of these concerns through technologies and studies designed to provide children with positive experiences through computers. Specifically, I have concentrated on addressing the lack of support for face-to-face interactions, the need for children to use their imagination and creativity, and the importance of accessing large curated collections. Addressing these issues demanded going from concepts, to building necessary technologies, to putting together applications, and conducting studies to learn from the process. The outcome is technologies, applications and guidelines aimed at supporting children’s collaboration, creativity and learning. The specific contributions are:

1.3.1.   MID

As an alternative to using computers, the Alliance for Childhood recommends creative activities that engage children with other children and adults. They are concerned that such activities are currently not available to children through computers. They want children to have the type of face-to-face interactions that researchers say are so important for their development. In particular, researchers recommend that children engage in art, storytelling, and other activities with other children and caring adults (Alliance for Childhood, 2001). An example would be two children painting on the same canvas.

Computers will never be a replacement for such activities, but can provide alternative ways for children to express their creativity and imagination. For example, two children could paint on a magical canvas where they could make their drawings come alive and use x-rays to reveal secret drawings. Supporting such an experience requires computers to give children the ability to work together to author content.

I addressed this requirement by using the single display groupware model in which multiple users work on the same computer, each controlling an input device (Stewart, 1999). This model supports multiple children authoring content together. For example, each child could use a mouse or some other input device to draw, and they could all paint on the same canvas, which could be a computer screen or a projection. One can think of single display groupware as being equivalent to providing each child with a paintbrush. The traditional model on the other hand, with one input device per computer, is equivalent to multiple children sharing one paintbrush.

After selecting the single display groupware model, I then faced the issue of how to write an application that could take input from multiple devices. Operating systems and programming languages provide inadequate support for writing such applications. At this point, I had three options: to give up, to write some quick code that would only work for a particular application and input device, or to write a general-purpose toolkit that would support applications getting input from multiple devices in a consistent manner. I chose the third option because I thought it could help me with future research and also help other researchers interested in single display groupware. The outcome of my effort was MID (Multiple Input Devices), a Java toolkit that I have used in two applications and which currently supports three different types of input devices (Hourcade & Bederson, 1999). MID makes it easy for programmers to access input from multiple devices, and has also been used by researchers at several of the universities involved in single display groupware research.

1.3.2.   KidPad

Building MID enabled me to pursue developing applications that support collaboration. It gave each child a paintbrush, now I had to build a magic canvas. The challenge was finding an activity children could pursue together on a computer that would encourage them to be creative and use their imagination. In addition, the activity had to give children the ability to express themselves in ways not possible without computers. This is in contrast to the type of activities criticized by some researchers; those that consist of preprogrammed scenarios and tell children what to do (Alliance for Childhood, 2001).

Working on the KidStory project, the activity of choice was collaborative storytelling, and the application I built was KidPad. KidPad gives children drawing, typing and linking capabilities in a large two-dimensional canvas (Benford et al., 2000; Hourcade, Bederson, Druin & Taxen, 2002). KidPad supports collaboration through MID by taking input from multiple devices, including MERL’s DiamondTouch (Dietz & Leigh, 2001). It gives children the ability to be creative and use their imagination together with their friends, classmates, parents or teachers. It does so by giving children a tool where they can express themselves in ways not possible without computers by zooming around a large canvas, creating spatial hyperlinks, making their drawing “come alive”, and hiding drawings that can only be seen through “x-rays”. These are the types of experiences children are often lacking with current software.

KidPad has been used as part of the curriculum at elementary schools in Sweden and in England, where a teacher won a regional award for her use of KidPad in the classroom. KidPad has also been available online where hundreds of people have downloaded it.

Evaluations of KidPad have confirmed its support for creativity and its correct approach towards supporting collaboration. In one study, children interpreted a story without words in one of three formats: book, traditional hypertext (i.e. scene by scene on a computer), and KidPad (with animated transitions between scenes). Children who saw the story in KidPad told more elaborate stories, suggesting KidPad helped children in being creative and using their imagination (Boltman, Druin, Bederson, Hourcade & Stanton, 2002).

In one study on collaboration, pairs of children were asked to create stories together sharing one mouse, or using two mice. Pairs of children with at least one boy were judged to have created better stories when using two mice (Abnett, Stanton, Neale & O’Malley, 2001). This suggests KidPad’s approach to supporting collaboration helps children in being more creative. Further analysis on the same study looked at the distribution of mouse interactions. When using two mice, the amount of interaction with the software was almost equal between children, while the opposite was true when they had to share one mouse (Stanton, Neale & Bayon, 2002). This suggests KidPad’s support for multiple input devices can help children have more equal interactions with their peers.

Figure 2: Children using KidPad on MERL's DiamondTouch.

1.3.3.   Digital Library User Interfaces

One of the great potentials of bringing computers to schools is to give children access to information and knowledge beyond what local libraries can provide. As mentioned earlier, the Internet poses some issues in this respect for preschool and elementary school children due to inappropriate, inaccurate or incomplete content. Other multimedia collections with curated content are often too limited to provide a valuable benefit (Druin et al., 2001; Hourcade et al., in press). Even if children could have access to a large, curated collection, the challenge is how to provide them with developmentally appropriate searching and browsing capabilities.

I began to address this problem by developing SearchKids, an application that enables children to access multimedia information about animals (Druin et al., 2001). SearchKids enables children to search and browse this collection through a developmentally appropriate user interface that requires no typing, little reading, and no knowledge of Boolean logic. SearchKids also helped me explore other aspects of collaborative software as I wrote it to support two different modes of collaboration: independent and confirmation. Under confirmation collaboration, both users need to agree on what item to interact with, while this is not necessary under independent collaboration. A study of SearchKids suggested that independent collaboration helps children concentrate on tasks, while confirmation collaboration is better for learning how to navigate software (Druin et al., in press).

SearchKids provided the basis for the development of the International Children’s Digital Library (ICDL), an application with a goal of making 10,000 international children’s books available online (Hourcade et al., in press). I was responsible for developing the ICDL’s user interface and information retrieval architecture until its launch in November of 2002. The ICDL currently gives children the ability to search, browse and read a collection of about 200 books in 20 languages. Providing children access to specially selected books from all over the world will likely give children more chances of finding books that will capture their imagination and give them creative ideas. In addition, the fact that children search the collections themselves instead of being fed information also encourages creativity when it comes to posing searches and finding books of interest.

An evaluation of the search interface in SearchKids (which is basically the same as that in the ICDL) showed that 2nd and 3rd grade students were highly successful in posing queries (Revelle et al., 2002). Early results from an additional study show that when comparing the same interface to a traditional text search alternative, it was better for about a third of 2nd graders and about a fourth of 3rd graders. This suggests, as expected, that the interface’s features avoiding the need to type and spell favor younger children.

1.3.4.   Study of Young Children’s Mouse Use

For children to be creative and to be able to access media through computers, they need to interact with interfaces that take into account their abilities. Interfaces that ignore this can cause children to become frustrated and feel like they do not have control of the technology; situations that can hardly encourage creativity and imagination. I noticed this frustration as I brought an early version of SearchKids to a preschool. Many of the children had difficulty clicking on icons because they were too small.

This and other experiences led me to develop some guidelines for the design of graphical user interfaces, which included the use of larger icons for younger children, using point-and-click versus drag-and-drop, and giving all mouse buttons the same functionality. While I found studies supporting the use of point-and-click over drag-and-drop for children (Inkpen, 2001; Joiner, Messer, Light & Littleton, 1998), I did not find studies providing guidelines for sizing visual targets, or providing evidence of children’s use of mouse buttons.

This need prompted me to conduct a study of 4 and 5 year old children’s performance with mice. In the study, I compared the children’s performance in point-and-click tasks to that of 19-22 year old adults in terms of accuracy, target reentry, and speed. The study enabled me to gain a better understanding of how children’s performance evolves, and what makes this type of tasks challenging for young children. It also enabled me to develop guidelines for sizing visual targets in software for children of that age.

MID

  • Toolkit provides consistent, convenient access to input from multiple devices

KidPad

  • Collaborative storytelling tool

Digital Library User Interfaces

  • SearchKids and International Children’s Digital Library

Study of Young Children’s Mouse Use

  • Compared performance of 4 and 5 year old children with adults in point-and-click tasks

Figure 3: Summary of contributions.

1.4.         Design Process

While I played a critical role in conducting the above-mentioned research, I did so as part of interdisciplinary, intergenerational teams. These teams consisted of children and adult researchers from various disciplines. Most of the design activities occurred at the Human-Computer Interaction Laboratory at the University of Maryland, where the adult and child researchers meet twice a week for one and a half hours during the school year. There are six to eight children in the group, and they are between 6 and 11 years old. This group works on a variety of projects to design children’s technologies. Some of the group members change on an annual basis. To work together, the teams of adults and children follow the Cooperative Inquiry strategy, explained in Druin (1999). This strategy consists of a combination of techniques from technology immersion (Druin et al., 1999), participatory design (Schuler & Namioka, 1993), and contextual inquiry (Beyer & Holtzblatt, 1998). Druin adapted these techniques specifically for children and adults to create new technologies together.

1.4.1.   Technology Immersion

Each of the teams’ sessions usually concentrates on one of the three techniques mentioned above. The process of creating a new technology often begins with technology immersion sessions. In these sessions, team members use and experience the relevant technology for the first time. The sessions are meant to make children and non-technically oriented researchers aware of the capabilities of these technologies. Since the projects I participated in produced software for use on regular computers, these sessions were not as important as in other projects that use technologies children are not familiar with. Nevertheless, making the children aware of zoomable user interfaces and the potential of multiple input devices enabled them to design with those building blocks.

1.4.2.   Participatory Design

While technology immersion sessions provide awareness of existing technologies, participatory design sessions help design and enhance new ones. Teams can use these sessions when they have a specific question about how to add functionality to technology, or how to approach a particular problem. In these sessions, a team member communicates a research question to the team, which then divides into groups, including both children and adults. In these groups, the team members discuss the problem and express their solutions using low-tech prototyping techniques. For example, teams working on software draw on large pieces of paper using crayons, markers, and other art supplies. After completing prototypes, the groups typically share their ideas with the whole team by writing in journals about the experience. A common misconception of the way teams at the Human-Computer Interaction Laboratory use participatory design is that the children make the design decisions and have the last word. This is not the case, as the design outcomes of these sessions are the product of ideas that come from children and adults working together.

1.4.3.   Contextual Inquiry

In contextual inquiry sessions, some children and adults use technologies while other adults and children observe and take notes. Towards the end of these sessions the teams often use sticky notes to express what members like, dislike, and would like to change. These “sticky-note sessions” often inform the basis of the next designs.

1.4.4.   Adult Researcher Tasks

In order to organize and conduct the design sessions, the adult members of the teams have to fulfill certain tasks that go beyond participating with children in the Cooperative Inquiry activities. Table 1 summarizes these tasks.

Adult Researcher Tasks

Facilitating

Asking Research Questions

Documenting

- lead design sessions

- motivate participants

- organize structure of sessions

- communicate to participants issues to address during session

- write in journals

- record video

- take photographs

Table 1: Summary of adult researcher tasks

1.4.4.1.        Facilitating

Facilitating is perhaps the most important task of all. It involves leading design sessions, motivating the children and the adults to participate, dividing them into groups if necessary, and ensuring that the research objective for the session is accomplished. The main skill necessary for this task is to be able to motivate and organize children and adult researchers.

1.4.4.2.        Asking Research Questions

The person who facilitates frequently also asks the team the research questions it needs to work on during the session. Asking research questions involves knowing where a project is headed, and what parts of a technology need design help, or need to be evaluated. One can think of facilitating as providing the syntax of a session, while asking research questions provides the semantics.

1.4.4.3.        Documenting

The best design session is not very useful if it is not documented. Documenting design sessions occurs during and after the session. During the session, researchers (both adult and children) regularly videotape and take notes. After a session, adults often get together and write down further notes, summarizing what the team learned from the session.  In addition, team members frequently take digital pictures of sketches and low-tech prototypes.

1.4.5.   Disciplines

As mentioned earlier, the adult researchers in the teams in which I participate come from a variety of disciplines. Some of the researchers are concerned with building technology. Others are experts in the target user population (i.e. children). A third group has expertise in a domain related to the project in question. Table 2 provides a summary of these disciplines.

Disciplines

Builders

User Population Experts

Domain Experts

-build technologies

-computer science, visual design, industrial engineering, etc.

- experts in children

- human development, child psychology

- experts in the matter the technology addresses (e.g. librarians for digital collections of books)

Table 2: Disciplines participating in design teams.

1.4.5.1.        Builders

Builders are the researchers who are actually in charge of building the technologies. They can come from a variety of fields, such as computer science, industrial engineering, electrical engineering, and visual design. In the projects I participated in, these researchers have come from two areas: computer science, and visual design.

Computer scientists develop the software and hence are an absolutely necessary part of the team. Besides programming, they actively participate in the design sessions, and decisions on the look and feel of the software.

Visual design experts contribute mostly by creating the art used in the software and by helping design the software’s look and feel. Because of children’s needs the software the teams developed is highly visual and hence requires expertise in this area.

1.4.5.2.        User Population Experts (Human Development)

Experts in human development and child psychology have also participated in every project. The reason is that they are experts in the target population for the technologies developed by the teams. They often lead the design sessions, plan studies, and make sure the team is aware of children’s developmental issues. While they are not necessarily researchers, elementary school teachers have also participated in every project. Children could also be considered experts at being children and could therefore fall under this category.

1.4.5.3.        Domain Experts

The teams have added experts for each project depending on the project’s domain. In building KidPad, which supports children’s storytelling, the team added professional storytellers. In making SearchKids, which gives access to multimedia on animals, the team added a biologist. In developing the International Children’s Digital Library, the team has counted with the contributions of several librarians.

1.4.6.   Design Roles

People outside the teams have also participated in developing the software. Druin (2002a) defines four roles people can play in the design of technology: design partner, informant, tester, and user. Everyone in the teams participated in the projects as a design partner, being closely involved in technology design decisions at every point in the development process. Informants have helped by providing more extensive feedback on technologies at key points in the development process. In developing SearchKids, child informants helped by participating in studies conducted to validate design decisions made at the lab with the smaller group of children. Testers have helped by using software and sending reports. For example, in the development of the International Children’s Digital Library, librarians at several locations throughout the United States have used the software and reported problems and good experiences back to the team. Users have participated by using the software and being observed. For example, during the development of KidPad, I observed children using KidPad while at a museum in Sweden.

1.4.7.   My Participation

Being part of the design teams, I always participated in the projects I worked on as a design partner.  However, my roles during design sessions evolved with time, and I also gained experience that helped me contribute through more than one discipline. When I began designing technologies for children I was a computer scientist with no experience working with children. Over four years, I emerged as a researcher who can facilitate and ask research questions during design sessions, and contribute to projects not only as a computer scientist, but also in terms of design and as an expert in the needs of children.

When I first started working with children while developing KidPad, my participation in the project was mostly as a builder, programming KidPad. While I took part in design sessions and would work on documenting them, I did not ask the research questions and did not facilitate the sessions. After working with design teams for about a year, when I was still working on developing KidPad and had started developing SearchKids, I began to take a more active role in the design sessions, sometimes asking the research questions. Even though facilitating design sessions seems to be a simple job, it requires a great deal of experience to be able to motivate, lead and communicate with both children and adult researchers. I did not start facilitating sessions until after three years of working in design teams. I still find it a challenge, as it is sometimes difficult to motivate children who are tired after a long day.  A further challenge when assigning children and adult researchers to groups is to find the combinations that will yield the best ideas. Lately though, I ask research questions in most of the sessions in which I participate, and I sometimes also facilitate them.

I have also seen an evolution in my contribution to the projects in terms of discipline. I have contributed to every project through my computer science skills, building the software. As I gained experience in visual design by taking classes and working with visual designers, I started participating in discussions on the visual design of the software I was building. While I did not contribute with art, I did give ideas on the look and feel of the software. Working with children for the past four years has also taught me about how they use technology. In addition, this past year, I conducted a study on children’s use of mice. The knowledge I gained from these experiences has enabled me to contribute to the projects as an expert in how children use technology (i.e. an expert in the user population).

While the change in my contributions to projects is the outcome of my personal experience, I expect that others who join similar teams undergo similar processes. In the following section, I describe the technologies I have developed, explaining my role in creating them, and relating the participation of others.

1.5.         Dissertation Organization

Section 2 describes MID, which I wrote in order to support input from multiple devices in KidPad. Section 3 explains the research involved in writing and designing KidPad. Section 4 covers my contributions to the development of SearchKids and the ICDL. Section 5 motivates and describes the study I conducted to gain a better understanding of young children’s use of mice. Finally, Section 7 summarizes the contributions of this dissertation.

 

 


2.       MID, A Single Display Groupware Architecture

Children often prefer to pursue activities together rather than alone. This includes using computers. Sometimes children are forced to share computers in schools due to the number of computers available. This may actually have a positive impact as psychologists and educational researchers have found that working in small groups can have positive effects on young children’s learning and development (Rogoff, 1990; Topping, 1992; Wood & O’Malley, 1996).

Children have traditionally collaborated on computers by sharing input devices. This type of collaboration is analogous to two children trying to draw something together with only one pencil.

Single Display Groupware (SDG) is a model for supporting co-present collaborative work.  An SDG application makes it possible for co-present users to collaborate using a single computer and display through the use of multiple input devices (Stewart, 1999). It is analogous to two children drawing something together where each child gets to use his/her own pencil. SDG provides a simple and cost-effective way for children to collaborate on a computer.  An extra mouse is often all the additional hardware needed by an SDG application.

Regretfully, operating systems and programming languages provide little or no support for getting input from multiple devices. This makes it difficult to develop SDG applications.

MID (Hourcade & Bederson, 1999), an architecture I developed, supports the development of SDG applications by providing developers with well-defined, consistent access to input from multiple devices. KidPad and SearchKids, two applications I developed, use MID.

2.1.  Related Work

Architectures for SDG have generally been developed by researchers to support particular applications. Early on, these architectures were not available to the research community, as in the case of MMM. By the late 1990’s some researchers made their architectures available. These architectures, such as Colt, Local Tools, and Pebbles were not designed to support a wide range of SDG applications. They also did not support a wide range of devices. I developed MID because I was looking for an architecture that would provide consistent access to input from multiple devices without being tied to a particular type of device, user interface metaphor, or operating system. The architectures released after MID, ICON and SDGToolkit, follow a similar approach, providing support for a broader set of applications and devices.

2.1.1. MMM

MMM was an early SDG system developed at Xerox PARC in the early 1990’s that supported input from multiple mice. There is not much information available on the architecture used for MMM, although it is apparent that its creators did not design it to be used with a wide range of applications (Bier & Freeman, 1991).  For example, it was written in the Cedar programming environment from Xerox PARC.

The architecture uses a system queue that receives event records that include a way of identifying the device that generated the event.  A process removes event records from the system queue and matches devices with users on a table.  It then proceeds to ask each of the “editors” in the system whether they are interested in the event.  These “editors” are the user interface elements in MMM.

The limitations of this architecture are that it is tied to a particular user interface metaphor and its potential use with other applications is very limited since it was not made available to the research community and apparently was not designed for this purpose.

2.1.2.   Colt

Lauren Bricker developed the Colt software toolkit as part of her Ph.D. thesis (Bricker, 1998).  The goal of Colt is to facilitate the creation of collaborative applications, in particular those dealing with Cooperatively Controlled Objects (CCOs).  Bricker defines controlled objects to be objects (in the object-oriented sense) whose properties may be manipulated by one or more users (Bricker, Baker, Fujioka & Tanimoto, 1998).

Colt’s support for multiple input devices is limited although it could clearly support any number of devices if the code to get the input from the devices was available.  Its main limitation is that it only supports devices that produce mouse-like data, and would have to be modified in order to handle input from other types of devices.

2.1.1.   Local Tools

Jason Stewart built the Local Tools architecture to support the development of an early version of KidPad, at the University of New Mexico. Stewart chose to develop KidPad and Local Tools on UNIX platforms (Stewart, 1999). 

The Local Tools architecture could accept up to three input devices supported by the XInput library (which supports mice, joysticks and tablets).  It was only tested with mice and tablets, and due to some bugs in the XInput library, it worked well only with mice.

The Local Tools architecture was highly tied to several technologies that are not very common in homes, schools or businesses.  It was based on Linux, XInput, a modified version of Tk, Pad++, and Perl.  The only implementation independent API was at the Perl level and it was tied to the use of local tools as a user interface metaphor.  It clearly was not the intention of the author to create an architecture that could support a wide variety of SDG applications.  The most important contribution of this architecture is the idea that programmers should be shielded from low-level details of how input from multiple devices is obtained.

2.1.2.   Pebbles

Brad Myers’ Pebbles project concentrates on the creation of SDG applications for personal digital assistants (PDAs).  The architecture behind the project supports multiple 3Com PalmPilots and Windows CE devices under Windows (Myers, Stiel & Gargiulo, 1998).

Similarly to the Local Tools architecture, this architecture ties user interface metaphors in the Amulet toolkit to input from multiple devices.  However, events from multiple devices are accessible at a lower level without the need of using Amulet. 

Amulet provides a set of features that make it easier to program multi-user applications using a standard user interface (with scroll-bars, menus, and palettes).  Amulet has widgets that only one person may use at a time, as well as widgets that can be used by more than one person at a time.

Amulet supports hierarchical events, also referred to as hierarchical command objects.  This type of events supports the development of applications in layers, where a group of low-level events can generate higher-level events (Myers & Kosbie, 1996).

No details are given on how a programmer would access input from multiple devices without going through Amulet.  Going through Amulet means having to code using Interactors and Widgets, a metaphor that programmers may not want to use.  This approach lacks support for a broader range of applications and ties the programmer to approach SDG applications in a certain way. Another limitation of Amulet is that it has not been adapted for use with input devices different from PDAs, making it unclear if this would be possible or not.

2.1.3.   ICON

ICON, by Dragicevic and Fekete (2001) is an editor designed to match input devices with actions on a graphical user interface. It is meant to bring the management of multiple input devices to a wider audience, rather than just programmers. As such, it is a programming layer above MID, and could use MID in order to obtain input from multiple devices. ICON gives users the ability to connect input devices to input processors which in turn connect to visual widgets. Programmers need to write an ICON module for each input device, input process and graphical widget to be used in ICON.

ICON could be used to support SDG applications by connecting modules for multiple input devices with modules for graphical widgets designed to work in an SDG environment. The main limitation of ICON is given by the need to develop ICON modules. ICON also forces its users to use a particular model for processing and using input from multiple devices. ICON is an example of the type of architectures developed after MID, which provide more flexibility in terms of the types of devices they support.

2.1.4.   SDGToolkit

SDGToolkit (Tse & Greenberg, 2002) gives programmers access to multiple mice and keyboards in platforms compatible with Microsoft’s .NET. It was developed by Saul Greenberg’s group at the University of Calgary, and resembles MID in many ways, most likely due to that group’s use of MID in the past. In providing access to multiple input devices of different kinds, SDGToolkit gives .NET platforms what MID offers for Java platforms.

2.1.5.   The Context Toolkit

The Context Toolkit (Salber, Dey & Aboud, 1999), developed at Georgia Tech by Dey and others, is meant to facilitate the development of context aware applications.  It does so by providing context information to applications while hiding the details of how it is obtained.  It also provides an XML based distribution model for the context events.  While the context toolkit was not built to support SDG applications, its architecture is similar to MID’s which suggests it could be used for this purpose. For example, it could “distribute” input from multiple mice to a particular computer. However, the fact that it uses XML messages to distribute events suggests that it may introduce unacceptable latency.

2.1.6.   BEACH

BEACH (Tandler, 2000), developed by Tandler and others at the German National Research Center for Information Technology (GMD), is a software infrastructure designed to support collaboration in a room environment in office settings. Like the Context Toolkit, BEACH distributes context events for context aware applications. It also supports group work by handling the distribution of objects between users’ information appliances. While BEACH was not designed for SDG applications, like the Context Toolkit, it is possible it could be adapted for this purpose.

2.2.         MID Architecture

As mentioned earlier, a major difficulty in writing SDG applications is getting input from multiple devices. I built MID, a Java package that addresses this problem and offers an architecture to access advanced events through Java.  MID supports input from multiple devices of the same type and devices not supported by Java.  In the following sections, I describe the features, architecture and limitations of MID.

MID provides developers using Java the ability to write SDG applications with a powerful yet straightforward and cross-platform way of getting input from multiple devices.  MID is not tied to any user interface metaphor and was not designed to receive input from a particular type of device. These features address the issues with previously designed architectures that were often confined to specific platforms, tied to a user interface metaphor, or to a particular type of device.

I wrote MID in Java because of platform independence.  MID consists of a cross-platform Java layer, and a native platform-specific layer.  The cross-platform layer consists of an event manager and of event sources that have to be written for each device type (one event source per device type).  A native class must be implemented on each platform for each device type for which data cannot be accessed through Java.  Applications using MID use just the cross-platform Java layer, and therefore do not have to be changed for use on different platforms.

So far, device-specific event sources have been written for USB mice under Windows 98 and ME, mouse data over a socket, Radio Frequency ID (RFID) tag readers, and the DiamondTouch (Dietz & Leigh, 2001).

MID consists of an event manager that is a hub for all MID events, and event sources (one per device type) that generate events the event manager delivers to appropriate listeners.  Figure 4 gives a visual overview of MID’s structure. 

2.1.1.   Event Manager

The event manager handles the registration of listeners and the broadcasting of events to these listeners. It receives these events from event sources. The event manager has no knowledge of any of the device specific event sources and will therefore work with any event source written in the future.  It is designed to make it as easy as possible for developers to add support for types of devices Java doesn't support. 

Event sources have to specify to the event manager what type of events they support, what listener interfaces these events may be sent to, and what devices they have available. Each device may generate multiple types of events, and there may be multiple devices for each device type.  Each event type may be sent to multiple listener interfaces.

When the event manager receives an event from an event source, it first posts it to the Java event queue.  When the Java event queue gives the event manager the possibility of coalescing the event, the event manager delegates this task to the event source that generated the event.  When the event is ready to be broadcast, the event manager sends it to all the appropriate listeners.


MID's event manager also supports the dynamic addition and removal of input devices by being a source of device change events.  Objects that register to listen to this type of event are notified when a device is added or removed.

Figure 4: Architecture of MID.

2.2.1.   Event Sources

Event sources are responsible for providing all the needed device-specific information for the type of device they support.  Their basic responsibility is to know when events should be generated.  They may also provide utility methods related to the particular type of the device they support.

Event sources work together with listener interfaces and event classes written for the particular type of device the event source deals with. Event sources communicate with native code in order to know when to generate events only if they cannot get data from the devices through Java.  I studied two different ways of communicating with native code.  One way is to have the native code in a library and use the Java Native Interface to communicate with it.  The other way is to use sockets for communication with an event server written in a separate process (which could be Java or native code, and could even come from another machine).  When event sources generate events they send them to the event manager, which is responsible for delivering them to the appropriate listeners.

Event sources are responsible of informing the event manager of the event types and listener interfaces they support as well as of the devices they have available.  They are also responsible of notifying the event manager when devices are added or removed. Event sources have the responsibility of deciding when to coalesce events.

If it makes sense for the type of devices they support, I recommend that event sources revert to some kind of default behavior if these devices are unavailable.  This behavior should be coded solely in Java.  Applications that follow this recommendation will work in some limited way when data from the devices is not available.

2.2.2.   MID Mouse Event Source

The MID mouse event source was the first event source implemented for MID.  It generates events from USB mice under Windows 98 and ME.  It reverts to Java mouse events if multiple mice are not available. This ensures that applications that use this event source will always work, although possibly with only one mouse.

The MID mouse event source communicates with a native code library through the Java Native Interface in order to know when to generate events.  The native code uses the Microsoft DirectInput API to get input from USB mice.

The MID mouse event source allows developers to set the location of mice, apply motion constraints, and the time span used for counting mouse clicks.  Setting the location of mice is particularly useful for an application that has to define its own cursors and needs to give them an appropriate initial location on the screen.  Constraining the motion of all mice and/or the motion of particular mice can be used to keep the cursor within the visible area of the window.  Another application would be to divide the screen into regions and have only one mouse operate in each region.

Another important feature of the MID mouse event source is that it enables applications to access all of the input buttons and movement axes on the particular mouse in use.  Thus, a mouse with a wheel generates a third button event and motion in the z-axis.

When supporting a mouse event's access to the keyboard's control, shift, and alt modifier keys, this event source currently assumes that there is only one keyboard available.  This access to the keyboard through mouse events demonstrates the low level at which the assumption of a single mouse and keyboard has been made. 

The implementation of event coalescing in this event source coalesces move and drag events if they come from the same mouse.  This means that events can be combined if too many pile up on the event queue.

2.2.3.   MID Socket Mouse Event Source

The MID socket mouse event source provides support for mouse events received over a TCP/IP socket connection. These events are generated by a program running in a process separate from MID and are sent across a socket. The MID socket mouse event source receives the events, interprets them, and generates corresponding MID mouse events that are passed on to the MID event manager. The default data this source expects to receive through the socket is Java mouse events.  However, this event source may be subclassed to provide other interpretations of the data received through the socket.

A colleague used this event source in an experimental program, which tracked a physical object with a camera using computer vision techniques.  The image processing program generated MID socket mouse events based on the position of the physical object, and the primary application worked as if a regular mouse was driving it.

2.2.4.   MID Switch Event Source

The MID switch event source, created by colleagues in the KidStory project, generates switch events for a Radio Frequency ID (RFID) Tag reader. The reader connects to one of the serial ports on a PC and each time a tag is placed on the reader it starts outputting IDs of that tag on the serial line until the tag is removed.

A switch event tells the user when a particular switch is opened or closed.  This is an analogy for the tag reader as the user is only interested in the change of mode, not continuous information of which tag is on the reader. The MID switch event source generates switch events for the tag reader by monitoring the serial port the reader is connected to. Each time a tag is introduced into the reading field of the reader a switch closed event is sent to the MID event manager. When a tag ceases to be detected through the serial port, a switch opened event is generated.

Some colleagues used this event source in StoryDice, a storytelling application developed in cooperation with children. The application lets the user record sounds on different sides of dice. These recorded sounds can then be played back. Each side of the dice has a tag, and when a side of the dice is placed on the reader, sounds can be recorded or played back.

2.2.5.   MID DiamondTouch Event Source

DiamondTouch (Dietz & Leigh, 2001) is an input device developed at Mitsubishi Electric Research Laboratories that supports multiple simultaneous users interacting with it.  It provides a touch-surface that can be used in combination with a tabletop display.  The technological advantage of DiamondTouch is that it is able to recognize multiple simultaneous users as they touch its surface.  This is a great improvement over what could be accomplished by a large touchscreen.

Working with an early beta version of a DiamondTouch, I wrote a MID event source that generates MID mouse events. I easily integrated this event source with KidPad and had a few successful sessions with children where they got to interact with KidPad through a DiamondTouch.

2.3.         MID Versus Typical Java Event Sources

MID follows the Java event model, having event objects sent from event sources to event listeners. Listener interfaces and event classes written for MID are comparable to those written for typical Java event sources. However, MID event sources are simpler to implement than typical Java event sources. On the other hand, the fact that MID supports multiple devices has the side effect of requiring listeners to register to listen in a slightly different way than they would with typical Java event sources.

2.1.1.   Differences in event source implementation

MID event sources have to inform the MID event manager of the event types and listener interfaces they support as well as the devices they have available.

Typical Java event sources have to manage the registration of listeners and the broadcasting of events; something MID event sources don’t need to do.  This requires the implementation of multicasters.  Typical Java event sources that deal with only one device can manage with one multicaster. When dealing with multiple devices, the task becomes more complicated as separate multicasters have to be kept for each device, with an extra multicaster for listeners that want to listen to all devices.  Typical Java event sources also need to implement methods listeners use to register to listen.

2.3.1.   Differences for listeners

While reducing the amount of coding required for event sources, MID also provides users with a consistent way of accessing events from multiple devices.  This has the cost of making registering to listen to events a bit different than it would be for typical Java event sources.

As an example, let us examine the differences between using Java mouse events and MID mouse events. A class that receives Java mouse events has to register with a component (usually the visible window where the events will occur) to get those events.

The class also has to implement the methods specified in the MouseListener and MouseMotionListener interfaces.  These methods will be called when mouse events occur, and a MouseEvent object describing the event will be passed to them.  Figure 5 shows sample code of a class that uses Java mouse events.

Figure 6 shows sample code of a class that uses MID mouse events. The most noticeable difference between this code and the code that uses Java mouse events is two lines of code that get a MID mouse event source (MIDMouseEventSource) object and a MID event manager (MIDEventManager) object.

Instead of registering with a component, the class has to register with the MID event manager to listen to MID events. The method call used for registering to listen to events is the same for any kind of listener because the listener class is passed as a parameter. 

Text Box: // Java code using standard Java events to access mouse and mouse motion events
//
class MyClass implements MouseListener, MouseMotionListener {
        // Constructor adds mouse and mouse motion listeners
    public MyClass(Component myComponent) {
        . . .
        myComponent.addMouseListener(this);
        myComponent.addMouseMotionListener(this);
        . . .
    }

        // Mouse listener events
    public void mousePressed(MouseEvent e) {
        . . .
    }
    public void mouseReleased(MouseEvent e) {
        . . .
    }
    . . .

        // Mouse motion listener events
    public void mouseMoved(MouseEvent e) {
        . . .
    }
    public void mouseDragged(MouseEvent e) {
        . . .
    }
}
The MID event manager offers the flexibility of registering to listen to all devices or to a particular device.  This supports applications written in two styles.  The first style dispatches events from all devices to each listener.  It is up to the listener to determine which device generated the event by calling a method on the event object that returns the ID of the device that generated the event.

Figure 5: This code fragment shows Java code that gets standard Java mouse and mouse motion events.

An alternative application style is to register to receive events from a specific device.  Then, an application would write one listener for each device, which would support decoupling of the devices.

Text Box: // Java code using MID events to access mouse and mouse motion events 
// from multiple USB mice.
//
class MyClass implements MIDMouseListener, MIDMouseMotionListener {
    int numberOfDevices;  // Number of devices available through MID
    public MyClass(Component myComponent) {
        . . .
MIDMouseEventSource midSource = MIDMouseEventSource.getInstance(myComponent);
MIDEventManager midManager = MIDEventManager.getInstance();
numberOfDevices = midManager.getNumberOfDevices();
midManager.addListener(MIDMouseListener.class, this);
midManager.addListener(MIDMouseMotionListener.class, this);
        . . .
    }

        // Mouse listener events
    public void mousePressed(MIDMouseEvent e) {
       int device = e.getDeviceID();       // Use mouse ID in application-specific manner
        . . .
    }
    public void mouseReleased(MIDMouseEvent e) {
int device = e.getDeviceID();       // Use mouse ID in application-specific manner
        . . .
    }
    . . .

        // Mouse motion listener events
    public void mouseMoved(MIDMouseEvent e) {	
int device = e.getDeviceID();       // Use mouse ID in application-specific manner
        . . .
    }
    public void mouseDragged(MIDMouseEvent e) {
int device = e.getDeviceID();       // Use mouse ID in application-specific manner
        . . .
    }
}
MID also allows listeners to register to listen to events from a particular event source.  This is useful for applications that want to react differently to events of the same type depending on what event source generated them.  In these cases, applications can implement one listener per event source to support decoupling of the event sources.

Figure 6: This code fragment shows Java code using MID events to access multiple mice.

In addition, there is a call to the MID event manager that returns the number of devices currently available.  This turns out to be necessary for most applications so they can build internal data structures and mechanisms to support the available input devices. When more than one type of device is available through MID, the application has to loop through the available devices to build the appropriate data structures.  This can be easily accomplished through the MID manager.

The MID mouse event source object is obtained so that it will start sending events to the MID event manager (in case it hadn’t previously been initialized). It is also obtained to provide functionality specific to the type of device it deals with, as described in the section dedicated to the MID mouse event source.

The fact that event listeners receive events from the MID event manager instead of event sources can make it simpler to test event listeners. For example, an event listener that in production would receive input from an event source that has a complicated physical setup could be debugged using an event source with a simpler setup that generates the same type of events. If the event source with the complicated physical setup reverted to a simpler setup solution when its original setup was not available, then the event listener code would not have to be changed.  If this were not the case, the only change needed would be to initialize the simpler setup event source instead of the one with the complicated physical setup. Without MID, the event sources would likely have different APIs, forcing developers to write more custom code for testing purposes.

2.4.         Support for Multiple Output Devices

MID may also be used to send output to devices.  An application wishing to send output to device A would act as an event source and the code that can actually send output to device A would act as a listener of these events.  This way, an application can be shielded from the details of the implementation of sending output to a particular device. MID would also provide the advantages of an event queue and event coalescing.

2.5.         Layers of MID

An application may also use MID in layers in order to construct higher level events from lower level ones.  When using MID in layers, the bottom layer is only an event source and the top layer is only a listener of events.  All the layers in the middle are both event sources and listeners of events.  These middle layers listen to events from the layer below and create events that are listened to by the layer above.

Event layering can be used to generate higher-level events.  For instance, a gesture recognizer could listen to mouse events and be a source of gesture events.

I tested this concept by building a simple demonstration application that consisted of three layers.  The bottom layer was the standard MID mouse event source, and generated events from several mice.  A middle layer listened to those mouse events and used the mode of the left button on each mouse to specify bits of a number.  Thus, four mice could generate the numbers 0 through 15.  The middle layer was also a source of “number” events that generated an event with the computed number whenever a mouse button was pressed or released.  Finally, a top layer listened to the number event and printed it out.

Although this application could have been implemented by having each event layer listen directly to the other event layer, using MID provided the advantage of a level of indirection.  This way, each listener of events only has to know that there is a source of events somewhere, but does not have to know where it is, or what code is generating those events. 

So, in the test application just described, the fact that the “number” events came from a specific module that computed its numbers from mouse buttons was completely unknown to the top layer.  It simply listened to “number” events.  For testing purposes, a different “number” generator could have used a keyboard.  I have often heard programmers say “when in doubt, add a level of indirection”.  This is typically meant half in jest, but half seriously as well.  Adding a level of indirection is a powerful software engineering principle that decouples related modules.  MID provides a structure for adding a level of indirection in event-driven multi-layer programs.

2.6.         Extending MID

MID can be extended in two ways. One way is to add support for devices already supported by MID through native libraries on platforms not currently supported by MID. Adding such support consists of creating a native library in the unsupported platform that interfaces correctly with the existing event source.

MID can also be extended by adding support for new devices.  Adding support for such devices consists of adding device specific event sources for each new type of device.  These event sources would use native code in order to know when to generate events if data from the devices cannot be accessed through Java.  Event classes and listener interfaces would also have to be written.

2.7.         Limitations

The most limiting aspect of MID is that it currently has only four implemented event sources (multiple mice, mouse data over socket, RFID tags, and DiamondTouch).  These sources themselves have their limitations.

The MID mouse event source works only with USB mice under Windows 98 and ME. The mice have to be USB mice because of a limitation of Windows.  Windows merges the input from the non-USB mouse with the input from USB mice into one event stream. Windows 2000 merges input from all mice at a very low level, making separate streams inaccessible to MID. Windows XP appears to have a new API that may give access to multiple streams of mouse data at a high level, but I have not yet tried to use it for MID.

Another limitation of the MID mouse event source is that it takes over the system cursor and users cannot send mouse input to other windows besides the application’s window unless they switch to them through the keyboard. This was done on purpose because the alternative is to have both mice control the single system cursor.

In the switch event source, the current software in the RFID tag reader only outputs data on the serial line when it is powered up or a tag is placed on it, thus there is no way to support a plug and play behavior.

Another limitation of the switch event source is that it supports only one reader. This fact makes the switch event source an example of MID being used to support input from devices that Java does not support rather than an example of MID being used to support input from multiple devices.

2.8.         Performance

While I have not measured the time to generate MID events, I have compared it to standard Java event code, and there are no noticeable differences that affected performance.  In addition, I have tested MID by using four mice simultaneously with KidPad running on a 266 MHz Pentium II laptop, and performance is acceptable (i.e. cursors followed mouse movement smoothly).

2.9.         Use

As mentioned earlier, KidPad and SearchKids use MID to obtain input from multiple mice. MID has also been used by Kori Inkpen’s research group at Simon Fraser University and Saul Greenberg’s research group at the University of Calgary among others.

2.10.    Lessons Learned

In developing MID, I learned about the importance of building architectures that are flexible enough to work with a variety of devices and user interfaces. Most of the architectures designed before MID were tied to specific types of input, or user interface metaphors, making their use limited for those who wanted to work on different types of applications.

Another characteristic of successful architectures is that they be easy to extend; in this case, making it easy to add support for new devices. Single Display Groupware architectures should do this while providing programmers with a consistent way of accessing data no matter what type of device it comes from. In providing this consistency, architectures should also give programmers ways of accessing input device data that are very similar to those provided by the platform in use. This makes it easy to switch to accessing the specialized devices and reduces the learning curve for programmers.

2.11.    Recommendations for Operating Systems Designers

A better alternative to creating and constantly updating architectures like MID is for operating systems to better support managing multiple devices. The best way to do this is for operating systems to provide a consistent and easily available way of accessing input from multiple devices of the same type. Doing so should be no more difficult that obtaining input from one mouse and one keyboard. In addition, the means to access this input should remain compatible between operating system versions, something that has been lacking in the past. Independent developers should also be able to give programmers access to new types of devices through the same architecture, and they should not have to read hundreds of pages to learn how to do this.

In addition to providing programmers with better tools, operating systems can also help users by giving them the option of using multiple cursors. While this could cause some problems in applications not designed to handle input from multiple devices if users attempted to work in parallel, it could provide users advantages if they took turns at doing work. This may be particularly helpful for cases in which someone is being taught how to use an application. Such support though would likely encourage application developers to include features in their applications to take advantage of what multiple users could do. This could be further encouraged if the operating system provided a set of widgets, such as local tools, that were designed to work with single display groupware.

 


3.       KidPad

With the help of MID, I developed KidPad, a collaborative storytelling application. Collaborative storytelling can play a positive role in children’s development as it can help children cultivate interpersonal and creative skills. Oral traditions are an example of how stories can provide an effective way of transferring and retaining information (Rubin, 1995). Storytelling also helps children develop communication skills. These skills are necessary for collaboration, and learning to work with others is another important skill to acquire (Wood & O’Malley, 1996).   Collaborative storytelling is often present when children play. However, there is currently little computer support for children’s collaborative storytelling. Computers can augment the collaborative storytelling experience by allowing for storage, and the ability to copy, share, and edit stories.  They can also provide an excellent medium to create non-traditional forms such as non-linear stories.

These were the motivations for developing KidPad. KidPad provides drawing, typing and hyperlinking capabilities in a large two-dimensional zoomable space*. Through these capabilities, children can create stories by drawing story scenes and linking them together in two-dimensional space.  KidPad supports multiple users through the use of multiple mice.

KidPad was created as part of KidStory, a three-year project (1998-2001) funded by the European Union’s initiative for Experimental School Environments. KidStory was a collaboration between the Royal Institute of Technology in Stockholm, Sweden, the Swedish Institute of Computer Science, the University of Nottingham, and the University of Maryland. The aim of the KidStory project was to support children’s collaborative learning experiences through technology. It accomplished this objective by forming an interdisciplinary, intergenerational, international design team that had the development of KidPad as one of its most important accomplishments.

KidPad’s most distinguishing characteristic is its collaboration capabilities.  I have consistently found in my research that children like to work with each other at a single computer (Benford et al., 2000), and yet most software has no explicit support for this. KidPad’s design team designed KidPad to support multiple simultaneous uses by giving each child control through their own mouse. The interface is designed explicitly to support this interaction style.

KidPad gives each user control of a local tool on the screen, which acts as a cursor and holds the user’s mode (e.g. a red crayon local tool draws in red color). Local tools­­ (Bederson et al., 1996) are organized in toolboxes that can be opened (showing the tools), or closed (hiding the tools).  Figure 7 shows KidPad with all its tools visible.

While KidPad appears to be a simple drawing application at first, it differs from most such applications in offering an enhanced authoring experience including zooming, panning, hyperlinking and collaboration capabilities. Children can create content in KidPad as they would in other drawing applications by drawing, typing and erasing.  However, KidPad’s use of vector graphics provides capabilities not usually found in bitmap graphics applications.  For example, children can reshape their drawings and make them wiggle. KidPad also offers virtually infinite panning and zooming capabilities. Because of KidPad’s use of vector graphics, drawings do not become pixelated when the view is zoomed in.


Links can be created from any drawing in KidPad to any other drawing or to a particular view of the KidPad canvas.  When following a link to a drawing, the screen view animates so the drawing occupies the entire screen.  When following a link to a view of the KidPad canvas, the screen view animates to the target view.  KidPad’s hyperlinking capabilities enable storytelling by allowing children to create links between story scenes they draw.

Figure 7: KidPad with all its tools visible

In building KidPad, I learned about creating technologies to support children’s collaboration and creativity. In particular, the improvements in the design of local tools, and KidPad’s authoring environment provide valuable lessons for future work.

3.1.             Related Work

3.1.1.       Single-Display Groupware User Interfaces

Other researchers have developed varied solutions to the challenges of building SDG interfaces. MMM and PebblesDraw try to adapt existing metaphors in interfaces designed for adults. These tend to be not concrete enough for children with their use of menus. SDG interfaces designed for children on the other hand tend to be more simplistic and do not offer a wide range of functionality.  KidPad, through the local tools metaphor, combines the flexibility of the interfaces designed for adults with the concreteness of the interfaces designed for children. Following are descriptions of the user interfaces of some representative SDG applications.

3.1.1.1.        MMM

MMM’s user interface is an example of adapting existing adult user interfaces to SDG. A typical MMM screen consists of five different items: a menu, a rectangle editor, a text editor, a color menu, and a home area (Bier & Freeman, 1991).  When a new user starts using MMM, a home area is created for that user.  A home area displays a set of user preferences, which in fact act as a mode.   Each user may have more than one home area (and therefore can switch between modes). In order for users to know what mode they are in, they can check their home area.  For example, it shows what color they are using when creating rectangles.

MMM has two types of tools.  The rectangle editor allows users to create solid colored rectangles and edit them.  The other type of tool is a simple text editor. MMM supports what its authors call fine-grained sharing, meaning that there is no locking of objects as users work on them.  Therefore, two users could be resizing the same rectangle at the same time.

MMM uses color to identify users and object selections are denoted by their color. If a user selects a few characters of text, these will be underlined with the user’s color.  Commands are available through menus.  There is an all-purpose menu that applies to all users all the time plus a menu for each user that is displayed in his/her home area with commands that apply to what the user is currently doing.

3.1.1.2.        Applications Built With Colt

A few collaborative applications were developed at the University of Washington using Lauren Bricker’s Colt (Bricker, Baker, Fujioka & Tanimoto, 1998).  Colt provides each user with a mouse cursor and supports toolbars. The cursors are colored so users can easily identify which cursor they are controlling.  Color is also used to show users what object they may act on.  For example, if a user can use a tool from the toolbar, the tool is outlined with that user’s color.

Bricker built a collaborative drawing application based on the Etch-a-Sketch metaphor.  A user gets to control the x-axis of a pen and the other user gets to control the y-axis.  Users were encouraged to cooperate by being assigned drawing tasks, or by trying to follow a shape in the background.

In a collaborative puzzle application users may solve a puzzle (such as putting together the Mona Lisa) in parallel (under one mode) or in CCO mode.  In CCO mode, they control the location of a piece of the puzzle by the average location of the mouse cursors.  They can also rotate the pieces by moving the cursors.

In a “chopstick” application, each user gets to hold a chopstick and the goal of the game is to pick up beans.  In a color picking application three users choose a color by each controlling the vertex of a triangle over a color palette.  The color is assigned based on the center of the triangle.

The applications written in Colt have a strong CCO flavor favoring what Bricker calls required collaboration, that is, users cannot use the application unless they collaborate.  However, the puzzle application shows that Colt can also be used for applications that do not require collaboration.  Bricker actually defines five different levels of collaboration: disallowed, discouraged, neither encouraged or discouraged, encouraged, and required.  These levels of collaboration may prove valuable in describing other SDG systems and in thinking about requirements for new SDG systems. In spite of this contribution, the interfaces developed with Colt do not provide a wide range of functionality, allowing users to perform only one function.

3.1.1.3.        PebblesDraw

PebblesDraw (Myers, Stiel & Gargiulo, 1998) is a simple drawing program, similar to Microsoft Paint, which allows multiple users to draw a variety of shapes. Designed for adults, it adapts traditional widgets to SDG. In PebblesDraw, each user’s actions are independent.  If a user selects an object, that object is selected only by that user.  In order to identify users, PebblesDraw uses shapes as opposed to color.  The shape is used as the user’s cursor, and is also used on selections, text edits, etc.  If a user executes a command, it will only affect the object that user has selected. PebblesDraw provides two types of undo’s.  One does an undo for the last operation (no matter who did it), and the other does an undo of the last operation by the user who invokes the command. 

PebblesDraw gives visual indications of modes. At the bottom of the screen, it shows the names of the users and their mode.  This is analogous to MMM’s home areas.  The cursor also gives feedback by consisting partly of a small box with a graphic corresponding to the current mode inside.

Myers mentions problems with popup menus and with graying out illegal options.  Popup menus can be problematic because they can obstruct other user’s views and it is also technically hard to invoke them  (at least in the platform Myers was using).  Graying out options is problematic because an option that is illegal for one user may be perfectly fine for another.  Therefore, options cannot be grayed out, and some kind of feedback has to be given to users that attempt to perform an illegal action.

3.1.1.4.        Klump

Klump (Benford et al., 2000) is a children’s collaborative character creation tool meant to aid in the telling of stories.  The Swedish Institute of Computer Science, the Royal Institute of Technology (in Stockholm, Sweden), and the University of Nottingham developed Klump. Klump supports up to two simultaneous users, taking input from two mice connected to separate computers but using only one display.  Klump runs on Windows NT systems.

Klump allows children to create the character of a story by “molding” a piece of play-dough-like material (known as a klump) on the screen.  Children can squeeze and stretch the klump, apply face-like looks to it, change its colors, and rotate it. 

Each child controls an arrow shaped cursor at all times.  Color differentiates the cursors. In order to change the color of the klump or give it face-like looks, the children can click on klump-like objects on the left and right sides of the screen.  Clicking on these objects performs an action on the klump.  Since there is only one object that can be acted upon this simple approach works well. 

The cursors can also act on the klump by squeezing and stretching it at a particular point or by rotating it about a point.  When a child drags the mouse on the klump, the klump will be either squeezed-and-stretched or rotated.  The mode the user is in depends on what option was last clicked on a klump-like toolbar on top of the screen.  Since the cursors remain the same, there is no way for the children to know if they are on rotation or squeeze-and-stretch mode unless they try to act on the klump.

Klump encourages collaboration by allowing users to mix colors or produce different faces if they click on two color-klumps or two face-klumps within a short time-span.  This is more easily accomplished if the two users agree on collaborating.

Klump manages SDG interaction by distinguishing users using color, selecting actions by using toolbar-like components, having only one object that users can act upon, and not indicating what mode a user is in. Klump’s main limitation is that its interface is designed to operate only on one object.

3.1.1.5.        Single Display Privacyware

Shoemaker and Inkpen (2001) at Simon Fraser University expanded the concept of Single Display Groupware with Single Display Privacyware (SDP). SDP allows users wearing special goggles to use the same application on the same computer, with one monitor, yet see different things. While no interesting applications were developed using this concept, it did provide an interesting expansion to the world of Single Display Groupware. The advantage of SDP is that it can provide users different views of the application space, like groupware applications used on separate computers, yet it does so in one computer with co-present users. The main disadvantage is that the current implementation forces users to wear goggles.

3.1.2.   Other Systems Supporting Co-Present Collaboration

Other researchers have looked at technologies to facilitate group interactions. While these systems are not necessarily SDG, they attempt to solve similar problems through the use of technology to support co-present collaboration. The common thread in these technologies is a move away from the personal computer into room environments and tangible interfaces.

3.1.2.1.        Colab

Colab was an experimental meeting room created in the mid-1980’s at Xerox PARC.  Its goal was to study how computers could aid problem solving during face-to-face meetings (Stefik et al., 1987).

The Colab room was equipped with six personal computers connected by a LAN, a large touch-sensitive screen, and a keyboard someone standing by the large screen could use.  These computers ran "meeting tools" software, which followed the WYSIWIS (what you see is what I see) idea.  WYSIWIS was applied in a relaxed manner though, allowing for private windows and not showing everyone else's cursors (unless requested).  The meeting tools also used locking of objects as a way of preventing conflicts (i.e. if a user started moving an object no other users could interact with the object).

The meeting tools developed for Colab were meant to help the group that developed Colab with the activities they pursued during their meetings.  These include Boardnoter, Cognoter and Argnoter. Boardnoter was meant to imitate a chalkboard.  It allowed users to type text, do freehand sketching and erase.  The Cognoter tool was meant to help users prepare outlines for presentations, talks, or papers.  The Argnoter tool was designed for the presentation and evaluation of proposals. Both the Cognoter and the Argnoter supported the different stages of the task at hand, changing their interaction style accordingly.

While using the meeting tools, in particular the Cognoter, the creators of Colab noticed that by allowing parallel independent actions to occur at the same time, the meeting group would lose a good characteristic of turn-taking: a shared focus.  It was harder for the meeting participants to keep track of all the ideas being put on the table if they were being entered in parallel.  Another problem found with working in parallel was that users at times would like to scroll in separate directions, or locate windows in different places, which gave way to "scroll wars" and "window wars". 

The Colab project was a pioneer in the use of computers for supporting co-present collaboration.  However, since each user had a computer, many of the issues that must be dealt with in SDG go away and are solved by the use of private windows and options.  Instead of having users share the way they access tools, systems such as Colab can provide traditional user interface metaphors and not worry about them being used by more than one person at the same time.  Locking objects was the solution chosen to prevent conflicts between user actions.  This solution is applicable to SDG.

3.1.2.2.        Tivoli

Tivoli is an application developed in the early 1990’s at Xerox PARC that runs on Xerox's Liveboard, an electronic whiteboard.  The goals of Tivoli were to explore interaction techniques for Liveboards and to create software to support small workgroup meetings using Liveboards (Pedersen, McCall, Moran & Halasz, 1993).

The creators of Tivoli and Liveboard designed them to be used in small meetings of up to eight people, where all the participants would have access to the Liveboard and would not have a problem seeing what was written on it.   The designers for Tivoli also decided that it should work as a whiteboard with extended functionality.  So if someone wanted to use it just like they would use a regular whiteboard, they could do so (this is a good transfer effect).  However, it did not end up working this way.

The extended functionality in Tivoli is provided through toolbars on the left and bottom sides of the Liveboard.  In order to draw, users must tap the draw button.  In order to erase, users must tap the erase button.  There are many other functions to manage drawn objects including selection, copying, pasting, applying properties to objects, etc. 

In order to provide extra space not available in a regular whiteboard, Tivoli also provides scrolling and the ability to move to new screens (referred to as slides in Tivoli).  Tools are available to manage these slides, including the ability to print and save. 

Liveboard supports the simultaneous use of up to three pens and Tivoli was designed to support multiple simultaneous users.  Tivoli keeps track of system state and pen state.  It also remembers which pen generated which actions so that undo’s only affect the actions performed by the pen that taps the undo button.  While basic support was provided for multiple users, the creators of Tivoli admit that they did not extensively test Tivoli with multiple users.

Tivoli makes use of gestures to provide extra commands.  They allow users to invoke commands without having to use toolbars and therefore bypass the problems generated by them.  Users can generate gestures by drawing on the board while a pen button is pressed. 

Tivoli is different from most other SDG applications because it deals with uncommon devices: a large electronic whiteboard, and pens.  While the system provides support for multiple users it does not do much to make this easier.  There are no indications of what mode a particular user is in; hence, confusion is bound to occur.  Tivoli also relies on conflicts in its use being solved through social rather than technological means (i.e. a user could start a new screen while another user is drawing). 

3.1.2.3.        Rekimoto’s Work

Jun Rekimoto has led a few projects with an emphasis on co-present collaboration at the workplace.  In 1997 Rekimoto presented pick-and-drop, a form of drag-and-drop that uses a tangible user interface and works across computers (Rekimoto, 1997).  With pick-and-drop, a user can pick up an object from a computer with a special pen, and then use the pen to drop it on the same computer or on another computer.  This is implemented by having all computers networked with access to a database where the pen's unique IDs are associated with objects.

Rekimoto has also studied how computers can aid people in meeting rooms (Rekimoto, 1998).  He proposed improvements over Tivoli by providing each user of the whiteboard with a hand-held computer for tool selection and data entry.  He thought his approach would solve the following problems: entering text, searching for existing information, and having physically unreachable user interface components.

Rekimoto's solution in a way went back to the Colab approach, giving each user a computer, with the exception that Rekimoto's computers are portables that were not available in the days of Colab.  His idea is to have users work on the whiteboard using their hand-held computers as palettes, just like painters do.  The operation used to transfer objects from the hand-held computer to the whiteboard is pick-and-drop. By using PDAs, users can enter text on their PDAs and then pick-and-drop it on the whiteboard.  They can also select commands and search for data in their PDAs. 

Some problems that were mentioned in the Colab system could occur again though, as other participants would not be able to see what someone is typing or searching for until they drop it on the whiteboard.  On the other hand, the lack of shared focus problem might go away as users must stand next to the whiteboard to enter the data.  Also, users can still do freehand sketching with the pen directly on the whiteboard.  Pick-and-drop can further be used for picking and dropping items within the whiteboard and back into user's PDAs.

Rekimoto's latest work on supporting meeting rooms is the creation of a special meeting room where a table and a wall have displays (Rekimoto & Saitoh, 1999).  He refers to this environment as "augmented surfaces."  In this room, users can drag objects from their laptops and place them on the table or on the wall.  They can also drag them to other people's laptops.  Physical objects can also have digital objects associated with them.  For example, a video tape could have an MPEG file associated with it.  The dragging can be performed using the mice attached to the laptop computers.  Rekimoto calls this type of dragging hyperdragging.

All the items in the room, including the laptop computers need to have a special barcode that is read from a camera on top of the room.  All the computers are networked and the objects are transferred to other computers through the network.  They are associated to physical objects through a database on the network (in a similar way that objects were associated with pens in the implementation of pick-and-drop).  Using this technique, if someone wanted to remember someone's phone number, instead of writing it on his/her hand, s/he could stick a barcode on his/her hand and drag the phone number to it.

3.1.2.4.        Contributions from the MIT’s Media Lab

Hiroshi Ishii’s Tangible Media Group at the MIT’s Media Lab has developed some tangible technologies for children that can support collaboration. Curlybot was a palm-sized vehicle in the shape of half a sphere that enabled children to record and play back motion (Frei, Su, Mikhak & Ishii, 2000). When asked to playback motion, curlybot would repeat the motion until stopped by pressing a button. Children could draw patterns with curlybot by attaching a pen to it. While two children could not interact with curlybot simultaneously, children could learn from each other’s use of curlybot, elaborating on the patterns. Patten, Griffith and Ishii (2000), designed a tangible interface for controlling more traditional robots. In order to program robots, children could connect two surfaces with hooks by using strings. One surface’s hooks represented events, while the other’s represented actions. Because of the physical nature of the interface, multiple children could work on programming the robot at the same time. Piper and Ishii (2002) developed a collaborative toy unrelated to robots called PegBlocks. PegBlocks are wooden blocks with protruding pegs that can be pushed by children. The blocks can be connected to each other. When children push a peg, a dynamo converts the motion to electricity. This electricity is transmitted to the connected PegBlocks, causing the pegs in them to move. Having one child control each PegBlock, this toy may support collaborative activities.

3.1.2.5.        StoryRooms

Montemayor et al. (2002) built StoryRooms, a room environment where children can combine low-tech and high-tech materials to create a story experience. In StoryRooms children can create props made of boxes, paper, and other materials they wish to use, and augment them with sensors and actuators. Children can program these sensors and actuators by using a magic wand that associates sensor events with actuator actions. Through this programming, children can create an interactive room and use it to provide other children with a room-sized story to experience.

3.1.3.   Studies of Children’s Use of Collaborative Software

There have been very few studies of children collaborating while using software. Besides the studies of KidPad described in section 3.10, no other studies have looked at how children collaborate through the simultaneous use of input devices. However, other studies have looked at children collaborating while taking turns with two mice, or sharing one mouse. Following are summaries of studies including a study by Kori Inkpen that suggests SDG may be advantageous for children.

Inkpen, Gribble, Booth & Klawe (1995) conducted a study that tested whether two children working with two mice are more effective at solving problems on a computer than two children with one mouse.  In the study, the children used a problem solving game on computers with two mice hooked up to them.  However, only one of the mice could control the cursor at a time.  To switch controls, she tried two protocols: give, and take.  In the give protocol, the child currently controlling the mouse could give control of the cursor to the other child.  In take mode, the child not in control could decide to take control of the cursor.  The researchers also had boys work with boys and girls work with girls. The significant results were that girls solved more problems with the give protocol than when working with one mouse; and that boys did better with the take protocol than with the give protocol.  These results indicate that having a mouse per child does make a difference in problem solving performance.

Cockburn and Greenberg (1996) studied children’s collaboration styles in TurboTurtle, a groupware system designed to help children explore concepts in Newtonian physics.  While users could sit next to each other in this system, they would not share the same screen. Singer, Behrend and Roschelle (1988) studied the collaborative use of the “Envisioning Machine”, a software program meant to help children learn Newtonian physics.  In this piece of software children had to share one mouse. Strommen (1994) reported on the collaborative use of a system for children that used one mouse.  While plenty of cooperation was observed, there were also clear problems with sharing.

3.2.         Design Roles

In designing KidPad, a variety of people participated in different roles, mostly due to the international nature of the project. While the actual software development took place at the University of Maryland, design ideas came from all the institutions participating in KidStory. As a design partner, I developed most of KidPad and asked research questions in many design sessions.

3.2.1.   Design Partners

Most of the participatory design and contextual inquiry sessions were held at the University of Maryland by the Human-Computer Interaction Laboratory’s team. This is the team that provided most of the information used in designing KidPad. While this team met on a regular basis, other teams met occasionally in both England and Sweden. The combination of teams counted with computer scientists, educators, professional storytellers, graphic designers, psychologists, teachers, and children.

3.2.2.   Informants

Since these teams in England and Sweden did not meet as often, and only gave feedback on major releases of KidPad, they could be classified as informants. Another difference with the Maryland team is that the children in these teams participated while at their schools and saw their sessions as school activities. These groups were also larger than the 6 to 8 children I worked with at Maryland. They helped mostly by requesting features and reporting issues.

3.2.3.   Testers

There was no separate group of testers during the development of KidPad, as the children that participated as design partners and informants did the necessary testing.

3.2.4.   Users

Even though KidPad is meant for children to use, some adults used it during its development. The researchers used KidPad as a presentation tool, especially when presenting research work involving children. Teachers in England used it to tell stories and present information to the children in their classes as part of their curriculum.

The child users were mostly the children who participated in the design process as design partners and informants. They used KidPad to play with each other, draw, tell stories, and make presentations. In addition, the children in the study described in section 3.10 used KidPad while being observed. More detail on the uses of KidPad can be found in section 3.6.

3.3.         Local Tools

The team designed KidPad to use “local tools” instead of menus or tool palettes (Bederson et al., 1996).  Local tools are graphical objects that act as cursors and hold their own mode (e.g. a red crayon local tool draws in red at its current location).  Local tools provide a concrete visual interface that is very easy to learn for young children.  In addition, this metaphor works well with the ability to work with multiple mice at the same time (Stewart, 1999). Multiple local tools can be active simultaneously, where each user controls one local tool at a time.  Users can change the local tool they are using by picking up an unused local tool or by exchanging local tools with another user.  Some local tools change their behavior when used collaboratively (Benford et al., 2000).  For example, two crayons that start drawing near each other combine colors and draw a filled shape.

The goal of local tools is to support co-present collaboration by avoiding problems with multiple users regarding modes.  Traditional alternatives such as menus and palettes assume that there is one global mode in the application.  For example, drawing programs will assume there is a global ink color and pen thickness. While this assumption works well when there is only one user, it is no longer useful when dealing with multiple users, as each user demands a mode of his/her own.  By holding their own mode, local tools get around this problem.

3.3.1.   Design Evolution

During the development of KidPad, the design team worked to improve the usability of the local tools metaphor. Following is a discussion of the issues encountered and solutions to them.

Initially, local tools lay on a large two-dimensional space.  While active tools were always visible, inactive tools could be invisible if the user(s) had panned or zoomed since the tool had been dropped.  A click on a special icon could be used to bring the tools back to the visible area of the working canvas.  The fact that tools could be invisible to the user violated the principle of recognition over recall, as the user had to remember that there were other tools available.  It also meant that users often had to take an extra step to use a tool.

The first attempt to solve this problem was to have the tools remain visible all the time, but still stay where they were dropped.  Instead of moving with the authored content in the two dimensional space, they remained at the same screen location (what in zoomable user interface jargon is known as “sticky”).

This approach worked well until the team added a few new tools to KidPad.  Soon, the tools were getting in the way of authoring content, obstructing portions of the screen.  They were also difficult to find, as they were dropped in different parts of the screen, forcing users to scan the entire screen in order to find a particular tool.

To deal with this problem, the team developed the idea of using toolboxes.  Toolboxes, represented by icons, hold tools and can be open or closed.  If a toolbox is open, its tools are visible.  If it is closed, its tools are not visible.  When a user opens a toolbox, its tools appear by its side.  Closing a toolbox removes its tools from the screen, giving more space for authoring content. Toolboxes can therefore be used to quickly organize tools and better manage the authoring space. 

Originally, users could change the tool they were holding by either picking up another tool or dropping the tool they were holding. Each user had a hand tool (used for panning and following links) that was picked up whenever a tool was dropped.  Having one hand tool per user contributed to clutter on the screen.  Having two different types of actions to switch tools also made the interface complicated. The fact that the right mouse button dropped tools also proved confusing, as according to my experience and as suggested by literature (Kaluger & Kaluger, 1979), many children cannot distinguish between mouse buttons.

The solution to these problems was to have only one method of switching tools: picking up another tool.  This avoided the need to drop a tool, eliminating an unnecessary function.  By eliminating the drop function I was also able to eliminate all the hand tools except for one (i.e. hand tools became no different from the other tools).  Since the hand tools were the tools held by users when KidPad was started, the team had to choose other tools for users to hold when starting KidPad.  The choice was to pick up crayons.  The choice is encouraging, as it has helped children to start drawing and experimenting with authoring right away.

While the implemented solutions helped in reducing clutter on the screen, KidPad still had clutter problems because when tools were picked up, previously held tools were dropped in various parts of the screen, obstructing the authoring area. To solve this problem, the team decided that a dropped tool should go back to its original location next to its toolbox.  If its toolbox is closed, then it goes inside the toolbox and becomes invisible.  Therefore, tools only take up space next to the toolboxes, which can also be closed.  This change reduced clutter.

As more tools have been added to KidPad, the space that the tools take next to the toolboxes has become a bit of a concern.  I have implemented the simplest solution to this problem: allow for higher resolutions. KidPad was using an 800x600 pixel default resolution, while most recent computer systems can easily handle higher resolutions.  I have therefore added options to run KidPad at higher resolutions.

I have also recently rearranged the tools and toolboxes.  While all toolboxes used to be at the bottom of the screen, I moved one to the top left of the screen.  This toolbox holds tools that cannot be picked up and therefore do not behave like normal local tools.  They handle loading, saving, printing and exiting (similar to a file menu).  While originally there were only crayon tools and an eraser tool in the bottommost toolbox (which is open by default), I moved the panning, zooming and linking tools to it, to encourage children to experiment with these tools. 

Through the work of the team with children, the team found that they often had trouble collaborating with each other.  The team thought of a way to encourage collaboration by providing some tools with collaborative behaviors.  If two tools of the same kind are used together, they behave in a different manner.  For example, two crayons fill the area between them with their combined color.  I also added collaborative behaviors to a few other tools (Benford et al., 2000).

These collaborative behaviors brought the need of having more than one tool of the same kind at times.  In order to accomplish this, I added a cloning tool that copies tools, creating a “clone”.  I later extended the capabilities of the cloning tool to copy authored material.

3.3.2.   Trade-offs of Local Tools

The main advantage of local tools is that they easily deal with the issue of modes in a multi-user environment. Having each tool hold the user’s mode takes care of this.  The other important advantage of local tools is that they make sense to children as using them resembles the way they use tools such as pencils and erasers in their everyday activities.  In this way, local tools are less abstract than menus or palettes (users don’t have to look elsewhere to see what their mode is), and thus are easy to learn.

The main problem I have found with local tools is the amount of space they occupy.  The evolution of local tools has had to deal primarily with how to make them more space efficient.  Even given the latest changes, the main issue with local tools is that they do not scale very well.  In this aspect, they face the same problem palettes face and are only useful up to a moderate number of tools. 

A possible solution to this problem is to have local tools behave a bit more like menus and less like palettes.  The toolboxes would be always closed.  When clicked on, they would open, and the user would be able to pick up a tool.  After the tool is picked up, the toolbox would close again.  The advantage of this method is that space would always be available.  The disadvantage is that it would sometimes take an extra click of the mouse to get to tools, and users would have to remember in what toolbox the tool they want resides. It may be that just like WIMP (windows, icons, menus, and palettes) applications use a combination of menus and palettes, SDG applications can use a combination of local tools approaches.

3.4.         Toolbox Design

KidPad’s tools are currently organized in three toolboxes.  Two toolboxes are located on the bottom right of the screen and their tools appear to their left when they are open.  A third toolbox is on the top left of the screen and its tools appear below it when it is open.

When KidPad is started, the bottommost toolbox is open while the rest are closed.  The bottommost toolbox (Figure 8) holds the basic tools needed to tell stories in KidPad.  It holds crayons to draw; an eraser to fix mistakes; a hand tool to pan the drawing area and follow hyperlinks; zoom-in and zoom-out tools; and a magic wand to create hyperlinks between story scenes.


Figure 8: KidPad’s bottommost toolbox

 


The other toolbox at the bottom of the screen (Figure 9) holds tools that help better manage stories and add some special features.  The selection tool is used to select and move objects. The grouping tool makes pieces of a drawing act as one.  The text tool types text. The turn-alive tool animates shapes, making them wiggle.  The clone tool makes copies of tools and objects. The filler is used to fill line shapes with color. The puller tool is used for reshaping lines and polygons. The x-ray tool makes x-ray windows. Drawing inside an x-ray window makes that drawing visible only through that x-ray window. The help tool plays audio files describing the functionality of other tools when moved over them. When picking up another tool, it demonstrates that tool’s functionality.


Figure 9: KidPad’s second toolbox from bottom

The toolbox on the top left of the screen holds tools that cannot be picked up.  These handle some of the items commonly found in “File” menus in many applications.  They support starting a new story, loading stories, saving the current story, saving the current story in HTML format, printing the current story, and exiting the application.  When one of these tools is clicked on, the operation is carried out, and the user keeps the tool he/she had before clicking.

The tool for loading stories takes the user to another screen (Figure 10) where he/she can pick a previously saved story from a bulletin board.  Clicking on a story’s thumbnail loads the story, while clicking on the thumbnail representing the current story (at the bottom of the screen), takes the user back to the current story.  This screen used to also include icons for saving, saving as HTML, printing and exiting.  Through children’s feedback the team found that the earlier interface was confusing and therefore opted to remove the functionality unrelated to loading stories.  Those options are now available as tools in the toolbox on the top left corner of the screen.

Figure 10: KidPad’s Bulletin Board

3.5.         Arriving at KidPad’s Design

KidPad’s design is the outcome of contributions by many adults and children, sometimes working together, sometimes generating ideas separately. An example of children and adults elaborating on an idea is the way crayons work in collaborative mode. There were two reasons behind this design.  The first was the observation by adult researchers of how children had difficulty collaborating. The team thought it may be useful to provide incentives for children to collaborate. At the same time, children were asking for new features, including a way to access more colors, and a way to draw shapes filled with color. The solution was to allow children to mix colors if they started drawing near each other, and draw a filled shape together. The design took into consideration the ideas of both children and adults, providing children with the new features they requested, but making them available only if they worked with each other.

Another example of children and adults working together was the development of the x-ray tool. The adult researchers had thought of including a lens like feature in KidPad, a movable window that would allow users to draw in it and whose contents would only be visible through it. When adult researchers showed this feature to the HCIL team, the children were not particularly excited by it, and for the most part did not understand how they could use it.  After having a session on the idea though, the team found a way that made it work. The key was to make the lens a see-through lens, so the contents of the lens would be shown over the existing drawing on the KidPad canvas. The other design decision that made it easier for children to work with the lens was to rename it the x-ray tool. This made a lot more sense to children and helped them understand how they could use the tool. By working together, the team took a tool that generated little interest, and turned it into one of the children’s favorite tools.

Sometimes, children are the ones who provide the ideas, with adults elaborating on them. An example of this is the turn-alive tool. This tool, that makes drawing wiggle, came about because of children’s repeated requests that sometimes they wanted to see their drawings come alive. This is another of the children’s favorite tools, one that adult researchers would probably not have thought of if it were not for the children’s ideas.

Sometimes it is adults who take the initiative to develop design solutions based on observations of children’s difficulty. This was the case with the design improvements to the bulletin board. After observing the difficulty many children had using the bulletin board, the team had a participatory design session and developed design alternatives that I later implemented. 

3.6.         Observations of KidPad’s Use

3.6.1.   Storytelling

Stories created in KidPad are composed of scenes that are linked together.  Scenes are composed of drawings, text and other features. The location of scenes in KidPad’s large two-dimensional surface can have a meaning within the story.  For example, drawing something inside a character’s head might represent the character’s thoughts.  Scenes are linked together by using the magic wand tool.  A scene may have more than one link coming out of it.  If this is the case, these links provide different paths through the story.  To tell a story, users follow links as they interpret the scenes.

Since the team built KidPad with the intention that it be used for storytelling, we are happy to frequently observe it being used in that way.  Most of the stories created by children are very visual, with very little text.  The children tend to draw separate scenes and then link them together.  Most of the time, the scenes are next to each other, the children panning to go from one to another.  Not surprisingly, older children tend to tell more elaborate stories with more scenes where the location of the scenes has a meaning and where zooming is often used in addition to panning.  Older children also tend to use more text. Figure 11 shows typical scenes from stories, one by younger children, another by an older child.


Figure 11: On the left, a scene from a story by a 7 year-old girl and a 9 year-old boy showing a special chair. On the right, a scene from a story by an 11 year-old boy.

A study by a colleague suggests children tell more elaborate stories when interpreted a story shown using KidPad’s animated zooming capabilities (Boltman, Druin, Bederson, Hourcade & Stanton, 2002). The study is described in more detail in section 3.10.

3.6.2.   Other Uses of KidPad

Children tend to first use KidPad as a drawing tool. This may be due to the fact that the drawing toolbox is the one that is originally open when KidPad starts. When more than one child is using KidPad, the users often divide the screen and draw in their “assigned” area. Children that concentrate on drawing often do not use the panning and zooming capabilities and limit their activities to drawing something on the available screen space.  I have recently moved the panning, zooming and linking tools to the drawing toolbox to encourage children to use them more often.

When children use KidPad collaboratively, some game playing can develop.  The most common form of playing is scribble wars.  Children scribble over each other’s drawings and sometimes even use the eraser.  Another form of play involves zooming while drawing.  One child will take over zooming while the other one draws.  They enjoy zooming through the shapes they draw in this collaborative form of game play.  A similar form of game play involves panning while drawing.   In this case, one child will be panning while the other one holds down a crayon.  In this way, the child that pans controls what shapes get drawn together with the child holding the crayon, giving way to some unexpected shapes that amuse the children.

The team has also used KidPad for presentations.  This use has mostly been by adults who want an alternative to Microsoft PowerPoint when presenting information about projects that involve children.  Children have also used KidPad to present some of their ideas.  While using KidPad for presentations is not for someone who does not have experience using KidPad, what can be accomplished by an expert user is very impressive.  The outcome is presentations that look nothing like traditional presentations.  They look very artistic, and at the same time are often easier to follow because the information is arranged spatially in a manner that helps the audience better understand the content.  The fact that KidPad’s text tool supports only a child-like font has limited the use of KidPad for presentations related to children.  Lance Good, one of my colleagues, is currently developing CounterPoint, a zooming presentation application for adults (Good & Bederson, 2001).

3.7.         Collaboration

One of the team’s goals in the design of KidPad was to support collaboration in the creation of stories.  Early on, the team decided that in order to effectively support collaboration, all parties involved in the collaboration had to have equal access to the application. In order to support this vision we had to look for hardware and software solutions.    We chose to support multiple mice on the hardware side.  We also considered tablets, but no USB tablets were available when the project started, making support for multiple tablets more difficult.  Mice were also attractive because of their low cost and wide availability.  On the software side, we implemented local tools, described previously. 

In spite of all the support for collaboration built into KidPad, the team still found that if children do not want to work together, they do not collaborate.  They will very often decide to work independently, or even worse, compete against each other with scribble wars, or erasing each other’s creations. 

As mentioned earlier, in order to encourage collaboration, the team added special collaborative behaviors to some of the tools.  When using these tools together, users can access extra functionality not available when working independently.  Of the tools I added collaborative behaviors to, crayons were by far the most popular in collaborative mode.  This might be due to the fact that they are the type of tool children are most likely to be holding at the same time. Through informal observations the team found that these collaborative behaviors encouraged communication between users and helped spark collaboration.  They were no silver bullet though, as children who did not feel like working with each other still did not collaborate.

While the goal has been for children to collaborate while using KidPad, the team has found that KidPad supports multi-generational collaboration.  We found this out by chance during a University of Maryland open house.  We setup KidPad on several computers and advertised our lab as a place where there were activities for children.  While our original intention was to have children work with each other, many children started using KidPad with their parents.  We were pleasantly surprised to find out how well they collaborated.  None of the visitors had seen KidPad before, yet they managed to use it and collaborate without trouble.  These observations are a strong suggestion that KidPad may be a good application to bring the family together at homes.  The other lesson we learned from this experience is that social roles are very important when it comes to collaboration.  While the parent-child role worked very well, other roles between children may lead to competition rather than collaboration.

3.7.1.   Programming Issues

On the programming side, supporting collaboration presented issues that do not develop when supporting only one user. Just as global modes may not be used in the user interface, the same is true of the internal data structures of the program. For example, as users create links between drawings, the source drawing is not stored globally, but instead is associated with the tool being used, and by extension with the user holding it. The issue of global modes also applies to objects on the screen. For example, with one user, if there were a need to highlight an object when tools are over it, it would be enough to turn highlighting on when a tool enters the object and turn it off when a tool leaves the object. However, with multiple users, counts of the current amount of tools on the object need to be kept in order to know whether there are any tools on the object.

Implementing collaborative behaviors provided the challenge of making it easy enough to enter into them purposefully, while making it difficult to enter into them by accident. For the collaborative behavior of crayons, where users had to start drawing near each other at roughly the same time, I had to find an appropriate distance at which the crayons would start collaborating and define a window of time in which both users would have to start drawing. I worked with the child design partners in the team in solving these issues and found a distance and a window of time that worked well for most children.

3.7.2.   Testing Issues

Testing single display groupware like KidPad poses difficulties beyond those found in testing software designed for one user. The problem is that some programming errors can only be uncovered if multiple users test the software. An example is the issue of highlighting. With one user, when the user’s cursor leaves an icon, the highlighting can be turned off. However, with multiple users, when a user leaves an icon, there may still be another user over the icon, so the highlighting has to stay on. To complicate matters, there may also be programming errors that only occur with a number of users greater than two.

This brings up the practical problem of needing more than one person to test the software. This can be difficult if, as in the case with KidPad, only one developer works on the software. While I could test some features by controlling two mice at the same time, it was not possible to test features that should work for three or more users.  For example, if I wanted to test some of KidPad’s collaborative behaviors to see if they made sense for more than two users, I needed help from other people. In practice, while I managed to find some programming errors on my own by controlling two mice, I found most of them with the help of the children in the design team, observing them use KidPad in groups of two or three.

In order to automate testing, it would be possible to use logs of mouse events recorded while children are using the software. However, this may prove problematic, as permissions would have to be obtained from parents and school officials (if the use occurred in a school). This would not be a problem with a different set of users, and it would also not prevent adults from logging cases that uncovered programming errors in the past.

3.8.         Creativity

Some of KidPad’s characteristics encourage creativity.  For example, KidPad’s support of collaboration helps children learn how to participate in group creative activities.  The exchange of ideas amongst group members can be beneficial to the creative process.

The fact that most stories children create in KidPad are very visual encourages their creative interpretation.  KidPad’s local tools make it easy to draw story scenes therefore helping children express their ideas.  KidPad’s support of non-linear stories also encourages creative interpretations because these stories can be told in multiple ways by following their multiple paths. 

The ability to place scenes in a meaningful location in two-dimensional space provides expanded semantics not available in other tools.  For example, zooming into a character’s chest area might show what the character is feeling. There are no other tools I know that allow children to express such ideas in a similar visual manner.

KidPad also encourages creativity when retelling stories. This is hinted by a study conducted by one of my colleagues, which suggests that animation and zooming encourage elaboration when retelling stories (Boltman, Druin, Bederson, Hourcade & Stanton, 2002). Section 3.10 relates this study in more detail.

3.9.         Extensions to KidPad

While KidPad on the desktop works well for small groups of children (three or less) it is not adequate for use in the classroom with a larger group of children unless they are divided into smaller groups.  This is mainly due to the physical size of desktop displays and confusion brought on by too many active local tools on the screen.  Children also have had difficulty agreeing on where and whether to navigate in KidPad’s two-dimensional space.  With my help, colleagues at the University of Nottingham, the Royal Institute of Technology in Stockholm and the Swedish Institute of Computer Science have studied and implemented ways of extending KidPad to a room environment in order to address these problems.  Aside from addressing interface and collaboration problems, bringing KidPad to a room environment enables children to interact with tangible physical objects, something closer to their world than local tools.  It also gives children more room for acting, which is not naturally performed while holding a mouse. 

The magic carpet (Stanton et al., 2001) was an early attempt to bring one aspect of KidPad (navigation) to the room environment.  By stepping on different parts of a carpet, children were able to pan and zoom a KidPad story projected on a big screen.  This type of navigation was complemented by the use of barcodes that could be associated with objects drawn in KidPad.  When a barcode was scanned, KidPad would zoom to the barcode’s corresponding object.

In thinking of a more comprehensive room extension to KidPad my colleagues and I considered the components of stories in KidPad and the role of improvisation and retelling of stories.  Stories in KidPad are composed of a series of linked scenes that provide at least one storyline.  While desktop KidPad effectively supports scenes in different places (a pond vs. the forest), it doesn’t support very well scenes that change over time (e.g. the same room later that day).  The team also recognized the existence of characters and other props in stories.  Characters tend to move between scenes while props (e.g. a cloud) may appear in several scenes at a time.

With these thoughts in mind the team designed a room version of KidPad.  KidPad was modified to incorporate a notion of virtual scenes and props, which are matched up with physical props and recorded sounds.  Children draw scenes and props/characters in KidPad, associate them with physical props and then interact with the story. 

The interaction involves navigating between scenes by using physical props that represent scenes, and pulling characters and props in and out of the story by using other physical props. The team used RFID tags and readers to implement this version.  They communicate with KidPad through MID.

3.10.    Evaluations

A study by one of my colleagues, which formed the basis of her Ph.D. dissertation, provided evidence that software that uses animation and zooming helps in storytelling (Boltman, Druin, Bederson, Hourcade & Stanton, 2002). The study looked at how different types of media affect how children interpret a wordless story. The participants were 37 boys and 35 girls from schools in Sweden and England who were between 6 and 7 years of age.  Each participant saw the story in one of three formats: on paper, on a computer with traditional hyperlinks (no animation), or on a computer using KidPad with animated panning and zooming between scenes. After going through the wordless story once, each participant was responsible for two tasks: telling the story as they saw it page by page (or screen by screen), and recalling the story without looking at it. The children told the stories to adults they were unfamiliar with. The adults collected the narratives so they could later be analyzed. The stories were evaluated based on story structure and content. A statistical analysis of the children’s stories revealed that the use of animated zooming and panning had a significant and positive effect, with children telling stories that were more complex in structure in the areas of clauses and subordinates, and more often discussing initiating events with an increased understanding of subordinate and superordinate goals. These results suggest KidPad’s use of animation, panning and zooming supports creative interpretations of stories by children more so than when experiencing the same story through a book or through traditional (no animation) hyperlinked media. This is significant, as it suggests there are advantages to using KidPad over other media.

Abnett, Stanton, Neale and O’Malley (2001) conducted a study to learn about the differences between pairs of children using KidPad with one mouse versus two mice. The participants were 18 boys and 18 girls aged 5 to 7 years old from an elementary school in Nottingham, England. The children had used KidPad on several occasions. Their teacher paired them so that there would be an equal number of female pairs, male pairs, and mixed gender pairs.  The teacher also balanced the ability of the children in each of three different types of pairs. The task assigned to the children was to use KidPad to tell a story based on a poem they were familiar with. They had 20 minutes to complete the task, after which they saved the story and presented it to a researcher. The process of creating the story and its presentation were filmed so they could later be evaluated and analyzed. An analysis of what the children said while creating the story showed that children in pairs that used two mice made significantly more statements about their own work, actions and intentions while at the same time not making less statements about joint work than pairs sharing one mouse. Females were overall better at collaborating, communicating significantly more than mixed gender pairs. When looking at the quality of the stories, the use of two mice benefited male pairs the most, while also providing advantages to mixed gender pairs, and not making a difference for female pairs. For example, for pairs sharing one mouse, stories by female pairs were judged on average to be twice as good as those by male pairs. When using two mice, the difference disappeared, and male pairs created stories judged to be as good as those created by female pairs. The researchers suggest the differences were due to the difficulty males had sharing a mouse, which caused fighting and diminished communication. The study’s conclusions suggest KidPad’s approach to collaboration is beneficial to groups of children that include males.

Stanton, Neale and Bayon (2002) further analyzed the interactions of 12 of the pairs that participated in the study. They particularly looked at how children interacted with the computer when using one mouse versus two mice. The first difference they found was that when using one mouse, children actively interacted (e.g. drew, typed) with the computer 42% of the time, while when using two mice they did so 73% of the time. When looking at the distribution of active mouse use, there were clear differences between the one and two mice conditions. For example, in the one mouse condition two of the six pairs had one child use the mouse over 80% of the time, and five of six pairs had one child use the mouse over 60% of the time. On the other hand, in the two mice condition, four of the six pairs had both children making less than 60% of the interactions. This further analysis provides more evidence to suggest KidPad’s approach to collaboration benefits children by providing more equal access to the computer and encouraging children to be more active in creative activities.


4.       Digital Library User Interfaces

While KidPad supports children’s collaboration and creativity through its support of storytelling, age-appropriate digital library interfaces can provide children with learning opportunities, and inspire and give them content for stories they may create. Digital libraries can also help satisfy children’s curiosity, and aid them in researching topics by giving them access to digital information.  However, most digital library interfaces currently available are not designed taking into account their abilities (Moore & St. George, 1991; Solomon, 1993a; Solomon, 1993b; Walter, Borgman & Hirsh, 1996). They often require reading, typing and spelling skills beyond young children’s abilities. They also often make use of abstract concepts such as Boolean queries, which are difficult even for adults to grasp.

To address these issues, I have played a major role in building graphical direct manipulation interfaces for searching, browsing, and viewing query results of digital libraries: SearchKids, and the International Children’s Digital Library (ICDL). SearchKids provides access to multimedia on animals, while the ICDL gives access to digital books.

4.1.         Related Work

4.1.1.   Designed for Children

Other researchers have contributed to addressing the issue of children accessing digital libraries. A well-known research effort led to the creation of the Science Library Catalog (SLC) at UCLA (Borgman, Hirsh, Walter & Gallagher, 1995; Hirsh, 1997). The SLC is an online public access catalog (OPAC) that uses the metaphor of a bookshelf to browse through a library’s collection of books.  The books are organized using the Dewey decimal system. To find a book, users click through the bookshelves drilling down the hierarchy established by the Dewey decimal system’s classification until they reach the desired book. The authors of the SLC found its main advantage over traditional keyboard entry systems is that it avoids spelling mistakes and dealing with Boolean queries. The main problem they found was that children could not always find the books they were looking for using the Dewey decimal system’s classification.  To get around this problem, a later version of the SLC incorporated keyword search capabilities that included spelling correction and stemming features. Other limitations of the SLC are that it does not offer the power of Boolean queries and it forces users to go four to six levels down the hierarchy (depending on the version) before they got to see books. Since the SLC was developed for 9 to 12 year-old children, it requires reading to select categories and books, as no specialized icons or graphics are used to distinguish between categories or books.

The Kid’s Catalog, by Busey and Doerr (1993) at CARL Systems, took some of the ideas from the SLC to create a user-friendlier OPAC. The Kid’s Catalog also uses hierarchical browsing of subjects, but uses icons to represent categories. These categories are based on a modified version of the Dewey decimal system that makes it easier for children to find subjects. The Kid’s Catalog also gives users the capability of looking for books by typing a search string, browsing alphabetized lists of authors, subjects or series, and browsing lists of recommended books. Users of the Kid’s Catalog cannot create Boolean queries using icons and are limited to looking for books by subject when using the software’s hierarchical browsing capabilities.

Pejtersen (1989) built Book House, another system for finding books.  BookHouse was meant for both children and adults and used the metaphor of a house as a place where books could be found. Different rooms held books for different audiences. Inside each room, users could find books in a variety of ways. One way, called analytical search, allowed users to combine search terms from different categories to specify search criteria. The system performed a logical union between items from the same category and a logical intersection with items from other categories. Users could access categories by clicking on icons, which in turn showed textual lists of terms. The remaining ways to find books provided users with browsing and similarity search capabilities.

Külper, Schulz and Will (1997) created Buecherschatz (German for book treasure), a prototype of a children’s (8-10 years old) OPAC.  Their work built both on the SLC and the Kid’s Catalog. The peculiarity of this prototype (it was never fully developed to be used at libraries) is that it uses a treasure hunt as a metaphor for finding books. In spite of the metaphor, the interface still consists of navigating a hierarchical set of categories (three levels) that lead to lists of books. They classified the books in a manner completely different from existing classification standards so that children would have an easier time finding them. However, the interface does not use icons to represent categories, meaning that users must read category names.

4.1.2.   Designed for Adults

Other researchers in the HCI community have devised innovative solutions designed for adults to search and view results of databases and digital libraries. Dynamic queries, and its successor, query previews, developed at the University of Maryland are examples. Dynamic queries (Ahlberg, Williamson & Shneiderman, 1992) makes use of sliders to specify numerical data, checkboxes and radio buttons for ordinal and categorical data, and text boxes for string searches. Results are coded using color and size, as well as positions in a plane. Spotfire, a commercial product, has emerged from this research.  Query previews (Tanin, 2001) helps users gain an understanding of where records lie in a database enabling users to narrow down their search criteria before submitting a query.

NaviQue (Furnas & Rauch, 1998), developed at the University of Michigan, uses zooming to explore an information space. The objects in this space, including user-created objects, can be used to formulate queries. The queries can be performed on a collection already present in the information space, therefore narrowing down query criteria. While NaviQue was not designed for children, the idea of organizing information in a zoomable space can be applied to children’s applications.

Movable Filters (Fishkin & Stone, 1995), developed at Xerox PARC, is another graphical interface for exploring an information space. It was conceived as an extension to dynamic queries, where information could be filtered through the use of lenses. Each lens represents a dynamic query with sliders and other controls. Overlapping lenses create compound queries. The abstraction of using lenses to represent search criteria would likely be difficult for children to understand.

Many researchers have looked at how to graphically express Boolean queries.  Some have chosen to use Venn-like diagrams. VQuery (Jones, 1998), developed at the University of Waikato in New Zealand creates an oval around every term entered on the screen.  These ovals can be overlapped, representing intersections and unions of terms. Users can select parts of the diagrams to select the query they want. The problem with Venn diagrams is that when dealing with many query terms, the diagrams tend to get very crowded and difficult to understand.

While there are many more researchers focusing on graphical direct manipulation interfaces for querying, the handful of examples just discussed shows promising possibilities.  However, there are definite limitations to these systems when young children are the users.

4.1.3.   Digital Books

When digital libraries hold digital books, it is important for the interfaces to these books to be appropriate for their readers. Many researchers have looked at the difficulty of reading on a computer screen. O’Hara and Sellen (1997) compared reading from paper to reading on a computer screen. These comparisons dealt mostly with document reading and marking by adults. Dillon (1994, 1999) has also done comparisons between electronic documents and paper and has proposed ways of evaluating interfaces for electronic documents. Shneiderman (1998) provides a nice summary of the research in this area.

Most of the research on book readers has come from industry. However, little has been published. Graham (1999) created the Reader Helper, a document reader meant to make it easy for adults to find relevant information in documents. The Reader Helper highlights relevant information in a document and shows annotated thumbnails of the document’s pages on the left side of the screen. Ginsburg, Marks and Shieber (1996) developed a reader for PostScript documents that shows thumbnails on the left side of the screen.

Today’s industry standard book readers come from Microsoft and Adobe. Their e-book readers for personal computers are Microsoft Reader (Microsoft Reader Home, n.d.) and Adobe Acrobat eBook Reader (Adobe Acrobat eBook Reader Home, n.d.), shown in Figure 12. Both Microsoft and Adobe suggest that the main advantage of using their products is their text-related features. Both products offer solutions for more screen-readable text (e.g. ClearType, CoolType). Both products also contain features for searching and annotating books, looking for words in the dictionary, and reading words out loud. Both products were mainly designed for adults reading long books or documents consisting mostly of text. Neither product provides major alternatives to their default screen layouts or visualizations of the books.

With these text readers, navigation between pages is primarily accomplished by using keyboard shortcuts or small next and previous page buttons.  These small buttons could prove difficult for young children. Fitts' law studies have shown that young children's performance in pointing tasks is significantly lower than that of adults (Kerr, 1975; Salmoni & McIlwain, 1979; Wallace, Newell & Wade, 1978), suggesting that they require larger visual targets in graphical user interfaces. A study I conducted, reported in section 5 supports this suggestion. In Microsoft Reader 2.0, the riffle control, a widget that appears when right-clicking at the bottom of a page can also be used to change pages.  It allows the user to move to the next and previous pages, the next and previous section, and shows a bar that tells users their position in the book and allows them to jump to other parts of the book. The table of contents can be accessed through popup menus. Due to the fact that the riffle control is accessible by right-clicking, accessibility for children is limited. From my experience working with children, I have learned that  many young children cannot distinguish between the left and right mouse buttons (Hourcade et al., 2002a; Hourcade et al., 2002b). My experience is supported by research showing that children do not achieve orientation with respect to themselves until age 6, and cannot apply the concepts of left and right relative to other objects until age 8 (Kaluger & Kaluger, 1979).

Figure 12: On the left, Microsoft Reader 2.0 with the riffle control open at the bottom of the window. On the right, Adobe Acrobat eBook Reader 2.2, showing its navigation bar at the bottom of the window.

Adobe Acrobat eBook Reader 2.2 has similar navigation options, with small buttons to move to the next and the previous page, and a widget at the bottom of the screen that has similar capabilities to Microsoft’s riffle control. The table of contents is also available through menus. While the right mouse button has the same functionality as the left mouse button in most cases, there is no full-screen option, so children can accidentally click on the operating system taskbar (on Windows).

Commercially available e-books for children generally use one of these two readers, although other options exist.  For example, Antelope Publishing (Antelope Publishing Home, n.d.) offers its books in HTML format for children to read in a web browser.

4.2.         SearchKids

 From 1999 to 2001, I developed most of SearchKids, an interface for searching, browsing, and viewing query results of digital libraries. The Discovery Channel and the U.S. Department of the Interior’s Patuxent Wildlife Research Center provided the initial content for SearchKids, consisting of multimedia information on animals.

The flexibility of the interface was demonstrated when I adapted it to access a digital library of children’s books as part of the International Children’s Digital Library (ICDL) project. Recently another member of the Human-Computer Interaction Lab adapted SearchKids to work with media from the University of Michigan’s Animal Diversity Web.

4.2.1.   Design Roles

SearchKids was developed in its entirety by a team at the University of Maryland’s Human-Computer Interaction Laboratory, with no involvement of outside institutions. The team had the help of child informants from Yorktown Elementary School in Bowie, Maryland. As a design partner, I developed most of SearchKids, in addition to facilitating and asking research questions in many design sessions.

4.2.1.1.        Design Partners

The design team for SearchKids was formed by computer scientists, educators, psychologists, graphics designers, biologists, and children. Most design ideas for SearchKids came from this team.

4.2.1.2.        Informants

SearchKids counted with the help from informants at Yorktown Elementary School in Bowie, Maryland. Second and third grade students at this school confirmed design decisions made at the HCIL by participating in studies.

4.2.1.3.        Testers

There was no separate group of testers during the development of SearchKids, as the children that participated as design partners did the necessary testing. However, there were a couple of occasions in which children from outside the design team came to the HCIL to briefly use the software.

4.2.1.4.        Users

I installed SearchKids in all the personal computers at Yorktown Elementary School’s library where students have been able to use it. Children at the University of Maryland’s Center for Young Children, a preschool, have also used SearchKids.

4.2.2.   Design Principles

Working with child design partners has taught me a few interface design principles that we applied to SearchKids.  The team made the interface very visual, avoiding the use of text as much as possible and therefore reducing the cognitive load.  I also made interactions with the mouse as simple as possible by using a one-click interface (i.e. no dragging, no double-clicking) with all mouse buttons having the same functionality.  The fact that we are using a one-click interface makes SearchKids easy to use on touchscreens therefore avoiding the problems many young children have controlling mice.

4.2.3.   Interface Description

SearchKids consists of three areas through which users can look for media about animals.  Figure 13 shows the initial screen and the three areas.  Users may navigate between areas and within areas by clicking on their destination or making use of the “home” and “back” icons that are always at the bottom left of the screen.

Figure 13: From left to right: SearchKids’ initial screen, the zoo area, the world area, and the search area.

The zoo area provides a way of browsing the contents of the animal database in a familiar setting.  When entering the zoo area, users see the map of a virtual zoo.  By zooming into parts of the zoo, children can find representations of animals and through them, access media.  For example, to access media about lizards, children can zoom into the reptile house and click on a representation of a lizard.

The world area provides a way for children to browse the animal database by looking for animals geographically.  It presents children with a globe they can spin and zoom into.  By zooming into a region of the world they can find representations of the animals that live in that part of the world and through them, access media.  For example, to access media about polar bears, children could zoom into the North Pole and click on a representation of a polar bear. 

The search area gives users the ability to visually specify and manipulate queries.  It also provides previews of query results.  The initial look of the search area is shown in the right-most picture of Figure 13. Figure 14 shows a more detailed view of the search area. The query region makes up most of the search area.  The icons in this region are the components from which queries can be formed.  They represent the hierarchies under which the animals in the database have been classified.  They enable children to look for media about animals based on what they eat, where they live, how they move, and a biological taxonomy. 

To explore these hierarchies, users can click on the piles under the icons.  For example, clicking on the “what they eat” icon brings into focus three icons representing animals that are carnivorous, herbivorous, and omnivorous. To move up in a hierarchy, users can click on the up arrow to the left of the hierarchical icons.  Figure 14 shows the search area after the pile under the biological taxonomy icon has been clicked on.  The icons on the right side of the figure are the “children” of the biological taxonomy icon and represent the types of animals present in the database.

When an icon is clicked on, it moves towards the characters on the top-left corner of the screen (Kyle and Dana, the “search kids”), to hold around their neck.  This icon becomes part of the query criteria.  Clicking on an icon that is on Kyle or Dana makes it go back to its original location therefore removing it as one of the criteria for the current query.

The icons on Kyle and Dana visually represent the queries children formulate.  SearchKids returns an intersection of the media items represented by icons selected from different categories and a union of the media items represented by icons selected from the same category.  This approach, while somewhat limiting expressive power, successfully enables children to specify their desired queries and does so without requiring them to explicitly distinguish between unions and intersections.


 


Figure 14: Search area when exploring biological taxonomy.

The red region to the right of Kyle and Dana shows the results of the current query.  By seeing the results of their queries as they pose them, children can quickly tell whether the database has any items that correspond to their query criteria.  Children can zoom into the region by clicking on it.  Further clicks inside the region zoom the user further and further in, until he/she selects an image. Each of the images shown in the results area represents an animal that matches the query criteria. Selecting one of these images takes children to a screen showing all the media available for that animal.

4.2.4.   Collaboration

SearchKids supports simultaneous multiple users through multiple mice by using MID. The team has experimented with two modes of collaboration.  In one mode, independent collaboration, a user’s click on an interactive item is enough to interact with that item.  In another mode, confirmation collaboration, all users must click on an interactive item in order to interact with it. In a study described in section 4.2.6, the team found that independent collaboration helps children concentrate on the task at hand, while confirmation collaboration helps them concentrate on how to use the software.

4.2.5.   Learning

SearchKids supports learning by giving children access to information in the form of pictures and audio. Besides learning what animals look like or sound like, children get to learn about animals by searching for them. The searching and browsing capabilities of the search area and the world area help children not only in finding the animals they are interested in, but in having questions answered about what animals fall under particular categories, and what animals live in particular parts of the world.

Also key to learning is ease of use. In developing SearchKids the team used a point-and-click interface with no dragging and no double-clicking. We also used large visual targets to help the younger children. In addition, we worked with our child design partners to make sure the animals were categorized in such a way that would be easy for the children to understand.

4.2.6.   Evaluations

The design team conducted a study at a local elementary school to learn whether children could construct Boolean queries using SearchKids (Revelle et al., 2002). The participants were 22 second and 28 third grade children. The children were grouped in same-grade same-gender pairs and were asked to participate in a “treasure hunt”. They were shown how to use the software and had a few minutes to play with it before starting the study. As a task, each child was asked to formulate three queries: one consisting of one element (e.g. “find all mammals”), one consisting of a union of two elements (e.g. “find all mammals and all reptiles”), and one consisting of an intersection of two elements (e.g. “find all mammals that swim”). To score the quality of the queries the team assigned 1 point for every completely correct query; 0.75 points for a two-element query with one correct element and one superordinate; 0.5 points for a two-element query with one correct element, a superordinate for a one-element query, or all correct elements with incorrect additional elements; 0.25 points for a two-element query with one incorrect and one superordinate element; and 0 points for a completely incorrect query. The average scores for the different types of queries were the following: 0.87 for one-element queries, 0.82 for union queries, and 0.86 for intersection queries. These scores suggest children can learn how to use SearchKids’ search area in a short amount of time and are able to pose queries.

In another study, the team learned about the differences between the two types of collaboration SearchKids supports (Druin et al., 2002). The participants were 98 second and third grade children who were grouped in same-gender pairs. Twenty-five pairs used SearchKids in independent collaboration mode, while the remaining 24 pairs used it in confirmation collaboration mode. The task given to the pairs was to find 20 animals in 20 minutes. The sessions were video-taped and input device use was logged. The children’s conversations were coded during the first five and the last five minutes of the sessions. When looking at the frequency of different types of statements, children talked more about the task at hand when using independent collaboration, while they talked more about how to navigate the software when using confirmation collaboration. In analyzing the content of the conversations, the children spoke almost exclusively about shared goals when using confirmation collaboration, while they spoke about equally about shared and individual goals when using independent collaboration. While children managed to find more animals in the list when using independent collaboration, second graders also selected many animals that were not in the list. For example, 42% of second graders selected 9 or more incorrect animals under independent collaboration while only 9% did so when under confirmation collaboration. These results suggest that it is advantageous for children’s collaborative software to provide choices as to which collaboration mode to support. These choices could allow educators to choose the right collaboration mode given the goal of a particular computer session.

The team is currently analyzing the results of a recent study, comparing SearchKids to traditional text search interfaces. The participants in the study were 122 children, 61 from the 2nd grade, and 61 from the 3rd grade. They were asked to participate in a “treasure hunt”, looking for animals in lists. Each child used both SearchKids and text searching and had a set amount of time during which they could use each interface. While there were no differences in the average performance of children with the two interfaces, when looking at the performance of individual children there were some interesting results. About 30% of second graders had at least two more correct answers when using SearchKids than when using text searching. Of the remaining 70% of second graders, 30% performed better when using the traditional text search interface (at least two more correct answers), and 40% performed equally well with both interfaces. Third graders showed similar results, with 25% performing better with SearchKids, 25% performing better with text searching, and 50% performing equally well. These results suggest that children need to be given choices of interfaces as they access digital libraries. They also suggest the SearchKids interface is beneficial to a portion of children over the more traditional text search interface. In addition, it suggests these benefits increase for younger children.

4.3.         International Children’s Digital Library (ICDL)

A natural next step for the digital library interfaces in SearchKids was to apply them to data that was not related to animals. In adapting these interfaces to children’s digital books, a team at the Human-Computer Interaction Laboratory created the International Children’s Digital Library (ICDL). The ICDL, released to the public in November of 2002, currently hosts over 200 digitized children’s books written in 20 languages. The long-term goal of this 5-year project is to host 10,000 children’s books. Unlike SearchKids, the ICDL is available online and has already had thousands of users. 

4.3.1.   Design Roles

4.3.1.1.        Design Partners

The ICDL’s design partners are part of the Human-Computer Interaction Laboratory. The members come from the following disciplines: computer science, visual design, education, and library science. The team also includes 6 to 7 children, 7 to 11 years old. My role as a design partner has been to develop most of the application, besides facilitating and asking the research questions in many design sessions. I also conducted the study described in section 4.3.4.

4.3.1.2.        Informants

There has been no group of informants in the development of the ICDL.

4.3.1.3.        Testers

The ICDL has received help from several testing sites located mostly at school and public libraries.  They had children perform some tasks in the ICDL and reported back to the design team with their observations.

4.3.1.4.        Users

Due to its availability online and a lot of publicity by the press, the ICDL has already had over 10,000 users from all continents except Antarctica. These users sometimes send email to the design team giving valuable feedback.

4.3.2.   Designing the ICDL

Even though the ICDL project did not officially start until the fall of 2002, the design team started to look into adapting SearchKids to digital books during the summer of 2001. The design team decided in the first few design sessions to concentrate on two areas: how to adapt the existing searching and browsing capabilities of SearchKids to the domain of children’s books, and how to support the reading of books on a computer. The team found it necessary that books be read on computer screens, as it would be difficult for public libraries with limited resources to own specialized book readers, or be able to print the books.

The team found ways of adapting SearchKids’ search and world areas to books. Hence, in the ICDL’s initial screen, users have access to the search and world areas, in addition to linking to help (see Figure 15). Figure 15 also shows some of the redesign of the user interface, which now includes buttons for accessing help and exiting the application in addition to the buttons to return to the initial screen and go back to the previous screen.

The team thought the world area could be used to find books based on where they are from, what part of the world they are about, and where they are popular.  In order to organize results in this way the team had to find a different way of displaying results than the one used in SearchKids. The solution was the use of PhotoMesa (Bederson, 2001), a zoomable image browser inspired in part by the results area in SearchKids. Figure 16 shows a screenshot of the ICDL’s world area, and Figure 17 shows a screen showing the results of clicking on Africa displayed using PhotoMesa.


Figure 15: ICDL’s initial screen. The buttons on the top right of the screen (from left-to-right) give access to help, take users back one screen, take users back to the initial screen, and exit the application.


 




Figure 16: World area in ICDL with Africa highlighted by user. The arrow on the right side of the screen spins the globe.


Figure 17: PhotoMesa showing results of clicking on Africa in ICDL's world area.

The team also worked on adapting SearchKids’ search area for the ICDL. The main issue with using the search area was to determine the different categories in which books could be searched. While the team thought of metadata many adults would think of, such as subject and popularity, it also decided books should be searched by the feelings they evoke, and the color and shape of their cover. The group then looked at how books should be categorized by subject.  After doing some research, the group concluded that existing cataloguing methods used at libraries such as the Dewey Decimal Classification and the Library of Congress Classification are not easy to follow by children. The process of finding appropriate categories is ongoing and has been spearheaded by some of the library researchers in the team. The current set of categories is a variation of that used in the Kid’s Catalog (Busey & Doerr, 1993).

During design sessions, the team noticed that in contrast to what happened in SearchKids, intersections of the data represented by icons within a category often yielded results. In addition, the children in the team seemed to expect intersections to occur when they selected more than one icon within the same category.  For example, if selecting a book by the color of its cover, they thought that adding blue and white colors meant they were looking for a book with a blue and white cover as opposed to all books with covers that are either blue or white. I therefore changed the way ICDL queries data by always returning an intersection of the items represented by the icons selected by the user.


Figure 18: Sample query in search area looking for bilingual book written in both English and Spanish that have an orange cover.

  1. User clicks on Language icon from top search categories.
  2. Language icons are shown. User clicks on English icon.
  3. Results show books in English, some icons are disabled because there are no books written in both English and that language. Notice English icon is now on caterpillar. User clicks on Spanish icon.
  4. Results show all books written in both English and Spanish. Notice both the English icon and the Spanish icon are now on the caterpillar. User clicks on “Top Search Categories” hypertext to go back to the top search categories.
  5. User clicks on Color icon.
  6. Color icons are shown; some are disabled because there are no books written in both English and Spanish that have those colors in their covers. User clicks on Orange icon.
  7. Results show only book in library with text in both English and Spanish that has an orange cover. Notice all three selected icons are now on the caterpillar. User clicks on results area.
  8. Results area shows book.

 

Returning intersections made it easier to add a very useful feature: query previews (Tanin, 2001). To give users an idea of how many books to expect from a query, each icon now includes a number specifying the amount of books that will appear in the results area should that icon be added to the query. These numbers are updated every time the current query changes. In addition, icons that if added to the query would not return any results are disabled (i.e. they are grayed out and do not respond to clicks of the mouse). These new features are meant to reduce the frustration of getting too many results or no results. By reducing the number of unwanted queries, these features were also meant to reduce communications with a remote server, since unlike SearchKids, ICDL accesses its data remotely.

Another change in the search area interface was the way an icon with “children” could be added to the query.  In SearchKids, these icons had a “shadow”. Clicking on the icon would add it to the query, while clicking on the shadow would show the child icons (e.g. clicking on the mammals shadow would show icons for apes, canines, cetaceans, etc.). This dual functionality would confuse children. The team decided to change this by having a click on a parent icon always show its child icons, and including one icon among these child icons to represent the parent icon. So now, when users click on the ICDL’s icon representing books about science and nature, they are zoomed into the icon to reveal its “child” icons, plus an icon labeled “all science and nature”. Icons that have child icons are now differentiated from leaf icons (i.e. icons with no children) by a green triangle on their side. The team is planning to change this, as it is too subtle a way of differentiating icons.

Figure 19 shows a screenshot of the ICDL’s search area, which includes options to search books by subject, genre, setting, characters, true vs. make believe, format, length, color, shape, age level, language, feeling, and rating. Figure 20 shows some of the other changes made to the search area. I added a button to quickly start over with a new query. It clears up any existing query and brings users back to see the top search categories. In SearchKids, this used to be more time consuming as users had to click on every icon that was part of the query, and in addition click on an up arrow button several times to get back to the top search categories. The button, which looks like a trashcan, can be seen immediately to the right of the results area. Also noticeably absent are Kyle and Dana, the characters of SearchKids. Since the software only needed one place to show icons that are part of the query, the children helped the visual designers in the team create the caterpillar that now holds them.

Figure 19: ICDL's search area showing top search categories.

In addition, the team decided to simplify the interface by having the back arrow take users one level back on the interface, no matter what path they took through the interface. This means that when in the search area inside a category, the back arrow will take users one level up the hierarchy all the way to the top search categories, and once these are reached will take users back to the starting screen. This made the search area’s up arrow that was used in SearchKids unnecessary.

Another change that helps in the navigation of the search area is the display of the path to the current set of icons shown in the search area. The path is shown above the icons, and provides another way for users to move up the hierarchy by clicking on the path’s hypertext. Besides easy navigation, the path gives users context.


Figure 20: ICDL’s search area showing grayed out icons due to query previews. The all Science and Nature icon is highlighted by the user and shows a tooltip with the full name of the icon.

The team decided to also use PhotoMesa to show the results of queries in the search area. Using PhotoMesa’s grouping capabilities, the search results group the books by subject. Figure 21 shows the results area.

Once a book is selected in PhotoMesa, either in the world or the search area, the book preview screen is shown.  This page shows basic information about the book including its cover, names of authors and illustrators, the number of pages, and a short summary. The page also gives choices on how to read the book using one of the three book readers available in the ICDL. Figure 22 shows a screenshot of the book preview screen.


Figure 21: ICDL’s results area.

 

 



Figure 22: ICDL’s book preview screen.

The team designed these book readers because it felt that in addition to traditional book reader features, children needed extra support for navigation. Partly due to the visual nature of many children’s books, and partly due to the children’s desire to flip through pages in a manner more similar to paper books, the team designed three alternate readers. These readers were designed assuming the books were very visual, and had no more than approximately 100 pages. The expectation was not that any one reader would be best in all circumstances, but that for some children, some books, and some tasks, different readers would be preferable. All readers shared the same controls, allowing users to move between pages, zoom into pages, zoom out, change the position of the controls from horizontal to vertical and vice versa, change the “skin” of the controls, access help, go back to selecting books, and exiting the application. The readers also take keyboard commands, with the left and right arrows and the “page up” and “page down” keys on the keyboard.

The first reader I built uses traditional interface mechanisms, and we will refer to it as the standard reader.  With this reader, a user can see one page at a time (Figure 23).  The main interaction with the user consists of going to the previous or next page. No animation occurs when changing pages. The team decided to build this as the baseline because it provides the same basic functionality as the commercial readers, and does not provide distractions to users who want to read a book page by page.


Figure 23: ICDL’s standard reader with labels explaining what the buttons do.

The comic strip reader lays out book pages as if they were part of a comic strip (Figure 24). When the reader is started, the user sees all the pages in the book. Besides using the buttons and keyboard, the user can interact with the book by clicking on pages. When seeing the entire comic strip, clicking on a page smoothly zooms the user into that page.  If the user is already zoomed into a page, clicking on that page zooms the user out, also accomplishable by clicking on the “zoom out” button.  When zoomed into a page, the left and right arrows take the user to the previous and next page respectively.  The transition between pages animates the view of the comic strip to the left or to the right as appropriate.  If the end of a strip is reached, the view zooms out and then zooms in to the destination page. Clicking on the left or right arrow buttons when seeing the entire comic strip zooms the user to the previous or next page with respect to the last page the user visited.  If the user clicks on the right arrow button before visiting any pages (i.e. when starting the reader), then the view is zoomed into the first page of the book.


Figure 24: ICDL’s Comic strip reader.

The spiral reader is the most novel of the readers I built. It shows the current page in the middle of the screen between two spirals (Figure 25). The spiral to the left shows the pages that come before the current page, while the spiral to the right shows the pages that come after the current page. The spiral reader has the same screen and keyboard controls as the comic strip reader. Using the arrow buttons changes the current page to the next or previous page and animates all pages accordingly around the spirals. Clicking on a page
other than the current page makes that page the current page, animating all pages around the spirals in order to bring the new current page to the center of the screen.  Clicking on the current page magnifies it so it occupies most of the screen in order to read its text more easily. Clicking on the current page again (or clicking on the “zoom out” button) takes the current page back to its normal size.

Figure 25: Spiral reader.

4.3.3.   Information Retrieval Architecture

Besides all the changes in the interface, there was an additional important change in the software in its transformation from SearchKids to the ICDL. While SearchKids accessed a local Microsoft Access database for data via ODBC and obtained the multimedia it displayed from the local disk, the ICDL accesses both a database and the books it provides to users remotely. This was implemented by having the ICDL software make HTTP requests to Perl scripts on a web server that access a MySQL database where the book metadata resides. I setup the ICDL software so it can access this MySQL database while still returning the same type of data it would get from a local database, therefore making it easy to add new types of queries in the future and switch back and forth between local and remote access. Figure 26 shows a diagram of the information retrieval architecture.

The database is designed to handle queries quickly and return query preview information. While the metadata on the books is stored in several relational tables, the Perl scripts access a large flattened table. This table provides a quicker response to queries than join queries. The team recreates this table by executing a query whenever the metadata on books is updated. The large flattened table has a few with columns basic information on the books followed by one column for each category available in the ICDL’s search area. Each tuple in the table represents a book and has the basic information on the book followed by 1’s and 0’s depicting whether the book fits a particular category from the ICDL’s search area. These 1’s and 0’s are used to quickly sum the numbers under each column representing categories from the search area, therefore generating the numbers necessary to update query preview information in the search area icons.


Figure 26: ICDL's information retrieval architecture. The ICDL software may access metadata on books either from a local or a remote database. Book pages are also accessible locally or remotely.

Remote access introduces the issue of latency, but this did not turn out to be as important an issue as I anticipated. The solution was the use of threads to keep the interface responsive while it is waiting for data, and showing progress in the loading of images when waiting for results to show or a book to load.

4.3.4.   Informal Evaluation of Book Readers

I conducted a pilot study to understand if the book readers I developed are effective for children. While the team thought that different users would prefer different readers under different circumstances, we needed to validate this.

4.3.4.1.        Materials

For this pilot study the team used a simplified version of the book readers. The controls used included three buttons at the bottom of the screen. A large left arrow button took users to the previous page. A large right arrow button took users to the next page. In the comic strip and spiral readers, a third button gave users the ability to zoom out to view all pages.

4.3.4.2.        Research Participants

The team used the last design session of the Fall 2001 semester as an opportunity to informally evaluate the book readers. We invited the children in the team to bring their parents, siblings, and friends which resulted in having 16 children, aged 5 to 11 (see Table 3 for ages), from diverse backgrounds, including four children of color. Seven of the children were part of the design team, while the remaining were their siblings or friends, and the children of staff. Nine parents also joined. None of the children, including our design partners, had seen any of the readers working before although two of our design partners were involved with the original design. The team wanted to independently observe the children using the readers, yet there were not enough adult researchers to do this. We therefore paired each child with one of their parents, having the parents observe their children.  If that was not possible, we paired the children with an adult member of our staff.

 

Age (average 8.6)

Number of participants

5

1

6

1

7

2

8

4

9

1

10

5

11

2

Table 3: Ages of participating children.

4.3.4.3.        Conditions

The study had a between subjects design so each child could try all three readers, each with a different book. For the evaluation, the order of the readers changed, but the order of the books stayed the same. The books we used, in the order they were shown, were: Underground Train by Mary Quattlebaum and Cat Bowman Smith (32 pages) (Quattlebaum & Smith, 1997), The Very Busy Spider by Eric Carle (26 pages) (Carle, 1984), and Elmer by David McKee (31 pages) (McKee, 2001). Changing the order of the readers yielded six different conditions. Children were assigned evenly to conditions, with four conditions assigned three children each, and two conditions assigned two children each.

4.3.4.4.        Tasks and Data Gathering

The task assigned to the children was to read the books and answer two content-related questions about each book, such as: “What do the monkeys snack on at the zoo?” (for Underground Train), “Who talks to the spider after the cat?” (for The Very Busy Spider), and “What did Elmer do to look like the other elephants?” (for Elmer). These questions were given to them before reading the books as part of a questionnaire. The children then attempted to answer the questions as they read each book. All the questions were about parts of the books that were midway through the story. After they were done reading a book, the children had to answer two more questions, giving feedback on how they liked the book they just read and the book reader they used. They answered these questions by circling a sad face, a neutral face, or a happy face. After they were done with all three books, the children were asked to select which book they preferred, and which book reader they preferred.

The team also gave the adults paired with children a questionnaire.  For each book reader used, the questionnaires asked the adults whether the children were confused or excited about the reader, and whether they thought the reader helped the children answer the questions. After the children were done with all three readers, the questionnaire asked the adults which reader helped the children the most in understanding the story, and which reader they would prefer the children use.

Besides collecting data from questionnaires, the software logged the use of each reader. The logs kept track of the pages that were visited, zoom in’s and zoom out’s for the comic strip and the spiral reader, and the amount of time spent in each page.

4.3.4.5.        Setup

All research participants performed the study at roughly the same time in two large rooms. Since the participants arrived at different times, they started the evaluation at staggered times. There were enough computers in the two rooms so that there was at least one unused computer between two subjects. Once a child was paired with an adult, an adult researcher led them to a computer, gave them the questionnaires, and explained to them what they were going to do. The adult researchers (not the adults paired with the children) took care of starting and closing each reader. The children were asked to call them when they were done with a book. Questions about how to use the readers were answered if asked by the children or the adults. The adults were told to observe and not tell the children what to do, or operate the computer. A majority of participants completed the evaluation within 30 minutes, and all completed the evaluation within 45 minutes.

4.3.4.6.        Results of Informal Evaluation

Given the informal nature of the evaluation and the small number of research participants, I decided not to use exhaustive statistical methods to analyze the data. Instead, I provide a frequency summary of the observations based on the data collected, together with some tables and charts to illustrate the suggestive results.

The questionnaires yielded information on the preferences of both children and the adults that were paired with them (Table 4). Children’s preferences were similar when it came to book readers. While more children preferred the standard reader, the standard reader was the least popular with children under 10. Adults, on the other hand, had a clear favorite: the comic strip reader. They also thought the comic strip reader helped children the most in understanding the stories they were reading. When it came to stories, children had a clear favorite: Elmer. However, this did not have an impact on the rating of the readers because the readers were rated evenly across all books (Table 5).

Reader

All children

children under 10

adults

adult’s perception of help in understanding

comic strip

4

3

10

8

spiral

5

4

2

1

standard

7

2

4

5

Table 4: Preferred readers for all children, children under 10, adults, and reader adults thought helped children the most in understanding the story (two adults did not respond to this last question).

Book

children’s preference

average rating of reader

Underground Train

4

1.4

The Very Busy Spider

1

1.5

Elmer

11

1.4

Table 5: Book preferences and average rating of readers by book. A sad face was assigned a rating of 0, a neutral face a rating of 1, and a happy face, a rating of 2.

Looking at the answers given to the questions at the story level, most children managed to answer all reading comprehension questions.  Only three children answered questions incorrectly asked about three different stories using three different readers. There were no major differences in the ratings of the readers (Figure 27).

Adult observers suggested that the spiral and comic strip readers were the most confusing to the children, while at the same time they were the most exciting.  This is not surprising, as it is common in visual design for the most exciting visualizations to be the most confusing. Adult observers also thought the comic strip reader helped children the most in answering the content questions. Table 6 shows the results.

 


Figure 27: Reader ratings by children. A sad face was assigned a rating of 0, a neutral face a rating of 1, and a happy face, a rating of 2. Error bars are two standard deviations in length.

Reader

confusing

exciting

helpful

comic strip

4

8

11

spiral

6

7

4

standard

0

3

6

 

Table 6: Number of adult observers that thought each of the readers was confusing, exciting or helpful

Analyzing the logs did not yield major differences between the readers. As a matter of fact, when looking at time spent in each reader, the number of pages visited, and the number of changes of direction, there are no clear differences between the readers across the three books. Nonetheless, I did find two minor trends:

·        On average, children spent more time reading all three books when using the spiral reader.

·        The spiral reader had the most time spent, page changes and changes of direction when reading Elmer.


In both cases though, the differences with the other readers were small, within one and a quarter standard deviations. Figure 28 shows the corresponding charts.

Figure 28: Charts showing log data for time spent (in seconds), number of page changes, and number of changes of direction for each book read under each type of reader. The error bars are two standard deviations in length.

There were also no major differences in the number of zoom in’s and zoom out’s between the comic strip and the spiral reader.  However, the fact that children zoomed suggests that at some point they found it useful to get an overview of the books they were reading. Some children also jumped from one part of the book to another, suggesting again that this type of capability may be useful for some children under some tasks.

Overall, the results of the evaluation show that there is no clear winner. Different children prefer different readers. This suggests that pursuing multiple solutions for book readers was a reasonable path. It also encourages the team to find out more about why children prefer their favorite reader, and what tasks are better suited for each type of reader.

4.3.5.   Learning

Just like SearchKids encouraged learning through information on animals, the ICDL provides learning opportunities by giving children access to books. By using categories that children feel comfortable with and relate to, such as feelings, and the color and shape of books, the interface enables children to find books they are interested in. In addition, the book readers give children options on how to read books, making it fun to flip through the pages of a book and look at pictures, or allowing children to concentrate on reading. In the same way as SearchKids, ICDL offers children interactions that are simple and easy, using only point-and-click and large visual targets. This can avoid frustration and therefore make the books in the ICDL truly accessible to children.

4.3.6.   Use

The ICDL has been available online since November of 2002. During the first seven weeks of use, over 10,000 distinct users used the software. Of those users, roughly two-thirds were from North America, 17% from Europe, 12% from Asia, with very few users from other parts of the world.

Users accessed most books (79%) through the search area rather than through the world area, using multiple category searches 32% of the time. The use of search categories was evenly spread, with no category reaching 20% of preferences. The top five categories chosen by users were: age (child target age), subject, language, genre, and characters. While the average user read only 1.5 books, there were over 100 users who read 10 or more books. More details on the use of the ICDL can be found in Druin et al. (2003).


5.       Study of Young Children’s Mouse Use

5.1.         Introduction

In developing KidPad, SearchKids and the International Children’s Digital Library, I developed some design guidelines based on my experience working with and observing children. I developed the first guideline while observing children use KidPad. An early version of KidPad dropped the current tool if users clicked on the right mouse button, while the left mouse button activated the tool. This confused some children, as they would for example drop a crayon when they wanted to draw with it. From this experience I adopted the guideline of giving every mouse button the same functionality.

Observing children use KidPad, I also noticed they had more difficulty dragging the mouse than moving it. From then on, I decided to avoid using drag-and-drop interactions as much as possible. That is the reason why SearchKids and the International Children’s Digital Library use point-and-click instead of drag-and-drop.

Taking SearchKids to a preschool provided me with another design guideline.  As I observed children at the preschool use SearchKids, I noticed they had difficulty clicking on some of SearchKids’ icons. These icons had not caused problems with the group of children with which I partnered in designing SearchKids, or with the 2nd and 3rd grade informants at Yorktown Elementary School. This led me to conclude that younger children require larger icons.

Searching for empirical evidence backing my design guidelines, I found evidence supporting the use of point-and-click over drag-and-drop (Inkpen, 2001; Joiner, Messer, Light & Littleton, 1998). However, I did not find evidence on children’s use of mouse buttons, or guidelines for sizing visual targets in graphical user interfaces based on age. Because of this lack of evidence and guidelines, researchers have mostly relied on their experience, design partnerships, and on testing to ensure that their designs are appropriate.

While experience, design partnerships, and testing are important elements in the creation of good designs, empirical data on children’s abilities with input devices can help avoid lengthier testing and lend a helping hand to those with little experience. Some researchers have conducted studies to assess these abilities. However, these studies have been mainly aimed at comparing input devices, not at providing guidelines for the design of graphical user interfaces. Section 5.2 provides a thorough literature review of studies on children’s motor skills and proficiency with input devices, and section 5.3 presents the results of a study I conducted to assess the abilities of four and five year-old children with mice. The aim in conducting the study was to provide guidelines for sizing visual targets, and using mouse buttons in graphical user interfaces designed for these age groups. Section 5.4 discusses the relevance and the implications of the results of the study.

5.2.         Literature Review

5.2.1.   Information Processing Speed in Children

As children get older, they improve the rate at which they can process information. Thomas (1980) provides a summary of the research in this area. In the past few years, Kail (1991) has proposed a model for this improvement in terms of reaction time (shorter reaction times equal faster information processing speeds). Equation (1) illustrates Kail’s model:

RTchild = (1 + be-c . age)RTadult                (1)

where for a particular task, RTchild is the predicted reaction time for children, RTadult is the measured reaction time for adults, b and c are empirically derived constants, and age is the age of the children. The ideal population used for determining RTadult is undergraduate students (eighteen to twenty-two years-old), as information processing rates are known to decline as adults age. Other researchers (Fry & Hale, 1996; Miller & Vernon, 1997) have evaluated Kail’s model and found it to fit their experimental data. Figure 29 shows a plot of Kail’s model with RTadult equal to 1, and the values for b and c reported in (Kail, 1991) (b = 5.16, c = 0.21). The values of these constants are still being evaluated, as both Miller and Vernon (1997) and Kail and Park (1992) have conducted further studies for this purpose.

Kail’s exponential curve indicates information processing speed increases more rapidly in young children than it does in older children. This means that young children will show greater improvements in their performance in information processing tasks between grade levels than older children. It also means that the variability in information processing speed for children the same age will be greater for young children than for older children.


Figure 29: Plot of Kail’s model with RTadult = 1, and the values for b and c reported in (Kail, 1991) (b = 5.16, c = 0.21).

While Kail reports children can greatly increase their performance in information processing tasks through practice, the same is true for adults (Kail, 1991). Kail believes there are no differences in the improvement children and adults can make through practice, and therefore practice does not have an impact on his model. He cites a study he conducted which confirmed his hypothesis (Kail, 1991).

Card, Moran and Newell’s model of human performance Card, Moran and Newell (1983), widely cited in the HCI field, explains the relevance of Kail’s model to children’s motor skills. This model of human performance shows that the human motor system depends on processed information from the perceptual system. Research by Schellekens, Kalverboer and Scholten (1984) and Salmoni (1983) has shown that pointing movements, such as those needed to operate input devices are made up of a distance covering phase and a homing phase. Movement in the homing phase is not continuous, but a series of micro-movements followed by micro-corrections (Schellekens, Kalverboer & Sholten, 1984). People with quicker information processing rates will be able to make more micro-corrections in the same amount of time, which translate into smoother motion and better performance. Thomas, in his review, also mentions how information processing rates have an impact on children’s movements Thomas (1980). Based on these models, young children’s performance in pointing movements, such as those performed with input devices should be below that of older children and adults.

5.2.2.   Fitts’ Law

Fitts’ law, a model that predicts pointing movement time based on target size and distance, was developed in the early 1950’s by Paul M. Fitts, an experimental psychologist. Fitts’ law models one-dimensional horizontal pointing movements. It states that pointing movement time is inversely proportional to the width of the target being pointed at and directly proportional to the distance from the center of the target to the starting point of the movement (theoretically, the target is of infinite height) (Fitts, 1954).

The equation that defines Fitts’ law has undergone improvements since its inception (MacKenzie,  1992; Welford, 1968), and this is its currently most accepted form in the HCI community (Douglas, Kirkpatrick & MacKenzie, 1999; International Organization for Standardization 2000):

      MT = a + b log2 (A/W + 1)     (2)

where MT is movement time, A is target amplitude (distance from the starting location to the center of the target), W is the width of the target, and a and b are empirically determined constants.  Other equations derived from Fitts’ law are (3) and (4):

      ID = log2 (A/W + 1)                (3)

      IP = ID / MT                           (4)

where ID is the index of difficulty, and IP is the index of performance. The index of difficulty expresses the difficulty of the pointing task (the same ID may be obtained through different combinations of A and W). The index of performance expresses the quality of the performance of participants pointing under the experimental conditions. It can be used to compare the performances of different groups of people under the same conditions (e.g. children vs. adults), or of people executing tasks under different conditions (e.g. using a mouse vs. a joystick). Sometimes the constant b is used to express similar concepts to IP as it corresponds to the slope of the function tying ID to MT (1/b is roughly equivalent to IP).

5.2.3.   Fitts’ Law Applied to Children

Psychology researchers have been studying how Fitts’ law relates to children for almost 30 years. Through studies, they have shown that Fitts’ law appropriately models children’s pointing movements and confirmed that young children have a lower performance in these tasks than older children and adults (Kerr, 1975; Salmoni & McIlwain, 1979; Sugden, 1980; Wallace, Newell & Wade, 1978). They have also found that younger children show a greater variability in their performance (Kerr, 1975; Salmoni & McIlwain, 1979). Both these observations agree with Kail’s model. Schellekens, Kalverboer and Scholten (1984), and Salmoni (1983) have also confirmed the existence of a distance covering phase and a homing phase in children’s pointing movements. In addition, Schellekens, Kalverboer and Scholten (1984) found the differences in performance between young children and older children and adults occurred in the homing phase, suggesting information processing speeds contribute to the difference. Also of note are Kerr’s findings of no gender differences, and no correlation between the skeletal age of children (assessed by X-rays) and their performance Kerr (1975).

Table 7 shows a summary of empirically obtained data from these studies. Since the data sets are so small, and the age of the adults in the studies is unknown, it is difficult to make any assertions as to whether they fit Kail’s exponential curve.

Study

Age

Empirically derived data

Kerr (1975)

5

a = 564, b = 139 (msec)

7

a = 227, b = 123 (msec)

9

a = 142, b = 108 (msec)

Wallace, Newell & Wade (1978)

4, 5

b = 97.25 (msec)

Adult

b = 43 (msec)

Salmoni & McIlwain (1979)

1st grade

b = 137.9 (msec)

5th grade

b = 99.0 (msec)

9th grade

b = 95.6 (msec)

University

b = 110.1 (msec)

Sugden (1980)

ID = 5.585

6

IP = 5.43 (bits/sec)

8

IP = 6.37 (bits/sec)

10

IP = 7.53 (bits/sec)

12

IP = 8.44 (bits/sec)

Table 7: Empirically derived data from four psychology studies of children’s performance in Fitts’ law tasks.

5.2.4.   Fitts’ Law Applied to Input Devices

While Fitts’ law was developed for one-dimensional tasks, it has been applied successfully to two-dimensional tasks, including selecting items on a computer screen with an input device.  Experiments by various researchers have shown very high correlation coefficients between pointing tasks using an input device and Fitts’ law predictions, as summarized by MacKenzie (1992).

When applying Fitts’ law’s equation (2) to pointing tasks on a computer, its components map to useful information. The constant a, is usually associated with the action taken to select the target, such as clicking a mouse button. The constant b, on the other hand, is associated with the difficulty of using the particular input device for the type of task being performed.  IP is also used for this purpose and has been the choice for comparing the performance of input devices (MacKenzie,  1992).

In the HCI field, Fitts’ law has been mostly used to evaluate and compare input devices.  The first to use Fitts’ law for this purpose were Card, English and Burr (1978).  Through their study, they compared the performance of a mouse, an isometric joystick, step keys, and text keys on the selection of text on a computer screen.  The consequences of this study can still be felt today as most computer users have a mouse sitting next to their keyboards; the same device Card, English and Burr found to be superior.

Scott MacKenzie has been one of the most active HCI researchers with regards to Fitts’ law since the early 1990’s. Perhaps his most important contribution is the proposal of equation (2) (MacKenzie, 1991), currently the most accepted for use in Fitts’ law experiments by the HCI community. He also made a significant contribution by studying how Fitts’ law applies to two-dimensional tasks involving rectangular targets (MacKenzie & Buxton, 1992). He found that in such cases, the smallest of the rectangle’s width and height should be used as the target width in Fitts’ law (or alternatively a measure of width based on the approach angle). MacKenzie (1992) also proposed that HCI researchers follow Welford’s advice (Welford, 1968) in using effective target width (We) for Fitts’ law calculations based on the normal distribution of the coordinates of study participants’ selections of targets.

Since conducting Fitts’ law studies became the accepted way of evaluating input devices, the International Standards Organization (ISO) now provides specifications on how to carry out these studies in the ISO 9241 Part 9 standard (Douglas, Kirkpatrick & MacKenzie, 1999; International Organization for Standardization, 2000). The specifications include equation (2) and MacKenzie’s proposal of following Welford’s advice on using effective width in equations (2), (3) and (4).

5.2.5.   Children and Input Devices

Many researchers have looked at children’s use of input devices in the last decade (Crook, 1992; Inkpen, 2001; Joiner, Messer, Light & Littleton, 1998; Jones, 1991; King & Alloway, 1993; Strommen et al., 1996). They have found high correlations between study data and Fitts’ model (Inkpen, 2001; Jones, 1991). They have also observed how children’s performance with input devices increases with age (Crook, 1992; Joiner, Messer, Light & Littleton, 1998; Jones ,1991; King & Alloway, 1993), and how younger children show a higher variability in their performance (Joiner, Messer, Light & Littleton, 1998; Jones, 1991). Both these findings are compatible with Kail’s predictions. Some researchers have also questioned the usefulness of Fitts’ law when it comes to children (Joiner, Messer, Light & Littleton, 1998; Strommen et al., 1996).

Jones (1991) is not one of them. He has been the only one to study young children’s Fitts’ law performance with input devices. He conducted a study with six, eight and ten year-old children comparing the use of mouse, joystick and trackball input devices in continuous (going back and forth between targets) and discrete (one target at a time) tasks.

The study’s tasks involved clicking on square and rectangular targets all at the same distance, at four fixed angles (up, down, left and right). When users missed a target, they had to repeat the task. They also had to repeat the task if they did not enter the square or rectangle through the side facing the original position of the cursor (this was an unusual requirement).

The study found children improved their performance with age, confirming the observations in the psychology studies and Kail’s model’s predictions. Table 8 summarizes the results for the continuous task with square targets. The ratios between the performances at each age are similar to those found in the psychology studies and to those predicted by Kail’s model (see Table 9).

Age

Fitts’ Constant b (msec)

6

735

8

578

10

510

Table 8: Empirically derived constant b for six, eight, and ten year-olds from Jones’ study for a continuous task with square targets averaged over all input devices used Jones (1991).


 


 

Source

Improvement in performance between ages

6 and 10

6 and 8

8 and 10

Jones (1991)

44%

27%

13%

Salmoni & McIlwain (1979)

39%

n/a

n/a

Sugden (1980)

39%

17%

18%

Kail (1991)

51%

26%

20%

Table 9: Comparison of improvement in performance with age between Jones’ Fitts’ law study (with input devices), two psychology studies, and predictions from Kail’s model.

Jones’ data also showed that younger children had more variability in their performance, as the standard deviation of children’s movement time was consistently higher for younger children. This coincides with the observations of Kerr (1975), and Salmoni and McIlwain (1979), and the predictions of Kail’s model.

As the study was conducted before MacKenzie showed how Fitts’ law works with rectangular targets, Jones took the “depth” of the rectangle with respect to the user’s original location to be the width of the target.  This made Jones incorrectly conclude that Fitts’ law did not apply to children when rectangular targets were involved. As far as comparing input devices, Jones did not find any of the devices to be clearly better than the others.

Another researcher who has looked into children and input devices is Kori Inkpen. Inkpen (2001) conducted a study comparing drag-and-drop versus point-and-click techniques with nine to thirteen year-old children using mice.  While it was not the main goal of her study, Inkpen applied Fitts’ law to her participants’ use of the mouse. She found that the children’s performance was comparable to those summarized by MacKenzie  (1992). She did not look at differences in performance between ages.

Joiner, Messer, Light & Littleton (1998) conducted two studies comparing children’s pointing and dragging. In the second study, children between the ages of five and twelve performed pointing and dragging tasks. The results were that the children’s performance increased with age as the variability in their performance decreased, again in agreement with Kail’s model. Joiner, Messer, Light & Littleton questioned the application of Fitts’ law to children because according to them children are not capable of expert or errorless performance.

King and Alloway (King & Alloway, 1992; King & Alloway, 1993) conducted two studies comparing children’s use of mouse, keyboard and joystick input devices while using an application designed for children. While the researchers did not use Fitts’ law, they did keep track of time to complete the given task. King and Alloway’s participants in the studies were four to eight years old. Children’s performance improved with age, but the variability of performance within an age group was not reported.  Confirming Kerr (1975)’s findings, no gender effects were found.

Crook (1992) conducted a study to find out if young children could use graphical user interfaces. His study concentrated on whether children could manipulate the tools usually found in such interfaces using a mouse. The participants were children aged three to eight years old, plus three teachers with no computer experience, and twelve adult expert users. In a point-and-click task, Crook reported a clear improvement with age (the numeric value of the variability of performance within an age group was not reported). But overall, the children did fairly well, with  second and third graders achieving similar performance as two of the teachers. Given the small sample of teachers though, this finding may not be significant. The third teacher performed significantly better than the other two, at a level comparable to the expert users. This discrepancy could also be due to the age of the two poorly performing teachers (but I do not know because their age was not reported). 

Strommen et al. (1996) studied three year-old children’s use of mouse, trackball and joystick input devices. The study’s task involved moving a cursor to click on targets appearing on different parts of the screen.  The results showed gender differences, as boys were able to click on more targets than girls. This may be due to boys being more motivated towards this goal-oriented task than girls. The inconsistency with other studies (Kerr, 1975; King & Alloway, 1992; King & Alloway, 1993) could also be explained by the fact that this study looked at younger children.

While the joystick ended up being the quickest device (with a slight advantage over the mouse), children entered and left the target more times when using the joystick than when using the mouse or the trackball.  The result of the joystick being faster may be due to the fact that children could press the joystick’s button before getting to a target, and as soon as the cursor touched the target, it would count as a click on the target. This type of button behavior is non-standard and should be avoided in future studies.

Instead of recommending the joystick, Strommen et al. recommend the use of the trackball, which the three year-olds found the easiest to use during the first session of the study, and had the least amount of target reentry. They also argue that the result of the joystick being quickest shows that speed (and by extension, IP in Fitts’ model) does not necessarily equal ease of use when it comes to young children. They furthermore add that while efficiency may be a goal for adults using user interfaces, this may not be the case with three year-olds, for whom play might be more important, even in what appear to be goal-oriented tasks.

5.3.         Study

5.3.1.   Motivation

While the reviewed studies provide some trends in the evolution of children’s abilities with input devices and some specific advice (e.g. point-and-click vs. drag-and-drop), they were not meant to provide specific guidelines for something as simple yet critical as the sizing of visual targets. In order to begin filling this gap, I conducted a study comparing the performance of 4 and 5 year olds with adults in the use of mice in pointing tasks.  I decided to study preschoolers because that is where I expected to find the largest differences between age groups (according to Kail’s model), and by extension, where data from a study would be most useful.

5.3.2.   Research Questions

In order to obtain guidelines, I needed to learn how age impacts children’s difficulty and efficiency in using mice in point-and-click tasks. The research questions I sought to answer through the study are the following:

·        Do age, target size, or distance to target have a significant effect on accuracy (whether the participant presses and releases mouse inside target), or target reentry? What are the accuracy and target reentry rates for each combination of factors that do have a significant effect?

·        Does Fitts’ law model children’s use of mice correctly when first entering the target, last entering the target, pressing the mouse button, or releasing the mouse button? 

·        Does age have a significant effect on Fitts’ index of performance?

·        Are there any patterns in participants’ use of mouse buttons?

5.3.3.   Participants

Thirteen four year-old children (6 girls and 7 boys, average age 4 years and 5 months), thirteen five year-old children (6 girls and 7 boys, average age 5 years and 6 months), and thirteen 19-22 year old adults (6 women, 7 men, average age 20 years and 6 months) participated in the study. The number of subjects is similar to that used in similar studies such as Crook (1992), Epps (1986), Jones (1991), King and Alloway (1993), MacKenzie and Buxton (1992), and Salmoni and McIlwain (1979). All participants were right-handed. The children were a racially and ethnically diverse group from a local pre-school located in the campus of the University of Maryland. The adults were a similarly diverse group of undergraduate and graduate students from the University of Maryland. I decided to include adults in the study because I believe data on children’s performance is more valuable when compared with adult performance tested under the same conditions. Just like the ratio of adult performance using different input devices (e.g. using mice vs. trackballs) holds across different experimental conditions (MacKenzie, 1992), I expect that the ratio of adult to child performance with the same input device will also hold across experimental conditions. I only recruited adults in the ages of 18-22 because this adult age group should provide data on peak adult performance, as adult performance decreases with age (Kail, 1991).

The children had access to one computer in their classrooms (they had to sign up to use it). I asked parents how often their children used computers on a weekly basis and found that among four year-olds 11 of the participants used computers between 0 and 1 hours a week, while the remaining two children used computers between 1 and 5 hours a week. Five year-olds used computers more often, as four of them used computers between 0 and 1 hours a week, eight used computers between 1 and 5 hours a week, and one used computers between 5 and 10 hours a week. Among adults, one used computers between 0 and 5 hours a week, one used computers between 6 and 10 hours a week, three used computers between 11 and 20 hours a week, and eight used computers more than 20 hours a week.

5.3.4.   Materials

I used a Pentium III 650MHz laptop with 128MB RAM running Microsoft Windows 98 at a resolution of 1024x768 pixels. As an input device, I used a Logitech USB Optical Mouse. The mouse produced a displacement of approximately 18 pixels for every millimeter of mouse motion (similar to what I obtained using the “medium” speed setting in Windows with no acceleration). I connected the laptop to a 21” monitor, yielding a control-display ratio of 0.15.

Tasks consisted of moving a cursor from a home area towards a target, and clicking on the target. The targets were red circles and always appeared to the right of the cursor’s starting location in the home area. Tasks ended as soon as participants clicked, regardless of whether this occurred inside or outside the target circle. To start a new task, a researcher initiated a 1.5 second animation of a yellow square from the top of the screen towards a black square representing the participant’s home area. When the yellow square covered the black square, a crosshair appeared in the middle of the yellow square. At this point, participants were allowed to move the mouse (moving earlier would cause the yellow square to restart its animation). Recording of elapsed time did not start until participants moved the crosshair.

To provide feedback to participants, on the bottom left of the screen, a blue bar showed the cumulative elapsed time, a pile of red circles showed the number of hits, and a pile of white circles showed the number of misses. Figure 30 shows a screenshot of the study software.

Figure 30: Screenshot of study software showing the yellow square representing the home area, the crosshair cursor, the target circle, and information on elapsed time and number of hits and misses on the bottom left.

I decided against having tasks one immediately after the other for three reasons:

·        Children have difficulty clicking on mice

·        I was not interested in measuring how quickly children would react to having a target appear somewhere else on the screen

·        I wanted the participants to have a chance to move the mouse to a comfortable position.

I implemented the animation of the yellow square to avoid any mouse motion before the start of a task. I decided researchers should initiate the yellow square animation because of the difficulty young children have in clicking the mouse, and because I did not want them to be distracted by having to press something on the keyboard. With this setup, participants were still the ones initiating the task (by moving the mouse), yet I made sure that before initiating the task the participants: had not moved the mouse, knew where they had to click, and had the chance to move the mouse to a comfortable position. I decided to present targets at only one angle because determining the differences in performance by angle of approach was not one of the goals, and past research has not found large differences in performance at different angles of approach when using mice (e.g. MacKenzie and Buxton (1992) found diagonal motion took 4% longer than horizontal or vertical motion).

5.3.4.   Procedure

The study was conducted in quiet rooms, one at the pre-school, and another at the HCIL. The room the children used was setup with chairs and a table of appropriate height for the children. During the study, participants sat down on a chair in front of a table that had the 21” monitor and the optical mouse on it. A re