Child’s Play: A Comparison of

Desktop and Physical Interactive Environments

Jerry Alan Fails, Allison Druin

Mona Leigh Guha, Gene Chipman, Sante Simms

Human-Computer Interaction Lab, Institute for Advanced Computer Studies

University of Maryland, College Park, MD 20742 USA

fails@cs.umd.edu, allisond@umiacs.umd.edu

ABSTRACT

The importance of play in young children’s lives cannot be minimized. From teddy bears to blocks, children’s experiences with the tools of play can impact their social, emotional, physical, and cognitive development. Today, the tools of play now include desktop computers and computer-enhanced physical environments. In this paper, we consider the merits of desktop and physical environments for young children (4-6 years old), by comparing the same content-infused game in both contexts. Both quantitative and qualitative methods are used for data collection and analysis. In this paper we also discuss the process of working with children of multiple age groups to develop the physical computer-enhanced environment.

Author Keywords

Children, educational applications, games, stories, desktop, physical interactive environments

ACM Classification Keywords

H5.m. Information interfaces and presentation (e.g., HCI): Miscellaneous.

INTRODUCTION

I played some games, then ate lunch, then swam with my cousin, went home to dry off, played some more games, ate dinner, and then played games until bedtime. – a six-year-old girl describing her day’s activities

A child’s work is play — and they work hard [35,36]. Play is an integral part of children’s lives. According to many prominent researchers, as children play, they develop cognitively, socially, and physically [30,35,39,40,23]. They learn via exploration, trial and error, collaboration, experimentation, role-playing, and pretending. This has also been evidenced by the recent strong emphasis on play in the curriculum of U.S. preschools [28].

Many people have explored how technology can be used to augment child’s play [37,38,43,44,45,34]. Our study furthers this research by comparing the use of desktop and physical interactive environments by preschool-aged children (ages 4 to 6). The contributions of this paper are threefold: a new design methodology with children; new technologies for content-infused story games; and, the first exploratory comparison of desktop and physical environments for preschool-aged children. These contributions revolve around the creation of the Hazard Room Game — a game that focuses on teaching children about environmental health hazards. The game includes role-playing story activities with potential hazards such as lead, particulates, pesticides and phthalates.

The first contribution consists of a design methodology where adult designers partner with children who are the target age users for the technology (ages 4-6) as well as with children slightly older than the target age (ages 7-11). Typically methods which involve children as design partners primarily involve the target age children in consulting or participatory roles. Here we also partner with children who are slightly older than the target age, affording advantages that are further discussed in later sections on the Hazard Room Game.

Using the ideas designed by our team, we built technologies that supported content-infused story games. The game consisted of two modes: exploring and learning. In the explore mode, children could explore, search and find story elements. In learning mode, children would hear stories emphasizing specific content knowledge. Thus the Hazard Room Game became a content-infused story game (referenced as CIS Game hereafter). Although this concept is not necessarily a new educational approach, our second main contribution includes developing technologies for the CIS game to be played virtually on the desktop or physically in a room with computer-enhanced objects. It should also be noted that although the specific content in the CIS game was environmental health hazards, the technologies we developed are not constrained to this specific domain. They can generalize to any number of content areas. For example, our technologies could be used to teach children about bugs and include various stories and props about ants, caterpillars, and butterflies.

The third contribution is a comparison between desktop and physical interactive environments. Physical interactive environments are places where interactive technology is embedded into physical objects [43,44,45,33,34]. These environments have sometimes been categorized with ubiquitous computing systems which enable users to perform digital manipulations in everyday environments [34,43].

RELATED WORK

Many people have explored how technology can enhance learning during child’s play [24,37,15,34]. The role technology can and should play in the education of young children has fueled much debate among researchers [27,29,31]. Some have postulated that technology suppresses a child’s creativity and even adamantly decry computers for young children, stating that technology should be completely avoided until children are “old enough” [29]. Contrary to these views, we firmly believe, along with several others whose work is cited hereafter, that technology should, when appropriate, be used to augment children’s learning. Despite its polemic nature, many have successfully ventured into the realm of technologically enhanced educational systems for young children. The main categories of research related to this study are the role of stories in learning, games and gender, desktop applications for children, physical environments, and CIS games. These topics are discussed in the following subsections.

Role of Stories in Learning

There is an Indian Proverb that says:

Tell me a fact and I’ll learn. Tell me the truth and I’ll believe. But tell me a story and it will live in my heart forever. [21]

This proverb is supported by a The (U.S.) National Council of Teachers of English (NCTE) Guideline which states that teachers discovered that children could easily recall whatever historical or scientific facts they learned through story [11]. Indeed, stories, as well as narrative exploration, are excellent for learning and teaching [21]. Stories meet the diverse needs of children [52], greatly facilitate language learning [25], and help children construct an image of self [10].

Because stories are such a powerful teaching tool, many systems have been created to foster story creation and sharing. Some examples have already been cited, however, there are a few in particular that seem to form a bridge between the physical and desktop environments. A few of these systems use a desktop to create, and a stuffed animal for sharing and reviewing previously created stories. SAGE [4,49] and Rosebud [18] developed at the MIT Media Lab and PETS developed at the University of Maryland [41] are three such examples.

Games and Gender

Games are an integral part of child’s play. Children love games [23]. Their “games” do not necessarily have to be the goal oriented games that adults generally play. Children’s games often have more to do with repetitive, simple activities as in Simon Says or Hide-and-Go-Seek [19].

Games can also be used for educational purposes. Renowned educator John Dewey has described the educational principles for teaching as: engaging children in a fun and playful environment, imparting educational content, and ideally sparking interest in learning more about the subject [12]. Indeed this is the purpose behind the Hazard Room Game.

In game research, there is also a lot of discussion about gender differences — especially for young children [10,23,19]. Most of the research addresses an apparent lopsided male interest towards the majority of games on the market. Researchers have found that there are many reasons for this imbalance between the genders when it comes to games. Some note the types of characters are not appealing to girls, others assert that popular “shoot-em-up” games are too superficial and/or repulsive [10]. This imbalance however has been countered by games targeted specifically for girls [50,10]. Psychologists have noted that for young children, the organized games of girls are simpler in their rule structure than are the games of boys, and require a lesser amount of physical skill. They also note that boys play games in larger groups. Also relevant to the CIS Games is psychologists’ assertion that girls seem to be slightly ahead of the boys in their ability to initiate fantasy play without the benefits of realistic props [23]. The discussion of differences broaches the age-old dichotomy of nature versus nurture, and much of the research notes how gender roles are greatly influenced by the nurturing received through the culture in which the children are raised [10]. In summary, there are gender differences to be recognized in this area of study.

Desktop Environment

The traditional desktop environment has been employed for educational purposes since its inception. Although the desktop offers a great deal of software that drill or test children on subject matter, one of the early software tools for children which employs a different approach and has been conceptually influential is, Logo [37]. Logo and its descendant StarLogo [42], as well as the relatively new Squeak [48], are educational platforms and programming environments that can lead to exploring the conceptual worlds of math, science and more. When using these environments, children usually start from example, and then manipulate and make changes to discover new mathematical and scientific concepts in action as they build new programs. These technologies support the educational approach called constructionism, which suggests children build as part of their educational process, and by doing so, can build their own conceptual models of the world around them [37,38]. A more traditional perspective of explorative educational approaches is called discovery learning. Constructionism and/or discovery learning (depending on your point of view) also play an important role in narrative storytelling systems, which have been implemented for both physical and desktop environments. Example of some of the open-ended narrative storytelling applications include: Hayes-Roth Improvisational Puppet system [22], MOOSE Crossing [7], StoryBuilder [2], and Renga [8,9].

There are numerous desktop “educational” products available for purchase. One need only walk into any software store to notice the barrage of programs touting to be educationally appropriate for young children. There are also popular websites like pbskids.org, noggin.com, googles.com. These websites have unique designs created for children, which is a necessary component for desktop applications for young children [15]. Some programs and websites utilize constructionism and discovery learning, however, the majority use drilling, quizzing, and games as their educational mechanism.

Physical Environments

Ubiquitous computing has become a popular paradigm that enables computational accessibility in everyday settings [17,26,20,16]. Recently this technology has been employed to aid the teaching of young children [43,44,45,33,34,1]. Typically, physical educational environments are revered as more natural and explorative environment for young children, since children generally play with and learn in physical environments. There are several physical educational learning applications that emphasize narrative storytelling, which are discussed more in the section of the role of stories in learning [34,1,5]. An example of these is StoryRooms, which enables children to create a story with technological props [33,34]. The child programs using physical objects a sequence of actions that correlate with their story, thus facilitating sharing, collaboration and reproduction. The children also benefit indirectly from the discovery of basic programming concepts [34].

Desktop vs. Physical Environments

Desktop and physical educational environments have been shown to aid learning [37,38,34,33,42,1], however, most studies stop there. To our knowledge there has been no study comparing desktop and physical environments for children. Researchers have studied different desktop collaborative configurations [46,47]. There have been many studies on how desktop environments can enhance learning [37,42,44]. Physical environments have asserted theoretical advantages on the grounds that children interact better in the physical world than in the two-dimensional space of the desktop [34]. Others still have attempted to bridge the gap by having physical and desktop components [18,4,49]. There is extensive literature about each individual environment, but it appears as though direct comparisons have been avoided. Herein we investigate the desktop and physical educational environments in direct comparison in the domain of young children’s learning.

Content-Infused Story Games

The concept of content-infused story games (CIS Games) is not necessarily new but builds upon other story game technologies [18,41,4,49]. Its basic premise is that games and stories can be combined to educate children. In CIS Games, stories are used directly to indirectly teach particular content. Because of their generality, CIS Games can be effectively implemented in both desktop and physical environments.

Many of the aforementioned approaches, both in desktop and physical environments, use creative story narrative as the educational mechanism. These systems implement constructive methods [37]. Although we support these open-ended approaches, we also believe there is a need for directed learning as well. CIS Games are a mechanism for teaching directed material that uses stories that were created before the game is played.

From a pedagogical standpoint, CIS Games use most of the recommended effective approaches as designated by The National Research Council. The four it specifically employs are: direct instruction, teaching through play, teaching through structured activity, and computers [6]. The other two are: engagement with older peers and child-initiated instruction. Both of these could be incorporated into CIS games — the former, by including multi-age dyads, and the latter by allowing the children to dynamically create their own CIS Games.

EVOLUTION OF THE HAZARD ROOM

The CIS Games we created are the product of a long evolution involving 35 children, ages 4-11, over an 18-month period. The specific target audience for these technologies was young children, ages 4-6. In developing the structure with children, we focused on environmental health hazards as our domain. The hazards that were covered in the game were lead, particulates, pesticides and phthalates. This domain was chosen due to the current need for such information in the Washington D.C.-area where lead in drinking water has been a daily occurrence. To develop the appropriate content we partnered with the Children’s Environmental Health Network (CEHN), a non-profit organization concerned with public education and policy development on these issues.

The evolution of Hazard Rooms follows a timeline of about a year and a half of discussion and prototyping with children of both the target age, and children slightly older. The evolution began with the children that were slightly older than the target users. These seven children, ages 7-11 used low-tech prototyping methods [13,14] to sketch new ideas with adults. Using bags of art supplies with teams of two to three children and two adults may ideas were suggested, such as: using x-ray glasses, carrying a magic lens to inspect items, and even being inside a body and exploring the damaging effects of the hazards. All of these initial ideas included exploring an environment and the ability to receive feedback when faced with a potential hazard. In further brainstorming the team’s ideas soon evolved to be environments filled with stories that had props identifying its major parts. Three, ninety-minute design sessions were accomplished over a two month period each helping to hone in on a particular set of design directions. (See Figure 1 for pictures of methodologies in action.)

Figure 1 Design methodologies for children ages 7-11; Left – brainstorming with bags of art supplies; Right – compiling ideas using sticky notes

Once these initial sessions with slightly older children were accomplished, we then worked with 17 younger, target-aged users. With these children, we brought in low-tech prototypes for them to elaborate on. For example, we gave children a hat and magnifying glass (also called a “Magic Lens”) and asked them how they could be used to tell or show you what the hazards were. We also brought in props that represented the health hazards and asked the children to tell us how other children would know they are bad (see Figure 2), an idea suggested by the older children. In each of these design sessions, four or five children and three or four adults worked together to brainstorm ideas. The ideas that emerged from these sessions over a three month period included the need for auditory and visual feedback. Specifically, the children suggested X’s on bad things, happy and sad faces and “yeah” and “yuck” labeling good and bad, and video vignettes showing what could happen if someone interacted with a hazard.

Figure 2 — Young children (ages 4-6) elaborating hazard stories from physical props.

Throughout this process the adult team acted as the liaison between the older and younger children by melding and sharing the ideas of both teams. The older children’s suggestions were subsequently incorporated in design sessions with the younger children. The resulting idea was a physical story environment where a child could learn about “good and bad things”. During this time, we built low-tech prototypes of the environment. Having collectively built a story learning environment, we then returned to the slightly older children who built upon it and made the story environment into a game.

Each time we worked with the older children, we used participatory design techniques to write or sketch stories. With these children, we always began with the open-ended problem of teaching other young children about environmental health hazards. However, with the younger children we constrained our brainstorming activities to elaborating on one or two aspects relating to the older children's ideas. With the younger children, we started with a physical object for our discussions and brainstorming. With something concrete to bridge our discussions, the children were quickly able to focus on the task at hand.

This cooperative inquiry variant builds upon participatory design, but enabled us to work with two sets of children without ever having to physically schedule them to be in the same place at the same time. The involvement of the slightly older children also enabled us to work with young people who could simultaneously consider the impact that these new technologies could have on different children, as well as to perceive how others may interact with the application [39,40]. A theoretical advantage to this approach is that perhaps, since they are further developed than the target users, the older children would inadvertently help design a system that provides a scaffold to deeper understanding, creating a Vygotsky-type bridge into a zone of proximal development [32,51].

It is important to mention two side notes relevant to the evolution of both desktop and physical versions of the Hazard Room Game. The older children we worked with had predispositions towards physical environments due to their involvement in the design of an earlier physical technology. Although the younger children did not have this prior experience, they too suggested a physical environment solution. Even though the children seemed to favor physical environment, since our goal was to widely distribute the final product we also developed a desktop version. This, along with the fact that there are several desktop educational applications, led us to question and eventually further study the merits of a desktop game versus a physical interactive environment.

HAZARD ROOM GAME

The Hazard Room Game has been implemented in both desktop (Figure 3) and physical interactive environments (Figure 5). Both environments use the same game structure, the same props (or pictures of the same props), the same hazards and the same stories. The props are located in the same manner around the periphery of the space, real or virtual.

Incorporated into the stories of both instances of the Hazard Room Game are the hazards lead, particulates, pesticides, and phthalates. The game is designed to be engaging as well as teach the children about what dangers the hazards pose and how to avoid them.

The game is played with two teams, each team having two players [46,47]. During the game, teams are exposed to two different stories for each hazard. Each story contains information about the hazard and explains what to do in that specific situation. Stories are heard as sound segments. For example, one of the stories says: “You get an apple. You wash the apple because the apple can have pesticides on it. Pesticides make you sick. You eat the apple”. In this story, the potential hazard is pesticides and it uses three props: apple, sink, and mouth.

During the game, teams alternate turns to fill two different roles: the I Spy team and the Finder team. The I Spy team helps the Finder team locate the props for the current hazard story. They do this by giving an initial clue, followed by locational clues for each prop. The Finder team explores the room, finding each prop, and depositing them in a special area defined as the Story Corner. There are three props associated with each story. Once all the props are collected for a story, all players hear the full story. The Finder team then puts the props in order as they were mentioned in the story. The technology then notifies the children whether they sequenced the props correctly or not. If not, the children listen to the story and order the props until the sequence is correct. Correct ordering of the props is indicated by the order the props occurred in the story. For example, in the apple story the correct ordering is: apple, faucet, mouth.

As described above the Finder team is the team that experiences the two modes. First, they explore, navigate, and collect the props; and then, they learn via the stories and sequencing.

Desktop Game

In the desktop Hazard Room Game, the two teams are connected via the Internet. The “room” in the desktop version is a window that occupies the full screen of the computer (see Figure 3). Each team has their own computer, which is not visible by the other team. Teams navigate the play area by using the arrow keys. Locational clues are given via buttons on the keyboard. These clues indicate whether the team is warmer (closer) or colder (further) from the props. As soon as the Finder team finds the object they are looking for it moves to the story corner. Once all objects are found sequencing is performed in the story corner which is zoomed in to fill the full screen (see Figure 4, Left).

Figure 3 — Screenshot of the desktop Hazard Room Game.

Once sequencing is completed correctly, the teams switch roles, so each have alternate opportunities to the by I Spy and Finder teams.

Figure 4 — Left: Screenshot of desktop story corner during sequencing; Right: Close-up of physic story corner

The desktop Hazard Room Game was built for the Windows platform using the Piccolo Toolkit [3] to enable quick development of a game that included interactive zooming and animation. The game, being a two-teamed game, was designed for two computers that could share a TCP/IP socket for communication.

Physical Game

Similar to the desktop version, the Finding team starts out in the center of the physical play area (shown in Figure 5). Teammates navigate by holding hands and walking around the play area. Once an item is found, the team places it by the story corner, where the sequencing bins are. A close-up of the story corner is shown on the right of Figure 4. Similar to in the desktop version, the children perform sequencing by placing the props inside the story bins.

The physical Hazard Room Game uses radio frequency identification (RFID) readers and tags. Each prop has an RFID tag associated with it; each story bin has an RFID antenna and controller that is connected serially to a wireless modem (BlackMagic [33]) that was engineered, built and tested in our lab. The wireless communication for each bin communicates to a Windows computer that has a Java application that orchestrates the correct sequencing and the sound clips.

Figure 5 — Photo of the physical Hazard Room Game.

EXPERIMENT AND ANALYSIS

To compare the desktop and physical environments the hazard stories were divided into two sets, Set A consisting of lead and phthalates, and Set B, particulates and pesticides. The sets were chosen to be similar in ease of learning by Daniel Schwarz, the former Director of CEHN and a content expert. The quantitative analysis showed no effects by hazard set, demonstrating that this was achieved.

The study participants were children ranging in age from 4 to 6. None of these children had ever seen the technologies tested or were among the design team that helped develop the technologies. Eight teams of two children each were randomly selected to counterbalance age effects. The random selection was constrained to have a boy and a girl on each team, to counterbalance gender effects. Two teams were then randomly assigned to a group, forming four different groups. Each group used one environment (desktop or physical) with either hazard Set A or B, for a session. They then used the other environment with the other hazard set in another session a few days later. The experimental design therefore was a two-by-two, fully counterbalanced cross with hazard set (A or B) and application type (Desktop or Physical), for a total of four treatments. Each treatment involved two teams of two, or four children. Each treatment consisted of two sessions. Each session consisted of two hazards with each hazard having a story for each team, resulting in four stories per session.

During each session four adult moderators were present, one for each child present. As such, each moderator administered the pre and post tests one-on-one with a child. During the sessions, there was a main facilitator (who explained procedures and moderated discussion), one person video taping, one taking notes, and another monitoring the technology.

The metrics of comparison were twofold: qualitative and quantitative. The quantitative metrics were measured via identical pre and post tests administered before and after each treatment. Since the sample size was small, we anticipated that the qualitative measurements would be more revealing. As such the quantitative analysis was performed to provide descriptive statistics to accompany the qualitative findings. The qualitative analysis is based on notes and video coding of questions asked by adult researchers after each, correct story sequencing.

Quantitative Analysis

The quantitative analysis consisted of scoring the pre and post tests and evaluating the difference between each participant. The tests consisted of six multipart questions which were administered verbally; the duration of the tests took approximately one to two minutes. The first four questions gave scenarios similar to those in the stories and asked the children what they would do and why they would do it. For example, recall the apple story from the Hazard Room Game. Each child was asked: “What should you do with an apple before you eat it?”, and then “Why should you {initial response} before you eat the apple”. For the last two questions, the child was asked which of a list of items could have the potential hazard. For example, “Which of these could have pesticides on it? … Water … Pacifier … Apple … Plastic Toy … Carpet”. A response was awaited after each item was read.

The tests were then scored and the differences between participants’ pre and post tests were used for analysis. For scoring the first four questions equal points were awarded for each of the three-part questions. The last questions were awarded a point for a correct answer and a half a point deduction for a false positive. With this scoring scheme, a completely correct test had a score of 16. Thus the largest possible difference between pre and post tests was 16. The differences between participants’ pre and post tests were then used for analysis.

The most important comparison we hoped to make was between the desktop and physical environments. As expected, none of the quantitative results were statistically significant due to the small sample size. Given the effect size that was observed between the desktop and physical interactive environments, the power analysis exposed that at least one hundred participants would be necessary to obtain significance. Such a large study was infeasible due to lab resources, thus necessitating qualitative analysis. The descriptive statistics will be used in concert with the qualitative observations to illustrate their correlation.

Another factor that could have impacted the quantitative analysis was our quantitative measurement — the pre and post tests. A measurement analysis showed that there was a small positive correlation on the pre tests (~0.49) and only slightly more on the post test (~0.73). The small correlation on the pre test was expected, as the children guessed, however, we hoped the correlation would be higher on the post test. We feel the reliability of the pre and post tests could be increased by lengthening it slightly and adding more redundancy. This would not impact the children too much, because the test could still be completed in two to four minutes if it were doubled in size.

Qualitative Analysis

The qualitative analysis was based on notes and video taken during verbal questions asked by adult researchers after each correct story sequencing. The two questions asked were:

“Tell me the story in your own words?”, and
“What did the story teach you?”

Coding Procedures

After reviewing notes and video of these qualitative questions, a coding scheme was developed. The coding scheme for each story included the number of times adults verbally prompted after each of the two verbal questions, as well as coding behaviors of each child throughout the questions. A prompt consisted of a question or comment from an adult facilitator that had the intention of eliciting a response from a child participant. Each prompt was separated either by a response from the children or a pause of 5 or more seconds.

For each child within a story, video evaluators made five observations:

Whether or not the child gave a verbal response
How many times the child communicated “I don’t know” either verbally or non-verbally (e.g. such as shrugging their shoulders)
The depth of response as defined by:
1. Incorrect or no response
2. Rote or slightly reworded, or just identifying the sequence of props
3. Processed or mostly reworded
4. A processed response including the causal effect (e.g. “You should wash the apple before you eat it because it might have pesticide on, which could make you sick.)
A frequency of the interaction types (pointing or touching of props) to identify which happened most frequently
A subjective interest level for the child on a scale of one to five, where one is very disinterested, and five, very interested

Findings

To establish reliability of the coding scheme, three different evaluators coded 25% of the stories on video. The inter-rater reliability was 79%. The subjective interest question had the highest variability among the raters, in fact only 40% of the responses were exactly the same among all three raters. The average standard deviation for each rating was 0.37 meaning that mostly what occurred was that one rater chose a rating that was one off of the other two. Having correlated a high percentage of exact answers and verifying the consistency of the qualitative measures, one researcher then coded the rest of the video.

The results of the coding yielded several advantages for the physical environments over the desktop environments. Specifically, there are four measures that indicate advantages for the physical environments. First, the number of prompts necessitated by the facilitators was 20% fewer in the physical environment. Second, the answer depth increased by 18% in the physical. Third, the number of “I don’t know” responses was reduced by 14% (counting only people) or 29% (counting multiple responses by the same person). Fourth, the average subjective interest was 16% higher in the physical than it was in the desktop case. These four measures collectively suggest that these children may have been more engaged and that they qualitatively learned more in the physical environment than in the desktop environment. These qualitative results agree with the descriptive statistics where the mean score differential between pre and post tests was greater in the physical (3.63) than in the desktop application type (2.69).

In addition, we also noticed some gender differences in our overall findings. The most marked gender distinction was manifest in the type of responses given by girls and boys. Boys tended to respond with rote responses, mimicking the stories almost word for word, but leaving out the causal effects. This was witnessed in observing many segments where the boys simply stated the ordering, but the girls would actually reword the story when asked to restate the story in their own words. The qualitative results dramatically illustrate this finding as the boys received rote depth ratings (code number two above) almost four times more than the girls did (15:4). Conversely, girls tended to respond with full causal depth (code number four above) twice as much as the boys did (15:8).

Gender differences were also evident when analyzing the data by environment. The girls verbalized more, as the boys pointed and touched the props considerably more both in the physical and desktop environments. The aggregate ratio of male to female point and touch interactions was 9:2). Not surprisingly touching the props occurred more in the physical environment, although, some children did point and touch the computer screens as they explained the stories in the desktop environment. In the physical environment, the ratio of touching the props again favored the boys, echoing the overall ratio of 9:2.

The quantitative descriptive statistics also pointed to a gender difference, which was not distinguished in the qualitative analysis. Comparing the difference between pre and post tests, the girls performed better on the physical (4.63) than on the desktop (2.19), whereas the boys were about the same on both physical (2.63) and desktop environments (3.19). This difference was not observed as strikingly in the qualitative results. Only minimal benefits less than 10% improvement) were observed in average depth and average engagement (using average interaction and subjective interest).

LESSONS LEARNED

Throughout the process of designing, developing and testing the Hazard Room Game, we have learned a number of lessons concerning design methods with children, the types of technology environments in the area of storytelling and the use of these environments by children. We present these lessons below as suggested guidelines for others building technology with young children.

Physical environments may have many empirical advantages over the desktop

Our qualitative study showed the physical environment to have many advantages over the desktop environment. These were namely interest and engagement, under the assumption that lack of “I don’t know” responses, fewer prompts and depth of response are representative of interest. This might suggest that we should consider embedding technology in the physical world, rather than simply presenting them with traditional desktop applications.

Gender may effect the types of interaction with the different environments

Although our qualitative study did not yield significant or authoritative differences by gender and application type, the observational data suggests that such differences may exist. We suggest further investigations into the differences that may exist between gender and educational, computational environments for young children, as this may benefit early childhood education.

Storytelling technologies, whether desktop or physical, can be used to teach children specific content

Even though there were varying differences across gender and environments, in nearly every case all children learned the content that had been infused into the games. This shows that, for at least this content, CIS games appear to be effective teaching mechanisms. We believe this probably generalizes to most other content areas; however, there may be content for which either the desktop or physical environment is a better fit.

Embed technology in the environment, rather than having children carry a hand-held, turn-taking device

As mentioned in the evolution of the Hazard Room Game, at one point we experimented with use of a “magic lens” which allowed the children to see and explore the environment. It was also used as a turn-taking token. This was not effective as the children were confused by having to hand the device back-and-forth between the teams. Instead, we opted for adding instrumentation to the environment, thus invalidating the need for a hand-held, turn-taking device. The children were very pleased with this modification and correspondingly were not confused. This may be due only to the device being used as a turn-taking token. Further studies are needed to generalize this lesson, however, we can and do encourage dispersion of the technology over requiring children to carry a device in situations were the device will be frequently exchanged between the children.

Slightly older children facilitate the design process

By adding slightly older children to the design process you can speed up design time as the older children can better articulate their ideas. They can write, draw, and cognitively process ideas more maturely. Also, the 7-11 year-olds that aided have the ability to perceive how others may interact with the technologies, whereas young children (ages 4-6) simply have not developed to the cognitive stage necessary to envision other peoples’ perceptions.

Provide ample support and focus for young children

Working with young children affords many opportunities and challenges. A significant challenge is to harness their creativity and energy. This is best accomplished by using a large team of adult designers, keeping the child to adult ratio low. Also important is keeping the children focused. For young children to be able to focus, the adult facilitators must help maintain that focus by defining the tasks well, and using physical props and concrete examples as a bridge for discussion and brainstorming.

CONCLUSIONS AND FUTURE WORK

Although we observed many interesting trends, the results must be interpreted with caution. The small sample size limited the analyses. Despite the small sample size, the high correlation of several, different measurements indicating advantages for physical environments over desktop environments, implies there may be an overall advantage. We suggest more research in this interesting area of computing with younger children, and we hope to build upon this.

Several ideas for future work in this area include conducting larger and more longitudinal studies. A larger sample size, a longer test, and a longitudinal study where children are asked the test questions at a later time (to test retention) will help address the limitations of this study due sample size. A larger sample is also needed to better understand the relationship between gender and application type (desktop and physical environments). Another big question is whether the type of testing affected our findings. In the future, we would like to ask children similar test questions in a real-life physical environment, instead of a short verbal test. By acting out the story, instead of just responding to verbal questions, the testing results may lead to a better understanding of the advantages and disadvantages of desktop and physical interactive environments for young children.

In conclusion, we view our findings as a springboard for more and continued research on the importance of desktop and physical environments for young children.

ACKNOWLEDGMENTS

We are grateful to all of the children who partnered with us to design or test the technology over a two-year period. We also thank the teachers and parents who supported and allowed this research to be conducted. We especially thank Daniel Schwartz for his support and domain expertise in helping us to create the Hazard Room. We thank Laura Hadley for aiding in the coding of video segments. We also thank Karen Samuelson for her kind assistance analyzing the quantitative data. This research was generously funded by an NSF Career Award over the last four years.

REFERENCES

1. Alborzi, H., Druin, A., Montemayor, J., Platner, M., Porteous, J., Sherman, L., Boltman, A., Tax’En, G., Best, J., Hammer, J., Kruskal, A., Lal, A., Plaisant-Schwenn, T., Sumida, L., Wagner, R., and Hendler, J. Designing StoryRooms: Interactive storytelling spaces for children. Proceedings of the Designing Interactive Systems (DIS-2000) (2000), 95-104.
2. Antle, A. Case Study: The design of CBC4Kids’ StoryBuilder. In Small Users – Big Ideas: Proceedings of Interaction Design and Children 2003 (IDC 2003).
3. Bederson, B. B., Grosjean, J., and Meyer, J. Toolkit Design for Interactive Structured Graphics. IEE Transactions on Software Engineering, 30(8), 535-546.
4. Bers, M., Ackerman, E., Cassell, J., Donegan, B., Gonzalez-Heydrich, J., DeMaso, D. R., Strohecker, C., Lualdi, S., Bromley, D., and Karlin, J. Interactive Storytelling environments: Coping with cardiac illness at Boston’s Children’s Hospital. Proceedings of the Conference on Human Computer Interaction (CHI 1998) (1998), 603-610.
5. Bobick, A., Intille, S. S., Davis, J. W., Baird, F., Pinhanez, C. S., Campbell, L. W., Ivanov, Y. A., Schutte, A., and Wilson, A. The KidsRoom: A perceptually-based interactive and immersive story environment. In PRESENCE: Teleoperators and Virtual Environments (August 1999), 367-391.
6. Bowman, B. T., Donovan, M. S., Burns, M. S., and the National Research Council. Eager to learn: Educating our preschoolers. National Academy Press: Washington, DC, USA, 2001.
7. Bruckman, A. MOOSE Crossing: construction, community, and learning in a networked virtual world for kids. Unpublished thesis, MIT, Cambridge, MA, 1997.
8. Cassell, J. and Ryokai, K. Making space for a voice: Technologies to support children’s fantasy and storytelling. Personal and Ubiquitous Computing (January 2001).
9. Cassell, J. Renga the Cyberstory. http://renga.www.media.mit.edu/ (Sited 04-Aug-2004).
10. Cassell, J., and Jenkins, H. From Barbie to Mortal Kombat: Gender and Computer Games. MIT Press: Cambridge, MA, USA, 1998.
11. Commission on Storytelling. The National Council of Teachers of English (NCTE) Guideline: Teaching Storytelling. http://www.ncte.org/about/over/positions/category/curr/107637.htm (Sited 04-Aug-2004).
12. Dewey, J. Experience & Education. Macmillan Publishing Company, New York, NY, USA, 1963.
13. Druin, A. Cooperative Inquiry: Developing New Technologies for Children with Children, in Proceedings of CHI '99 (Pittsburgh, PA, May 1999), ACM Press, 592-599.
14. Druin, A. The Role of Children in the Design of New Technology. Behaviour and Information Technology, 21, 1 (2002), 1-25.
15. Druin, A. (Ed.) The Design of Children’s Technology. Morgan Kaufmann, San Francisco, CA, 1998.
16. Fails, J. A., Olsen, D. R. “Light Widgets: Interacting in Everyday Spaces.” Proceedings of Intelligent User Interfaces (IUI ’02) (January 2002), 63-69.
17. Fitzmaurice, G. W., Ishii, H., and Buxton, W. Bricks: Laying the foundations for graspable user interfaces. Proceedings of the Conference on Human Computer Interaction (CHI 95) (1995), 442-449.
18. Glos, J. W., and Cassell, J. Rosebud: A place for interaction between memory, story, and self. Proceedings of the Second International Conference on Cognitive Technology (1997), 88 – 97.
19. Goldman, L. R. Child’s Play: Myth, Mimesis, and Make-Believe. Berg: New York, NY, USA, 1998.
20. Gorbet, M, Orth, M., and Ishii, H. Triangles: tangible interface for manipulation and exploration of digital information topography. Proceedings of the Conference on Human Computer Interaction (CHI 98) (1998), 49-56.
21. Green, C. Teaching tips: Storytelling in teaching. American Psychological Society, 17, 4, 37-54.
22. Hayes-Roth, B., and van Gent R. Storymaking with improvisational puppets and actors. Standford Knowledge Systems Lab KSL 96-05 (1996).
23. Hughes, F. P. Children, Play, and Development. Allyn and Bacon: Needham Heights, MA, USA, 1991.
24. Inkpen, K., Booth, K. S., and Klawe, M. (1994). Cooperative learning in the classroom: The importance of a collaborative environment for computer-based education. http://citeseer.ist.psu.edu/inkpen94cooperative.html (Sited 04-Aug-2004).
25. Isbell, R. Telling and Retelling Stories: Learning, Language, and Literacy. Young Children 57, 2, 26-30.
26. Ishii, H., and Ullmer, B. Tangible bits: Toward seamless interfaces between people, bits and atoms. Proceedings of the Conference on Human Computer Interaction (CHI 97) (1997), 234-241.
27. Jesdanun, A. Parents Reconsider Technology for Kids. http://www.lsj.com/news/business/040727_kidstech_10d.html (Sited 13-Sep-2004)
28. Johnson, L., LaMontagne, M. J., Elgas, P. M., and Bauer, A. M. Early Childhood Education: Blending Theory, Blending Practice. Paul H. Brookes Publishing, Baltimore, MD, USA, 1998.
29. Levin, D., and Rosenquest, B. The Increasing Role of Electronic Toys in the Lives of Infants and Toddlers: should we be concerned? Contemporary Issues in Early Childhood, (2), November 2, 2001.
30. Liebschner, J. A Child's Work: Freedom and Play in Froebel’s Educational Theory and Practice. Lutterworth Press: Parkwest, NY, USA, 2002.
31. Marsh, J. Electronic Toys: shy should we be concerned? A Response to Levin & Rosenquest (2001). Contemporary Issues in Early Childhood, (3), November 1, 2002.
32. Martin, C. A. and Colbert K. K. Parenting: A life span perspective. McGraw-Hill, New York, NY, USA, 1997.
33. Montemayor, J., Druin, A., Chipman, G., Farber, A., and Guha, M. L. Tools for children to create physical interactive storyrooms. Computers in Entertainment (CIE) 2(1), January 2004.
34. Montemayor, J., Druin, A., Farber, A., Simms, S., Churaman, W., and D’Amour, A. Physical Programming: Designing Tools For Children To Create Physical Interactive Environments. Proceedings of the Conference on Human Computer Interaction (CHI 2002) (2002), 299-305.
35. Morrison, G. S. Early Childhood Education Today, Ninth edition. Pearson Merrill, Prentice Hall, New Jersey, USA, 2004.
36. Paley, V. A Child's Work: The Importance of Fantasy Play. University of Chicago Press: Chicago, IL, 2004.
37. Papert, S. Mindstorms: Children, computers, and powerful ideas. Basic Books, New York , NY, USA, 1980.
38. Papert, S. The Children’s Machine: Rethinking School in the Age of the Computer. Basic Books, New York , NY, USA, 1993.
39. Piaget, J. Psychology and Epistemology: Towards a theory of knowledge. New York: Viking Press, 1971.
40. Piaget, J. To understand is to invent: The future of education. New York: Grossman, 1973.
41. Plaisant, C., Druin, A., Lathan, C., Dakhane, K., Edwards, K., Vice, J.M., and Montemayor, J. A Storytelling Robot for Pediatric Rehabilitation. Proceedings of Assistive Technologies (ASSETS ’00) (2000), 50-55.
42. Resnick, M. StarLogo: an environment for decentralized modeling and decentralized thinking. Proceedings of the Conference on Human Computer Interaction (CHI 96) (1996), 11-12.
43. Resnick, M., and Ocko, S. Lego/Logo: Learning through and about design. In Constructionism. Harel, I., and Papert, S. (eds). Ablex Publishing, Norwood, NJ, USA, 1991.
44. Resnick, M., Martin, F., Berg, R., Borovoy, R., Colella, V., Kramer, K., and Silverman, B. Digital manipulatives: New toys to think with. Proceedings of the Conference on Human Computer Interaction (CHI 98) (1998), 281-287.
45. Resnick, M., Martin, F.,.Sargent, R., and Silverman, B. Programmable bricks: Toys to think with. IBM Systems Joumal 35, 3 (1996), 443-452.
46. Scott, S. D., Mandryk, R. L, and Inkpen, K. M. Understanding Children’s Interactions in Synchronous Shared Environments. In Proceedings of Computer Supported Collaborative Learning (CSCL '02), (January 2002), 333-341.
47. Scott, S. D., Mandryk, R. L., & Inkpen, K. L. Understanding Children's Collaborative Interactions in Shared Environments. Journal of Computer-Aided Learning, 19(2), 220-228.
48. Squeak. www.squeakland.org (Sited 13-Sep-2004)
49. Umaschi, M., and Cassell, J. Storytelling systems: Constructing the innerface of the interface. Proceedings of the Second International Conference on Cognitive Technology (1997), 98 – 108.
50. Vail, K. Girlware. Electronic School, (1997), June, A18-21.
51. Vygotsky, L. S. Mind in Society: The Development of Higher Psychological Processes. Harvard University Press, Cambridge, MA, USA, 1978.
52. Whaley, C. Meeting the diverse needs of children through storytelling. Young Children 57, 2, 31-34.

Contribution and Benefits

Presents a new design methodology for technology development for young children; describes and compares desktop and physical content-infused story games. Presents differences between physical and desktop environments for young children.