University of Maryland
Institute for Advanced Computer Studies
Human-Computer Interaction Lab
College of Education, Human Development Department
Children play games, chat with friends, tell stories, study history or math, and today this can all be done supported by new technologies. From the Internet to multimedia authoring tools, technology is changing the way children live and learn. As these new technologies become ever more critical to our children’s lives, we need to be sure these technologies support children in ways that make sense for them as young learners, explorers, and avid technology users. This may seem of obvious importance, because for almost 20 years the Human-Computer Interaction (HCI) community has pursued new ways to understand users of technology. However, with children as users, it has been difficult to bring them into the design process. Children go to school for most of their days; there are existing power structures, biases, and assumptions between adults and children to get beyond; and children, especially young ones have difficulty in verbalizing their thoughts. For all of these reasons, a child’s role in the design of new technology has historically been minimized. Based upon a survey of the literature and my own research experiences with children, this paper defines a framework for understanding the various roles children can have in the design process, and how these roles can impact technologies that are created.
Categories and Subject Descriptors:H.1.2 [Models and Principles]: User/Machine Systems—human factors; H.5.2 [Information Interfaces and Presentation]: User Interfaces—evaluation/methodology; interaction styles
General Terms: Human Factors, Design, Theory
Additional Key Words and Phrases: Children, design techniques,
participatory design, evaluation, educational applications
(Researcher Notes, April 3, 1999, Quote from an 8 year-old child).
All of this adds up to a large amount of personal experience about young people that we may or may not choose to bring with us when we develop new technologies for children. But as we know, these personal impressions may not be enough to support today’s children. While they are fast becoming tomorrow’s power-users of everything from the Internet to multimedia authoring tools (Report to the President on the Use of Technology to Strengthen K-12 Education in the United States, 1997; Fulton, 1997). They are still children that must go to school and depend on their teachers and parents for learning and living in this complex world. In addition, as we know, young children have a more difficult time verbalizing their thoughts, especially when it concerns abstract concepts and actions (Piaget, 1971; Piaget, 1973). While children can be extremely honest in their feedback and comments concerning technology, much of what they say needs to be interpreted within the context of concrete experiences (Druin, 1999).
For all of these reasons, a child’s role in the design of new technology has historically been minimized. In the Human-Computer Interaction community, we have a short but rich history of developing shared paths for communication between diverse users and technologists. However, this history of shared communication is even shorter and less developed for our children as users, testers, informants, and partners in the technology design process. With the emergence of children as an important new consumer group of technology (Heller, 1998), it is critical that we support children in ways that are useful, effective, and meaningful for their needs. With this in mind, we need to question how we can build new technologies that respect children for their ability to challenge themselves and question the world around them. We need to understand how we can create new technologies that offer children control of a world where they are so often not in control.
I believe it is in understanding the role that children can play in
the technology design process that will lead to answers. The better we
can understand children as people and users of new technologies, the better
we can serve their needs. This paper will suggest a framework for understanding
the role children have historically had in the technology design process.
How these roles can impact the technologies developed and the research
methods that are used will be discussed based upon a survey of the literature.
How these roles for children compare to adult participation will also be
examined, along with the strengths and challenges associated with children
in the design process. By understanding this framework in regards to the
child’s role, it is my belief that we can make more informed decisions
about our research and development practices that can have lasting effects
for the future.
These early discussions focused on the impact that new technologies could have on children as learners. With this understanding, researchers suggested new directions for future technology development, and new possibilities for future learning experiences with technology. During these early years, there were only rare instances where children had more direct involvement with technology developers, and actually tested experimental technology before it was in wide release. Interestingly enough, the development of programming languages such as Logo (Papert, 1977) and SmallTalk (Goldberg, 1984), brought children into the process more than any other technologies created for children during the 1970s and early 1980s.
In terms of the HCI community, the first conference paper publication concerning children and HCI issues, was published at the 1982 Gathersburg Conference that led to the establishment of Special Interest Group on Computer Human Interaction (SIGCHI) (Malone, 1982). This paper discussed a study that was done by Tom Malone (at the time from Xerox PARC) in which he analyzed children’s use of games. From his results, he proposed general HCI guidelines for designing enjoyable user interfaces. Malone’s paper was the only one of 75 papers in the proceedings that discussed children as users. Subsequent Computer Human Interaction (CHI) conference papers on children and Human Computer Interaction (HCI) issues were not published until five years later, at CHI+GI’87 (Frye & Soloway, 1987; Verburg et al., 1987). Interestingly enough, the paper presented by Fry & Soloway (1987) was entitled Interface design: A neglected issue in educational software.
The sporadic appearance of papers that discussed children and HCI issues would not significantly grow until the early 1990s (e.g., Berkovitz, 1994; Noirhomme-Fraiture et al., 1993; Pausch et al., 1992; Steiner & Moher, 1992; Strommen, 1994). As the literature grew, so too did the active involvement of children in the technology development process. By the mid 1990s, children’s roles as informants and design partners were discussed in papers that focused on everything from initial technology brainstorming experiences to final evaluation phases (e.g., Cypher & Smith, 1995; Druin et al., 1997; Druin et al., 1999; Inkpen et al., 1997; Oosterholt et al., 1996; Scaife et al., 1997).
Based upon an analysis of the literature and my research with children as design partners, I have come to see four main roles that children can play in the technology design process: user, tester, informant, and design partner. In the role of user, children contribute to the research and development process by using technology, while adults may observe, videotape, or test for skills. Researchers use this role to try to understand the impact existing technologies have on child users, so future technologies can be changed or future educational environments enhanced. In the role of tester, children test prototypes of technology that have not been released to the world by researchers or industry professionals. As a tester, children are again observed with the technology and/or asked for their direct comments concerning their experiences. These testing results are used to change the way future iterations of the pre-released technology are developed. In the role of informant, children play a part in the design process at various stages, based on when researchers believe children can inform the design process. Before any technology is developed, children may be observed with existing technologies, or they may be asked for input on design sketches or low-tech prototypes. Once the technology is developed, children may again offer input and feedback. And finally, with the role of design partner, children are considered to be equal stakeholders in the designof new technologies throughout the entire experience. As partners, children contribute to the process in ways that are appropriate for children and the process.
Figure 1: The four roles that children may have in the design of new technologies
I have come to see that each role, user, tester, informant, or design partner can shape the technology design process and impact the technologies that are created. While each role for children is used today by some portion of researchers or developers, each role has its own historical roots with its own challenges and strengths. These roles are not necessarily different from that of adult users however the methods, context, and challenges can be different thanks to the involvement of children. Choosing to use any of these roles with children may depend, among other things, on the project’s resources, timeframe, and the philosophy of the researchers involved. While each of these roles have clear differences, each role includes aspects of those roles that historically have come before it (see Figure 1). For example, in the role of informant, children may be asked to test certain prototypes (as a tester), as well as be observed with competing software (as a user).
The sections that follow will present a detailed analysis of what it
means to have children as users, testers, informants, or design partners
in the design process. The historical context of each role, the research
methods needed for such a role, the impact that this can have on technology,
and the strengths and challenges will be presented. While all four roles
will be discussed in a somewhat similar manner, it should be noted that
this paper was written by a researcher who is actively involved with children
as design partners. Whatever biases and experiences I have had with children
will no doubt color my discussions. In particular, a more personal look
at the role of design partner will be presented. It is my belief that with
a better understanding of all of these roles, we can make more informed
decisions about our design practices that can have lasting effects for
the future of children’s technologies.
The first and oldest role that can be seen in the literature is that of the child as user. With this role, the child is a user of technology while the adult looks to understand the child’s activities with various methods. Children may be observed, videotaped or tested before and/or after technology use. In this way, researchers can come to understand the impact technology has had on the child’s learning experience. There are generally two reasons for researchers to ask children to take on the role of technology user: (1) To test a general concept that may help inform future technology developers (2) To better understand the process of learning which may contribute to future educational practices. With this role, the technology used is not continually being developed and changed. The technology has been created and distributed widely for commercial or research purposes.
The role of child as user is perhaps the oldest and today remains a common role for children in the research process. This role first emerged in publications, in the late 1960s and early 1970s (e.g., Suppes, 1969; Hunka, 1973; Stodolsky, 1970). This was a time when mainframe computers were common, and educational applications were by and large "drill and practice" experiences in everything from math to English. The computer was an individualized teacher and led a child through a series of carefully moderated exercises. The curriculum was broken down into small concept blocks with exercises that had different levels of difficulty. When the computer presented reading materials and questions to answer, and the child was asked to respond. If for example, a correct answer was given, the child was rewarded by being allowed to go to the next level of materials. If the child answered incorrectly, he/she was asked to try again (Suppes, 1969; Davis, 1976; Solomon, 1986).
Figure 2: Sample interaction with "drill and practice" experience
While these technologies automated the learning experience, they did not offer a great deal of control to the child learner and user. What was to be covered and how it was to be presented was pre-programmed by the computer system. In some sense, this lack of user control was also reflected in the limited involvement of the technology user in the development process. In the 1960s and 1970s SIGCHI did not exist and the first CHI conferences were not until the early 1980s. But even in the early years of CHI conferences, papers still discussed users in regards to technology development as not really knowing what they needed. In one paper that discussed a survey of 445 designers, "almost nobody recommended that potential users become even for only brief periods of time, part of the design team" (Gould & Lewis, 1983, p. 51). This can be strongly contrasted with the cooperative design movement that was emerging at the time from the Scandinavian countries which promoted the notion of co-design with users (Bjerknes et al., 1987).
In terms of children as technology users, the majority of the literature
during the 1970s and 1980s reflected their limited involvement in the technology
development process (the few exceptions will be discussed in later sections).
The terminology that was used to describe children’s involvement, offers
a glimpse into the role of users at that time. Such phrases as, "the subject’s
task", "allow the user", "children should be used", were common and all
suggest that users, especially children, had little control in the research
process. The main contribution of children as users was seen in the observations
that researchers could make of them, the work children accomplished using
the technology and the tests children took before and after using technology.
These experiences could tell researchers more about the impact of technology.
The role of child as user can still be seen today. It is more common in
the literature of educational and child psychology, as well as the broader
educational research community, but it still can be seen as a tool to consider
the future of new technologies and new educational uses of technology.
In addition to video, participant observation where researchers are in the room with users can also be of value (e.g., Nicol, 1988; Pelegrino et al., 1991; Plowman, 1992). It is common for researchers to become a part of classroom activities demonstrating software, answering questions, and more. At the same time, it is quite common to include teachers in the research experience. They too can collect data thanks to their own first-hand experiences in the classroom (e.g., Koenemann et al., 1999; Rose et al., 1998). Children’s use of the computer can also be captured and understood through system logs showing patterns of interaction with different system tools (e.g., Candy & Edmonds, 1982; Jackson et al., 1998; Neal & Simons, 1983; Searle et al., 1974). These methods were useful, for example, in understanding early hypermedia technologies with children. Researchers at Apple Computer observed children’s reactions to HyperCard’s menus and commands, as well as tracked their navigation patterns in various information spaces (Nicol, 1988).
In addition to activity observation, data concerning user impressions can also be collected. Qualitative surveys can be given to children concerning their like, dislikes, difficulties, and interest areas. For instance, interviews can be conducted after the use of technology, which can help to clarify children’s motivations and pinpoint specific reactions to particular technology features (e.g., Jackson et al., 1998). More formal quantitative surveys can also be administered, where questions are answered on a numerical scale or with various options (e.g., Burov, 1991; Lester et al., 1997; Salzman et al., 1999). These kinds of surveys can be at times difficult to develop. The survey language needs to be age appropriate, and easily comprehensible.
Information can also be collected concerning the impact that technology has on the child’s learning of a subject area. It is common for tests to be given to children before and after the use of technology over a period of time. Typically, these tests are quantifiable instruments concerning subject matter knowledge (e.g., spelling, math, science, etc.) (e.g., Burov, 1991; Candy & Edmonds, 1982; Hunka, 1973; Salzman et al., 1999; Searle et al., 1974). In some cases, ethnographic or qualitative descriptions of children as technology users are done to capture data as well. In these case studies, a small number of children can be observed over an extended period of time (e.g., Blomberg et al., 1993; Plowman, 1992; Solomon, 1986). Data collection can be done, by asking children to write their thoughts in journals. Teachers and researchers may also write down their observations over time. Interviews with the children and teachers may also be conducted.
Finally, results of the children’s work using the computer can be analyzed as well. For example, researchers at the University of Michigan collected software models created by 9th grade students (age 15-16). These students used special software to build for example, models depicting environmental changes in streams or global warming. These software models were later collected and analyzed by researchers to better understand the students’ use of certain software tools (Jackson et al., 1998).
Many times researchers will collect information about users in multiple ways, so that results from one research method can explain another. For example, in the case of Vanderbilt University’s researchers, they gave children an initial "paper and pencil" test that measured problem-solving capability in mathematics. They then observed classes of teachers and students with technology (Pelegrino et al., 1991). Afterwards, they again administered a similar test. What they found was that children’s test scores rose considerably after their use of technology. Their classroom observations showed them why. What they saw was an active engagement with the subject matter through the use of technology (Pelegrino et al., 1991).
Data collection like this can also be useful when comparing users and non-users of technology. Many early studies compared children who used computers with children that had never used the technology. The Vanderbilt researchers did just this with their study. Not only did they look for change in those children who used their technology, but they compared those changes with children who did not use their technology. This exhaustive study followed over 1,300 children for over a year and offered numerous insights into the use and impact of the Jasper Woodbury technology (Pelegrino et al., 1991).
Figure 3: "Get Out the Vote": Part of the Jasper Woodbury Series on Complex Trip planning
Students must prepare plans to drive as many voters as possible to the polls on election day. Students must prioritize goals, identify strategies, organize data and develop algebraic shortcuts. (http://peabody.vanderbilt.edu/projects/funded/jasper/)
When analyzing the data collected in studying children as users of technology, again, there are numerous methodologies. Depending on the kind of information researchers are looking for, can dictate the ways data should be analyzed. For example, some researchers look to see how fast a task can be done (e.g., Burov, 1991; Fell et al., 1994; Searle et al., 1974; Suppes, 1969); others look to see how many content questions can be answered after a child uses a piece of technology (e.g., Burov, 1991; Candy & Edmonds, 1982; Hunka, 1973; Salzman et al., 1999; Searle et al., 1974). Still others try to understand changes over time in children’s activities. Do they recognize their mother’s voice faster? (Fell et al., 1994) Do they get motion sickness? Is their motion sickness reduced? (Salzman et al., 1999). In all cases, the research methods are used to understand the impact that technology has on the child user. From this understanding, future technologies may be changed or developed. In addition, these new insights can offer a better understanding of how children learn, which can lead to new theories for education and new teaching practices with technology.
3.3 Impact of this Role on Technologies
When children are in the role of users, adult researchers can stand back from day-to-day development issues, and look at the big picture. Researchers may have concerns about certain kinds of software for children, or they may wonder what the best ways are for children to learn with particular technologies. Whatever their questions, researchers look to develop general recommendations for the future. One such recommendation concerning multimedia was made after a recent study with children as users. Researchers suggested, "Multimedia interactive learning environments need to be designed so learners are able to both find narrative coherence and generate it for themselves" (Plowman et al., 1999, p. 316).
How much immediate impact can these general research recommendations have on technology? It is not clear. The time between the development process, the child as user, and the published study could take years. In the meantime, technologies are continually changed, revised, and updated. For this reason, the role of child as user is used more commonly by researchers rather than industry practitioners. The impact that this role can have on technology may be less immediate or more difficult to pinpoint.
3.4 Challenges of Child as User
The challenges of this role for children can be reflected in the limited input they have in the technology development process. Children can be thought of as objects to be watched or tested, they do not initiate changes in research techniques. More traditional research methods of surveys and written tests can be difficult or stressful for young children to negotiate. Therefore, this role may be one that children are not as comfortable with as other roles later described. They may be frustrated with the lack of control or uninterested in the activities. For example, in the Vanderbilt University study (Pelegrino et al., 1991), researchers reported that, "Assessments were often less pleasant and informative for students and teachers than hoped" (McGilly, 1995, p. 76). In other words, the data collection experience of testing was something that students and even teachers found difficult. Researchers need to keep this in mind when designing ways to understand the impact technology has on children.
For teachers, the role of child as user may also be challenging. No matter how simple the research methods are, teacher involvement is still needed, from changing class time to accommodate researchers, to spending their own time contributing to the data collection. In today’s schools, where national curriculums and mandated testing are common, little time can be afforded for such outside research activities.
For technology developers this role may offer less timely feedback to the development process. If children are only to use already released technology, then the results of the research are limited to offering suggestions for future researchers interested in developing a similar technology. If children can use technologies before they are released, children may have more immediate input into the technologies that are being developed (see next section on the role of children as tester).
3.5 Strengths of Child as User
An important strength of this role for children is that they can be incorporated into the technology development process somewhat easily. Since this role asks for children to be users, little if any changes need to be made to a child’s school day. Obviously parental permission needs to be obtained and teachers need to accommodate researchers’ technology, but the majority of today’s children already use technology in some form. In addition, it is not unusual for children in school to be continually observed and assessed by adults. From the child’s standpoint, little needs to change in their day-to-day activities to be included in the technology research process.
Another strength of this role is the outcome of such comprehensive research studies as the Vanderbilt University work (Pelegrino et al., 1991). When researchers analyzed over 1,300 students (8-12 year old) during the 1990-1991 school year, classroom educators felt comfortable with the striking results that showed technology helped children learn. These results can affect technology being developed in the future. But more immediately, results such as these can start to make changes in the classroom. More traditional educators can begin to integrate new technologies in their classroom in ways reflected by the research.
For researchers, when a child is in the role of user, it may be easier for them to accomplish their research more quickly. Researchers have some semblance of control when defining the research activities. Children are told to do one activity and then the next. Once these activities are completed the research can be analyzed and conclusions can be drawn. With children in other roles (described in later sections), children have more input into the research design, which can at times slow down the process.
The last strength of this role is for HCI researchers. The exciting part of when children are users, is that researchers come to better understand children. No matter the limitations, this role enables adults and children to answer research questions that can have far-reaching impact on the future of technology and education.
A more recent role for children in the development process is that of tester. With this role, children test prototypes of emerging technologies. The goal of this role is for children to help in shaping new technologies before these commercial products or research projects have been released to the world. As a tester, children may again be observed with technology, and the impact on children can be assessed. Many times adults may ask for direct feedback from children by asking them such questions as, "What did you like?" "What was too boring?" "What was too hard?" It is important to note, that with this role the initial brainstorming and design phase has already been accomplished by adults. Children do not begin their role as tester until initial prototypes have been created.
The role of the child as tester was rare until the late 1980s and early 1990s. Before that time, only a few unique instances of child as tester could be found. Interestingly enough, these instances have come to be considered pioneering work in technologies for children. Today, there are few people focused on HCI and children, who do not know of Seymour Papert’s Logo research group at MIT. In the late 1960s and early 1970s, this group developed not only a new programming language for children, but pioneered a new approach to teaching and learning with technology. They suggested that the computer need not tell the child what to do, but the child could tell the computer what to do in ways that the child chose. In doing so, the child could construct his/her own paths to knowledge. This has since come to be called a "constructivist" or "constructionist" approach to using technology (Hoyles, 1985; Papert, 1972; Solomon, 1986).
As it happens, these ideas evolved and new technologies were developed with the role of the child as tester. It may well have been Papert and his colleagues’ deeply held belief in children as builders, scientists, and learners that led to the early inclusion of children in the technology design process, much earlier on than most researchers of their time. Papert and his colleagues frequently point out in talks even today, of instances where children changed the way Logo researchers considered implementing a feature or where children found a problem that adults never saw (Solomon, 1986). For example, when the Logo program was first developed, it was completely text-based. Logo programs could be created to manipulate words and sentences, but not images or graphics. Realizing the need for more concrete objects to play with, the Logo "turtle" was developed, so that children could draw with the Logo programming language (Papert, 1980). This could not have been developed without the input of children as testers during the earliest prototyping stages of Logo. Now going on 30 years, the Logo research team still works in a similar way: develop a working prototype, try it out with children and teachers, and then revise it based on input.
Figure 4: The Logo "Turtle" was originally a robotic creature that moved around on the floor (A.)
It later became a "Screen Turtle" (B.) Both could be directed by typing Logo commands at the computer.
Was this common practice in the early 1970s? No, this was not in any way. What was much more common was the design practices of Patrick Suppes and his group at Stanford. They were the driving force behind what we have come to know as "drill and practice" (Druin & Solomon, 1996). Their research activities consistently involved children as users. Were Papert and his team the only researchers at that time to involve children as testers? Not exactly, Alan Kay and Adel Goldberg at Xerox PARC developed a programming language in the 1970s with children as testers (Goldberg, 1984). This language was called SmallTalk. While it was not expected to be used only by children, Alan Kay believed that children could offer powerful new insights into future technologies (Solomon, 1986).
It was not until the late 1980s and early 1990s, that children as testers were consistently reported in the literature. This coincided, not surprisingly, with a general trend in the HCI literature toward developing better interfaces for non-programming end-users. No longer was the HCI community primarily concerned with developing better interfaces for programming (Grudin, 1990). By CHI’90, numerous papers and panels reflected this move toward embracing non-technical users and bringing them into the design process. Participatory Design, (Blomberg & Henderson, 1990; Johnson et al., 1990; Montford et al., 1990) Heuristic Evaluation (Nielsen & Molich, 1990), and Contextual Design (Wixon & Holtzblatt, 1990) were all discussed in paper and panel sessions.
At that same time, researchers and industry professionals were beginning to discuss the child as tester in paper and journal publications. An early example of this was in the late 1980s, at the Bank Street College of Education where researchers were developing Palenque, an interactive multimedia environment based on the popular Voyage of the Mimi product (Wilson, 1988). It enabled children to explore virtual multimedia worlds that included tropical rainforests, rivers, a temple, museum and palace. Children tested the technology’s navigation tools, museum database of multimedia information, the appeal of the interface, and the comprehension of the menus and icons (Wilson, 1988). Continual feedback from children at the Bank Street School for Children, helped to shape and change this technology.
During the 1990s, the role of child as tester has come to be common
in both industry and academia (e.g., Berkovitz, 1994; Noirhomme-Fraiture
et al., 1993; Moshell & Hughes, 1996; Strommen, 1994). CD-ROM products
have been child-tested at companies such as Children’s Television Workshop
(Strommen, 1994), Electronic Arts (Super et al., 1996), and Living Books
(Druin & Solomon, 1996). Even today’s interactive plush animal interfaces
have been child-tested (e.g., Microsoft’s Barney (Strommen, 1998), MIT
Media Lab’s SAGE (Umaschi Bers et al., 1998). Today, it is a surprise,
if children do not test commercial technology products.
When children are included in the development process as testers, the methods used by children and adults can be diverse. As with the case of child as user, researchers and industry professionals look to understand the child tester’s activity patterns, likes/dislikes and changes in learning. However, with the role of child as tester, the goal of the child’s involvement is somewhat different. Children are testing the technology to see if it meets the design goals. Larger research questions about education and future directions in HCI are less the focus. What matters more often are the more immediate issues: What parts of the technology are confusing? What parts do children like? Can children learn with the technology? Where are the bugs?
The number of times that children and adults may attempt to answer these questions with test sessions can vary. In the case of the Living Books Company (now a part of Broderbund which is a subsidiary of The Learning Company) developers work with children after every few screens they develop. For example, with the CD-ROM title, The Tortoise and the Hare, children let developers know that they were unhappy when they selected a particular hotspot (Druin & Solomon, 1996). With this selection, the Hare would run out, read a newspaper, crumple it up, and leave it on the ground. Many children felt that the hare was littering. So designers added an additional hotspot animation. Today, with the resulting product, if children select the crumpled paper on the ground, the Tortoise says, "Hey Hare, did you forget to recycle that paper?" (Druin & Solomon, 1996).
Figure 5: From the Living Books CD-ROM, Tortoise and the Hare (http://www.livingbooks.com)
Other product teams do not have the time or resources to work with children so often. In the case of Kid Pix, when it was first released back in 1989, Broderbund sponsored their first Kid’s Day. It was a weekend testing day for 20 children to try out Kid Pix. Developers offered their testers cookies for a break and a crafts table with paper, glue, glitter, etc.—just in case the testers got bored. As it happened, no one ate any of the cookies or used any of the crafts. Broderbund determined that Kid Pix would be a hit (Druin & Solomon, 1996).
How focused or broad the testing activities are, depends on the needs of the product or project. There may be certain areas of the product that developers have questions or concerns about. Therefore, that particular area will be heavily tested. In the case of Microsoft’s Actimates/Barney, developers wondered if it was alright for a child to be interrupted in a song or game if the child selected something else. Therefore, developers observed children with a Barney that could "not be interrupted". Through child-testing, they found that children became frustrated with Barney, if they could not interrupt themselves and move on to another activity (Strommen, 1998).
The number of children needed during the testing process can vary. If the prototype is still in its early stages, then a few children for a few hours, can be all that is needed to spot the big problems. For example, at Northwestern University, researchers worked with six Middle School students (ages 12-14) to initially test the general functionality of the Progress Portfolio software (Loh et al., 1998). Thanks to these early observations of children using the software, researchers were able to pinpoint the need for additional work in the areas of capturing and annotating (Loh et al., 1998).
The number of children as testers may also be limited if the methods used offer large amounts of data. In the case of SAGE (Storytelling Agent Generation Environment), researchers were interested in better understanding how this stuffed-animal interface for storytelling could support seriously ill children (ages 7-16) in a cardiac unit of Boston’s Children’s Hospital (Umaschi Bers et al., 1998). With this understanding, researchers hoped to change SAGE for the future, and develop design recommendations that might be useful for future researchers working with this same user population in hospitals. The way they chose to answer their questions, was to interpret the stories of eight children who used SAGE. Researchers looked at these stories to try to understand the role that the child plays when using the technology, what personality the child takes on, and in what way the child might be symbolically representing themselves in the story. With countless pages of data to read and analyze, eight children offered researchers the information they needed.
Figure 5: SAGE: Storyteller Agent Generation Environment
How much can this role impact new technologies? It truly depends on the time the university researchers or industry professionals have to listen. If a product has a short development cycle, then it is unlikely that large changes will be made to technology even if children offer feedback. In addition, we need to keep in mind that ultimately the technology started out in development by adults. The brainstorming, idea generation, and the initial technology creation all happened before children were asked to test the prototype. Therefore, while children can have some impact in the development of new technologies, their input is minimal compared with that of adults.
4.4 Challenges of Child as Tester
As was the case for child as user, the challenges of this testing role for children have to do with the limited input they can have in the technology development process. With the role of child as tester, young people can have more immediate impact on the technologies than with the role of child as user. However, children’s impact can still be limited. Adults have the first say in what will be created before children ever see something to test. Once children have an opportunity to suggest changes, there is a chance that these changes may never get made, since it is ultimately up to adults to make those changes. If adults don’t agree with the feedback, or if adults decide that the changes are less important then getting the product out the door—changes just won’t happen.
For teachers, this role may also be challenging. As was the case with the child as user, teachers may need to be involved. Spending class time on testing new technology rather than learning spelling, may be difficult to negotiate. In addition, for parents this role may also be problematic. It may necessitate children to be taken to a lab outside of school, or researchers may ask to come to their home. This necessitates time and energy, which many of today’s parents have little.
For technology developers, this role may offer more than they expected. Children are incredibly honest and at times harsh in their assessments of technology. Children have little patience for what they don’t like and they will let technology developers know exactly just that. Even adults can have their feeling hurt when they hear a child say they don’t like a piece of technology that took months or years to create. If children are not a part of the initial brainstorming process, then children as testers may offer some serious surprises. While this research information can ultimately make a successful product or project, it can also derail a development schedule with some surprising results.
4.5 Strengths of Child as Tester
The strength of this role for children is that they can feel empowered. They can feel that adults want to listen to what they have to say about new technologies. Again, as with the role of child as user, the role of child as tester asks children to do little more than use technology. Many children already use technology, therefore few skills need to be learned to be included in the technology design process.
When a child is in the role of tester, extraordinary amounts of time may not be needed to find initial results. For researchers or product developers on a tight schedule or budget, this can be an important consideration. A one-day workshop can be offered at the lab or in a school. After-school programs can be developed that need little teacher involvement. Depending on the kind of technology that is being tested, school complexities may be kept to a minimum.
Ultimately the strength of this role can be in the impact that such
a role can have on new technologies. For educators, it can mean more usable
technologies for teaching. For parents it can mean, better technologies
for their family’s home entertainment and informal learning experiences.
For children it can mean technologies that they want to use, rather ignore
or be frustrated by them.
5 THE CHILD AS INFORMANT
With this role, the child plays some part in informing the design process. Before any technology is developed, the child may be observed with existing technologies, or they may be asked for input on paper sketches. Once the technology is developed, the child may again offer input and feedback. With this role, the child plays a part in the design process at various stages, based on when researchers believe they can be informed by children.
5.1 Historical Context
The role of the child as informant did not emerge in the HCI literature until the middle of the 1990s. There was literature before this time that discussed children informing the design process, but primarily as users for observation or as testers of prototypes. Until the 1990s, children were not discussed as design participants who offered design directions, or prompted the start of new projects. Interestingly enough, the emergence of this informant role for children coincided with the establishment of a new CHI conference submission category called, "Design Briefings". This publication category focused on the methods of design, rather than the technology results. While these conference submissions were not just about the design process with children, they may have helped to bring to the attention of the HCI community the presence of children in the process (e.g., Druin, Stewart, Proft, Bederson, & Hollan, 1997; Halgren, Fernandes, & Thomas, 1995; Oosterholt, Kusano, & de Vries, 1996; Piernot, Felciano, Stancel, Marsh, & Yvon, 1995; Rader, Brand, & Lewis, 1997; Scaife, Rogers, Aldrich, & Davies, 1997).
From this point on, the design of children’s technologies and the design processes used, became frequent publication topics at CHI conferences. This timing also coincided with the growth of the multimedia industry worldwide. CD-ROM software titles were becoming financially lucrative and CD-ROMs were being sold as a standard component in most PCs. According to the Software Publishers Association, over one billion dollars (US) of educational software was sold in 1994 (Investors Business Daily, 1995). In all likelihood, children as informants were a part of the design process much earlier in HCI history, but adult researchers were not formally recognizing their presence. For example, folklore could be heard among industry professionals that described how children shaped and changed design directions. Craig Hickman, creator of Kid Pix and Roger Wagner, creator of HyperStudio, were among those with stories to tell, but little was documented in academic journals or conference publications (Druin & Solomon, 1996).
It wasn’t until 1997, that the role of child as informant became more clearly defined. It was at that time that Scaife et al., presented Designing for or designing with? Informant design for interactive learning experiences (1997). In this critical CHI 97 publication, they described the notion of "informant design." The authors questioned when children should be a part of the design process, and what contributions could be important for the design of technology. Before this time, numerous researchers were including children in the design process, but not making a distinction of when. Were children testers at the end of the design process? Were children partners working throughout the process? Were children informants helping the design process at various critical times? Scaife and Rogers (1999) continued to question these important notions in their follow-up publication, Kids as informants: Telling us what we didn’t know or confirming what we knew already. In this book chapter, they explained, "What is not in doubt, then, is that children can be brought into the design process and make a contribution. What is less clear is whether we can generalize about the relationship that they can be expected to have with designers" (p. 30).
Out of these critical discussions by Scaife and Rogers, a clearer understanding of the child as informant began to emerge. As this role has come into sharper focus, both industry professionals and academic researchers have found it quite useful. An example of this informant role can be seen in the design of Knowledge Adventure’s popular CD-ROM, My First Encyclopedia. This educational software for young children throws away the traditional interface elements of windows and menus, and instead uses a picture of a tall tree as an interface mechanism. By selecting any of the tree’s branches, video guides support young users in finding information. This simple visual interface for an encyclopedia was designed by a team led by Roger Holzberg. When this team began their work, Holzberg went to daycare centers and preschools looking for children’s input. He asked children, "Where do you most like to play after you go home from school or daycare?" Their most common replies were, (1) "play outside" and (2) "climb a tree" (Researcher Notes, April 5, 1995, Telephone Interview with Roger Holzberg).
Therefore, thanks to the inspiration of children as informants, a tree was developed as an interface. Could Holzberg and his colleagues have developed a tree without the help of children? Perhaps, but with children suggesting directions at the very start of the design process, a tree quickly became obvious as an interface metaphor. Holzberg’s experience is not unique. Many industry professionals or academic researchers have now come to acknowledge the role of children in setting directions for everything from new digital library interfaces (Wallace et al., 1998) to new programming languages for children (Smith & Cypher, 1999).
5.2 Methods Used
When and how children are informants varies a great deal between design teams. Some, as in the case of Knowledge Adventure, have an idea for a product, but are looking for an interface direction. Others, such as Cypher and Smith (1995, 1999), may wonder if their initial project idea is even appropriate for elementary-school children. While still at Apple computer, Cypher and Smith asked the question—can children program their own interactive simulations? To begin to address this, the design team worked with fifth-grade children (ages 10-12) and asked them to program a friend around a room by placing 3M Post-It notes with programming instructions on their clothing. For each Post-It note, another command could be executed. The team learned that children could program these kinds of simulations, and that they might really like to do so (Cypher & Smith, 1995). Since that time, a product first called KidSim and now called StageCast Creator has been developed, and the design team has started a new company (http://www.stagecast.com).
There are numerous ways to bring children as informants into the design process. At the start of a project or product design, teams may decide to observe children using existing technologies. In this way, design directions may not necessarily be expressed directly by children, but may be implied by their actions. These methods of observation are similar to the ones outlined when children are users or testers. What differs from those methods is when these observations happen and how directly it can affect the design of new technology. In the case of researchers from the University of Michigan, they began their project by observing 6th and 9th grade students (ages 11-16 years old). They watched the students’ use of web search engines and browsers while studying science (Wallace et al., 1998). From these observations, researchers became convinced that web tools were not sufficient for learners searching out information. The search engines returned too many hits, and students seemed to become bored. In response, the research team developed Artemis, software that supports learning with digital information resources. This is now a part of the University of Michigan’s Digital Library initiative.
Figure 6: An example of the Artemis software that was developed at the University of Michigan
Initial ideas or observations of children from the start of a project are not the only time children can inform the design process. They can be involved at any time the design team believes it needs direction or support. For example, Scaife and Rogers (1999) after realizing a prototype they had developed was considered "dull" by children, decided to probe further. To do so, they used low-tech sketching materials and artifacts. They asked the children to sketch what they thought software could be like that would teach other children about food webs. After minimal results from this method, the team gave laminated cut-outs of organisms to the children. The children manipulated these cut-outs and told researchers what could be done to make software. With this method, the children didn’t get caught up in the details of drawing animals (as they had while sketching) and concentrated on the interaction and behaviors of the animals.
Low-tech materials, interviews, design feedback on prototypes, can all be used continually as methods for informants. What is critical, is that the materials and methods are age appropriate for working with children. Many of these methods are similar to previously described techniques for when children are in the role of user or tester. What differs is when and how often these techniques are used during the design process. There is no magical formula of what to do when. However, what is certain, is that a design team can choose to include children as informants in various ways and at numerous times. Depending on the needs of the team and the specific project, differing methods may be chosen.
5.3 Impact of this Role on Technologies
When children are informants, they can have an impact on technology from the very beginning of the design process. While children are not continually a part of the design process, children can have an impact on what directions are taken, how the technologies are shaped, and ultimately how they are evaluated.
How much can this role impact new technologies? It truly depends on the university researchers and/or industry professionals that choose to work with children. If the project is pressed for time, then there may be fewer opportunities for adults to work with children. If the adults on the project choose not to listen or not to agree with the children’s input, then in all likelihood, children’s impact may still be minimized. The dilemma has been explained in this way, "On one hand, the kids come up with many wonderful suggestions that the design team would not have come up with… On the other hand, many of their ideas are completely unworkable in computational terms, and furthermore, could conflict with pedagogical goals of the software… So how do we know when to say yes and when to say no to kids ideas?" (Scaife & Rogers, 1999, p. 44). Ultimately, it is the adults on the team that choose what ideas may be used and the times to be informed by children. Therefore, while children can have a great deal of impact throughout the design process, it depends very much on the context of the experience.
5.4 Challenges of Child as Informant
As has been the case for all the previous roles, the challenge of this role for children is that ultimately adults are still in charge. While children can have more continual impact on the technologies that are being created, their impact can still be limited. Adults have to decide when to work with children, how to work with them, and ultimately what they choose to hear from children. Directions are set by adults, as well as deadlines. Therefore, while this role of child as informant gives children the most say of any of the previous roles, it still has its challenges.
For teachers, this role may be challenging. No longer may there be regular times to meet with researchers. Thanks to the flexibility of this role, there may be times when researchers are not in the classrooms. There may be other times that researchers are in the classrooms continually. This research flexibility may be problematic to structure given the limitations of the school day.
For technology developers, this role may need more time than any of the previous roles discussed. Because it brings children into the design process from perhaps the start of the project, more resources need to be allotted for activities to be accomplished. Therefore, the role of informant can cost time and money to accomplish.
5.5 Strengths of Child as Informant
The strength of this role for children is that they can feel empowered and challenged by the experience. Children can feel that adults want to listen to what they have to say about new technologies. Since children may be in a position to offer input at various times, children may also be challenged by many aspects of the problem-solving and brainstorming experiences.
For researchers, when a child is in the role of informant, there can be flexibility in when and where activities take place. In some cases, it may be more appropriate for children to work in schools. At other times, the university lab or industry offices may be more suitable. Schedules can be worked around for teachers and researchers, since there will be various times that it is not necessary to work with the children.
Another strength of this role is the impact this role can have on new
technologies. As with previous roles, when children are involved in the
design process exciting new technologies for the home and school can be
designed. For children, this can mean technologies that are less frustrating
and more compelling to use.
6 THE CHILD AS DESIGN PARTNER
The role of child as design partner is similar to that of an informant, however, this role suggests children will be a part of the research and design process throughout the experience. With this role, the child is an equal stakeholder in the design of new technologies. While a child cannot do everything that an adult can do, they should have equal opportunity to contribute in any way they can to the design process. For example, adult researchers that are visual artists or educators can support the technology design process with domain specific expertise and experience. The same can be said of child researchers. They too have special experiences and viewpoints that can support the technology design process that other partners may not be capable of contributing (Druin, 1999). With this role of design partner, the impact that technology has on children may not be as significant as the impact children can have on the technology design process.
This role for the child is one that my research is strongly committed to supporting. For the past five years, my research teams have been developing new technology design methodologies to support children in their role as design partners. What follows is a discussion of how these methodologies evolved, what methods we use today, and how these methods can impact the technologies that are developed with children as design partners. As with each of the previous roles, the strengths and challenges will be examined in comparison to other approaches.
6.1 Historical Context
We have a belief at the University of Maryland that partnering with users is an important way to understand what is needed in developing new technologies. This belief has been heavily influenced by research practices over the past 20 years: the cooperative design of Scandinavia (Bjerknes et al., 1987; Greenbaum & Kyng, 1991; Sundblad, 1987), the participatory design of the United States (Blomberg & Henderson, 1990; Greenbaum, 1993; Johnson et al., 1990; Schuler & Namioka, 1993), and the consensus participation of England (Mumford & Henshall, 1979). As Greenbum and Kyng (1991) have explained, "We see the need for users to become full partners in the cooperative system development process….Full participation of (users) requires training and active cooperation, not just token representation" (pp. ix-1).
This partnership between users and researchers from different disciplines was first exemplified in the Scandinavian cooperative design work beginning in the 1970s. It was during this time that employee influence through trade unions grew, and collaborations between workers, management, and researchers influenced how new technologies could be created for and used in the workplace. Cooperative design methods supported the development of new technologies for carpenters, typographers, bankers, manufacturers, and more (Bjerknes et al., 1987; Greenbaum & Kyng, 1991; Schuler & Namioka, 1993; Sundblad, 1987).
This approach to design attempted to capture the complexity and somewhat "messy" real-life world of the workplace. It was found that many times there were not sequential tasks accomplished by one person, but many tasks done in parallel and in collaboration with others. Interestingly enough, this description could also easily refer to the complexity and "messiness" of a child’s world. In any case, this workplace design approach was not confined to the Scandinavian countries for long. By the 1990s, these practices were being adapted and applied to research with children (Druin, 1996; Druin et. al., 1997; Druin, 1999; PDC'96: Participatory Design Conference, 1996).
As an individual researcher, my methods with children first took root in an intellectual environment that embraced building technology for children in a constructivist model of education. In the early 1980s, at the MIT Media Lab, I was a part of a community of researchers that deeply felt children should construct their own paths to knowledge, and that computer tools should support children as builders, designers, and researchers. It was a community that was grounded in years of developing Logo and Smalltalk programming languages for children. Yet, surprisingly enough, if you looked closely at the design practices of this community of researchers, it was not common to find children as researchers or partners in developing those constructivist tools. Children were primarily testers, and adults came up with the great ideas.
It would take me almost five years to begin to understand the full extent of a design partner, why children could be partners, and how partnering can come about (Druin, 1999). It did not happen suddenly one day, but rather, these concepts and understandings evolved slowly over time with numerous research and development experiences with children. I found that I personally as a researcher moved from working with children as testers, to informants, to finally and firmly as design partners. In my early work at the MIT Media Lab as a Masters student developing NOOBIE, children tested ideas, offered suggestions, but I was clearly the one with the idea to build a 6-foot stuffed computer that replaced the keyboard and mouse with hugging and squeezing (Druin, 1987). In my later work with children in New Mexico, children were clearly a part of the brainstorming process, but not continually (Druin et al., 1997). While I referred to them as my partners even then, it has now become clear that they were only a part of the design process more sporadically than continually.
Today, children are most definitely our partners in all that we do at the University of Maryland’s Human-Computer Interaction Lab. Twice a week, children ages 7-11, join researchers from computer science, education, art, robotics, and more. Together we have become what I now call an "Intergenerational Design Team" pursuing projects together, writing papers, and creating new technologies (Druin, 1999; Druin et al., 1999). This intergenerational design team has produced research projects that include storytelling robots (Druin et al., 1999), collaborative zooming software for storytelling (Benford et al., Submitted) and most recently web pages for kids about the United States Census Bureau (http://www.census.gov/dmd/www/2khome.html).
This partnership with children is not isolated to the University of Maryland. Children as design partners have migrated to Europe becoming a critical part of our research methodology in a three-year project funded by the European Union’s i3 Experimental School Environment initiatives (Druin et al., Submitted). KidStory, is a collaboration between almost 100 children and 25 adult researchers in Sweden and England to develop new collaborative storytelling technologies for children (Benford et al., Submitted). Researchers at the Swedish Institute of Computer Science, the Royal Institute of Technology, Sweden, and the University of Nottingham are collaborating with us at the University of Maryland in generalizing our methods of children as design partners. While KidStory has just finished its first year, we can already see how children as design partners have impacted the technologies we have developed. Two hundred twenty-two design suggestions have been collected from the children’s journals that have led to significant development efforts in designing new storytelling technologies.
Figure 7: An example story being created in the KidPad collaborative software. This work is developed as a part of the KidStory research project in partnership with children in Sweden and England (http://www.sics.se/kidstory)
6.2 Methods Used
The research methods we use, whether in Maryland, Sweden, or England, have come to be called cooperative inquiry (Druin, 1999). These methods have evolved and developed over the last five years. They began as methods for bringing adult users into the technology design process. Such methodologies as contextual design (Beyer & Holtzblatt, 1998), cooperative design (Bjerknes et al., 1987), and participatory design (Greenbaum & Kyng, 1991; Schuler & Namioka, 1993), call for adults from different domains to partner with technologists during the technology design process. From brainstorming methods that ask users and designers to sketch out ideas (participatory or cooperative design), to interviewing methods that can capture user tasks, roles, and design ideas (contextual design), innovative research methods are being found to work with users. While these methodologies for adults offered an excellent beginning structure for our research, they needed to be adapted to suit a team that included children, for example, to overcome the teacher-student paradigm invoked by groups of older and younger researchers in favor of co-equal partnerships. Over the years, our note-taking practices, interview procedures, data analysis, and day-to-day team activities have evolved. For example, we have found that interviewing procedures for adults are not appropriate for speaking with children. We have since changed everything from what team members should use as notepads (small and inconspicuous) and how they should dress (informal), to the process of capturing and synthesizing data (Druin, 1999).
In general, we have found that both children and adults need time to negotiate a new "power structure," in which neither adults or children are completely in charge. Both must begin to work together toward common goals. Children need to learn their new role as design partners. We do this by introducing the notion of invention, by asking such questions as: What is an invention? How are inventions created? When do we know something needs to be invented? Children work with team members on introductory design experiences, such as inventing a new sandwich; redesigning a new milk carton; and finding objects in their classroom to fix. In each case, children and adults work together in small groups to brainstorm and discuss "what is wrong" with the existing "technologies." Teams might, for example, decide that the problem with a milk carton is that it is too difficult for young children to pour from, and therefore it needs to be redesigned so that children can’t spill milk easily. We have had groups "prototype" the perfect spill-proof milk carton out of plastic tubes, clay, and straws. We have had others groups decide that milk containers should be more fun and so children should be able to spill in interesting ways. (KidStory Research Notes, Class Session #8, Stockholm, Sweden, March 16, 1999).
We have found, as children accept their role as design partners, they better understand their role in evaluating and redesigning computer-related technologies, such as a new mouse or a piece of software. Research partners young and old become accustomed to working together as critics, designers, and inventors. Adults do not "give assignments" to children who "do all the work." Instead, all design partners establish common goals and participate in collaborative development activities. "Low-tech" prototyping tools (e.g., paper, crayons, clay, string, LEGO bricks, etc.) provide material to sketch ideas. Researcher journals for children and adults serve as a repository for ideas and research evaluation. These journals may be used to sketch design ideas, collect photos of technology artifacts, or reflect on team activities. Depending on the age, discipline, or note-taking style of the researchers, different methods of describing or capturing their thoughts can be used (e.g., drawings, text, photos, computer printouts, etc.).
As time goes on, our team members have begun to see themselves as technology design partners: children begin to see themselves as researchers and adults begin to see themselves as partners. This can take as long as 6 months, but the team moves from "wondering how this is done," to planning "what will be done" (Druin, 1999). Children and adults alike gather field data, initiate ideas, test, and develop new prototypes. Team members do what they are capable of, and learn from each other throughout the process. We try to keep in mind that it is not easy for an adult to step into a child’s world, and likewise it is not easy for a child to step into an adult’s world. We have found that no single technique can give teams all the answers they are looking for, so a combination of techniques has been adapted or developed that form the methodology of cooperative inquiry (Druin, 1999). These techniques do not necessarily offer a magic formula for working with children, but rather a philosophy and approach to research that can be used to gather data, develop prototypes, and forge new research directions. Cooperative inquiry activities include:
(1) Contextual Inquiry: To observe what children do with what technologies they currently have. Younger children can have a difficult time abstractly discussing the world around them. Merely asking children what they want in new technologies will not produce the user input that is needed for the design process. Therefore, observation techniques specifically developed to understand children’s exploratory activity patterns are used. This includes having adults observe children and having children observe children using technology. Notes are taken with drawings, words, and video. It is critical that children are as much a part of the data collection as adults. For example, we rarely if ever have adults use video cameras to capture children’s interactions. We found that the camera was obtrusive and children could rarely feel comfortable (Druin et al., 1999; Druin et al., 1997). We have since discovered however, if children take the video footage of other children, the discomfort disappears.
When using contextual inquiry observation and note-taking, we often look for children outside of the team to observe, so that all team members (children and adults) have a chance to "watch". The note-taking techniques of adults and children obviously differ. But two techniques been developed to suit the needs of adults and children. We have found that adults gather data effectively by writing short text descriptions of conversation and activities (see Table 1). On the other hand, children seem to be effective in combining drawings with small amounts of text to create cartoon-like flow charts (see Figure 8). Once the adult notes have been compiled for a session, the adult notes are compared with the child notes. The adult notes are highlighted in the places that the child researchers have recorded in their notes. In this way, child and adult perspectives are captured. We have found that the child researcher summaries of the data, enabled adult partners to see ideas they had originally overlooked (Druin, 1999).
Besides comparing the adult and child notes, we also analyze the text
descriptions of the adults. We begin by analyzing the quotes and activities
for activity patterns. By this we mean experiences children have
repeatedly during a session. After identifying these patterns of activity,
we are able then to identify the roles that children take on as
they use different technologies (Note: these roles are not the same roles
described in this paper, but the roles of technology use—e.g., child as
storyteller, collaborator, leader). Lastly, we look at all of the previous
information and formulate design suggestions that can lead to further development
of project work.
|0932||F: No, you’re only erasing all the time. Lena, stop!||Struggling
[To adult:] Can you help me, I’m trying to draw a circle.
F: I know how to!
|Asks adult to help her||Seeks help||Learner||Help option|
Hello, I want to move it here!
F: Get the red instead!
|L. is taking the mouse from F, puts the tools back again by help of the box||Struggling for control of input device||Leader||Multiple input devices and/or collaborative software tools|
F: Now you really have to stop!
|L. takes the hand, takes the yellow crayon, draws a curve||Drawing||Artist
Not a head!
F: What do you want, then?
L: A sun!
|F. takes the mouse, rubs everything away||Struggling for control of input device||Leader||Multiple input devices and/or collaborative software tools|
Table 1: Portion of a contextual inquiry diagram created by adults observing
two 7 year-old children in a School
Figure 8: Contextual inquiry notes by a 7-year old child in the United States
(2) Participatory Design: To hear what children have to say directly by collaborating on the development of "low tech" prototypes. In addition to collecting data through observation, we have found that there is a need to hear from children directly (Druin et al., 1997; Druin, 1999). Participatory design techniques can enhance what we have come to understand from observation. This does not mean that participatory design must follow contextual inquiry. However, we have found that contextual inquiry enables us to first explore numerous ideas through observation. Then, during our data visualization, we focus on an area of interest to pursue more in-depth participatory design prototyping. For example, our contextual inquiry observations led to an understanding that children wanted to collaborate with technology. This insight was taken into a participatory design session where low-tech materials were used to prototype collaborative storytelling technologies for the future (e.g., see Figure 9).In these participatory design sessions, small groups of three to four children with two to three adults create low-tech prototypes out of paper, clay, glue, crayons, etc. The low-tech tools give equal footing to adults and children. We have found that there is rarely a need to teach people how to prototype, since using basic art supplies comes naturally to the youngest and oldest design partners. The low-tech prototypes that are developed support the brainstorming and idea generation stage of the design process. This form of prototyping is inexpensive, yet quite effective in quickly brainstorming new ideas or directions. It is from these low-tech prototypes that high-tech prototypes emerge.
Figure 9: An example of a "low-tech prototype" for a new storytelling technology. The design team explained that you can tell a story by talking through the straw/microphone. Feathers on the machine tickle you and make you laugh at the story. You can look into the machine’s eyes to see the story going on. In addition, the machine can fly to other places to re-tell and collect other stories (KidStory Researcher Notes, Nottingham, England, November, 1998).
(3) Technology Immersion: To observe what children do with extraordinary amounts of technology (similar to what they might have in the future). This process came out of our need to understand how children can use large amounts of technology over a concentrated period of time (Druin et al., 1997). We have found that if children are only observed with the technology resources they currently have, then what they might do in the future with better circumstances could be missed. Many children still have minimal access to technology in their homes or school. If time is not a limiting factor then access to the newest technologies can be. However, in the future we expect these limitations to change. Therefore, by establishing today a technology-rich, time-intensive environment for children, the observation techniques of contextual inquiry can be used to capture many of the activity patterns that perhaps might be over-looked in lesser circumstances. With technology immersion, it is critical that children not only have access to technology intensively, but are also supported as decision-makers in this technology environment. All too often, we are only able to glimpse what children do with technology, and those activities are heavily influenced by what adults say they must be. There must be the freedom for children to accomplish a task that is meaningful for them. Without these ingredients, it is difficult to understand children’s technology wants or needs. Technology immersion experiences can be as large as a CHIkids program (60+ children and 25+ adults) or it can be as small as camp-like experience for six children and four adults in our own labs (Druin, 1999).
The combination of observation, low-tech prototyping, and time-intensive technology use, we have found can lead to the development of new technologies. Activity patterns and roles can suggest new design directions. Artifact analysis on low-tech prototypes can suggest new technology features. And technology immersion with alpha and beta technologies can lead to revision and eventual products.
6.3 Impact of this Role on Technologies
The impact that children can have as design partners is enormous. Throughout the design and development process, children’s voices are heard and can have a dramatic effect on the design of new technologies. While children are a critical part of the team, they do not dictate what must happen. They contribute as partners with adults in changing and designing technologies.
How much can this role impact new technologies? Again, as with the role of informant, the amount of impact truly depends on the university researchers and/or industry professionals that are a part of the team. Being a design partner with children is not something that comes naturally for adults and therefore, can slow down a team when a difficult situation arises. As with any interdisciplinary team of researchers, diverse individuals and experiences can offer a richness of ideas and talents. It can also be difficult to negotiate effective collaborations and communication paths. Therefore, while children can have a great deal of impact throughout the development process, it depends very much on the context of the design partners.
6.4 Challenges of Child as Design Partner
The unique challenge of this role, is that adults are not in charge, but neither are children. Design partners must negotiate team decisions. This is no easy task when children are accustomed to following what adults say, and adults are accustomed to being in charge. Methods of communication, collaboration, and partnership must be developed that can accommodate children and adults. Due to this unique challenge, the development process can take more time than with other roles. If tight deadlines are looming, this can be very difficult on a team.
When children are design partners, the traditional structures of school can also be a challenge to negotiate. The design team activities must work around the limitations of the school day. If children are to be an on-going part of a design team that is not located in a school, then parents must take on the burden of transportation as well. In addition, the challenge of an on-going partnership with children must also be considered. No longer are children only a part of the research activities for a day, or a month. On-going years of collaboration, means at a very young age, a commitment to research team activities that can infringe on a child’s afterschool activities.
Another challenge that must be overcome is the difficulty in finding researchers or industry professionals that want to work with children as partners. It is assumed that educators have been trained to do this kind of work, but they have been taught to "teach" not "partner" with children, and therefore, old habits must be challenged. With computer scientists, artists, and many other disciplinary professionals, the patience, experience, or desire to work with children may not be the reasons why they went into their respective professions. Therefore, team members need to be selected that can enjoy the "messiness," noise, and unconventional research activities this kind of collaboration can bring.
Yet another challenge may be in deciding how to best understand the changes that are occurring in child and adult partners. Traditional methods of observation or testing of children may get in the way of developing a sense of partnership among team members. Educational researcher Jan Hawkins has pointed out, it is critical that we develop evaluation methods that can be "a system in which the pedagogy is not in tacit conflict with the accounting." (Hawkins, 1996). This is no small challenge if children and adults are truly to be partners. Therefore, we believe that it is important to look for change in social and cognitive development using procedures that are supportive of the partnership experience.
6.5 Strengths of Child as Design Partner
The strength of this role for children is that they can feel quite empowered and challenged by the design partner process. Children have so few experiences in their lives where they can contribute their opinions and see that they are taken seriously by adults. This experience can build confidence in children academically and socially. It can also produce what we have come to call "design-centered learning" (Druin, 1999). This is a kind of learning that can come out of design experiences. We have seen that children and adults can experience changes over time due to their partnership and common design goals. Children can grow to see themselves as something more than users of technology. They can come to believe that they can make a difference. In the case of the KidStory project, we asked all children to keep a journal in the first year. After coding these journals, we found that there were 13 instances in the Fall, as opposed to 164 instances in the Spring where children displayed examples of being an inventor. While there are almost 100 children we work with, 164 does not seem like a great deal overall, but the change over time is very interesting. We are examining if this frequency change continues in subsequent years of the project.
For adults, they too can change as collaborators, researchers, and developers. Research methods long-used by experienced professionals may have to change due to the introduction of children. In addition, research directions may drastically change, again thanks to this collaboration with children. A unique strength of the design partnering experience is that there is no waiting to find out what direction to pursue. Instant feedback from children at every moment can be had if needed. This offers a great deal of flexibility for development activities. If researchers know that children will always be available at certain times, then less formal schedules need to be made.
Another strength of this role is the impact that such a role can have
on new technologies. For educators, parents, and children, it can mean
innovative technologies for teaching, entertainment, and learning. While
this role of design partners is still relatively new, it has shown promising
results for future new technologies.
It is that process of how we come to understand users, which our HCI
community must continually explore and refine. The users we must understand
are many times children, with their unique needs and strengths. There are
many ways to come to know what children want and need in new technologies.
It is our challenge to understand those ways, and take advantage of what
they have to offer. This paper has discussed the various roles children
can play in the design of new technologies (see Table 2 for summary). With
each role there are difficulties, complexities, demands, and exciting possibilities
for children and adults. Depending on the development goals, research questions,
resources, and personal philosophies, a certain role for children may be
most appropriate. It is important to remember that no one role is suitable
for all research and development needs.
||Primarily academic researchers|
|Tester||A few examples in the 1970s—Began primarily in the late ‘80s/ early ‘90s||
|Design Partner||Mid 1990s||
||Primarily academic researchers with industry professionals beginning|
Table 2: Summary of the roles of children in the design process
In analyzing these four roles for children, we may wonder if there are ever inappropriate roles for children in the design process. Are there roles that children should not be asked to consider? This can be answered by asking if there are ever inappropriate roles for artists, or educators, or even computer scientists in the design of new technology. I believe the answer is yes. If we ask people to be something that they cannot be, then it is inappropriate. If we do not take advantage of all that an artist or a teacher or a musician can offer the design process, then it is wrong. I believe the same can be said for children. We must understand what they have to offer the design team process. We cannot expect them to program as well as computer scientists. We cannot expect them to know what educational goals need to be covered in a school curriculum as a well as a teacher does. But we can expect children to tell us what excites and bores them, what helps them learn, and what can be used in their homes or schools. We can expect children to be creative, honest collaborators.
In the future, we can look forward to greater challenges given the proliferation
of new technologies and new more demanding users that are young people.
We have a chance to change technology, but more importantly we have a chance
to change the life of a child. Every time a new technology enables a child
to do something they never dreamed of, it offers new possibilities for
This paper could not have been written without all of the children over
the years who have taught me how to be a design partner—in particular my
partners: Alex Kruskal, Hanne Olsen, Isabella Revett, Thomas Plaisant-Schwenn,
Lauren Sumida, and Rebecca Wagner. In addition, I would like to acknowledge
all of my adult colleagues at the University of Maryland that have been
and continue to be my design partners: Jaime Montemayor, Jim Hendler, Angela
Boltman, Lisa Sherman, Gene Chipman, Houman Alborzi, Britt McAlister, Aurelie
Plaisant, Ben Shneiderman, Catherine Plaisant, and Ben Bederson. Our research
could not have been accomplished without the generous support of the Kennedy
Krieger School, 3D Open Motion, Microsoft Corporation, and the United States
Census Bureau. In addition, a significant part of our research has recently
been supported by the European Union’s i3 Experimental School Environments
Initiative 29310. Finally, I would like to acknowledge my husband who is
also my colleague, who enables me to think about grand plans while he builds
our technological dreams.
Report to the President on the use of technology to strengthen K-12 education in the United States. (1997). Executive Office of the President of the United States, Washington, DC.: President's Committee of Advisors on Science and Technology.
Alpert, D., & Bitzer, D. L. (1970). Advances in computer-based education. Science, 167, 1582-1590.
Amarel, M., & Swinton, S. (1975). The introduction of innovative instructional systems: Implementation and program evaluation. American Educational Research Association (AERA) .
Benford, S., et al., (Submitted). Designing Storytelling Technologies to Encourage Collaboration Between Young Children. Human Factors in Computing Systems: CHI 2000 ACM Press.
Berkovitz, J. (1994). Graphical interfaces for young children in a software-based mathematics curriculum. Extended Abstracts of Human Factors in Computing Systems: CHI 94 (pp. 247-248). ACM Press.
Berman, R. (1977). Preschool knowledge of language: What five-year olds know about language structure and language use. C. Pontecorvo (Ed.), Writing development: An interdisciplinary view (pp. 61-76). Amsterdam: John Benjamin’s Publishing.
Bjerknes, G., Ehn, P., & Kyng, M. (1987). Computers and democracy: A Scandinavian challenge. Aldershot, UK: Alebury.
Blomberg, J. L., & Henderson, A. (1990). Reflections on participatory design: Lessons from the Trillium experience. Human Factors in Computing Systems: CHI 90 (pp. 353-359). ACM Press.
Blomberg, J., Giacomi, J., Mosher, A., & Swenton-Wall, P. (1993). Ethnographic field methods and their relation to design. D. Schuler, & A. Namioka (Eds.), Participatory design: Principles and practices (pp. 123-156). Hillsdale, New Jersey: Lawrence Erlbaum.
Burov, A. N. (1991). Development of creative abilities of students on the basis of computer technology. First Moscow International HCI'91 Workshop (pp. 289-296).
Candy, L., & Edmonds, E. A. (1982). A study of the use of a computer as an aid to English teaching. International Journal of Man-Machine Studies, 16(3), 333-339.
Colella, V., Borovoy, R., & Resnick, M. (1998). Participatory simulations: Using computational objects to learn about dynamic systems. Extended Abstracts of Human Factors in Computing Systems: CHI 98 (pp. 9-10). ACM Press.
Cypher, A., & Smith, D. (1995). KidSim: End-user programming of simulations. Human Factors in Computing Systems: CHI 95 (pp. 27-34). ACM Press.
Davis, R. B. (1976). The children's mathematics project: The Syracuse/Illinois component . Journal of Children's Mathematical Behavior, 1, 2-59.
Davis, R. B. (1984). Learning mathematics: The cognitive science approach to mathematics education. New Jersey: Ablex Publishers.
Druin, A., Bederson, B., Boltman, A., Miura, A., Knotts-Callahan, D., & Platt, M. (1999). Children as our technology design partners. A. Druin (Ed.), The design of children's technology . San Francisco, CA: Morgan Kaufmann.
Druin, A., & Solomon, C. (1996). Designing multimedia environments for children: Computers, creativity and kids. New York: Wiley.
Druin, A., Stewart, J., Proft, D., Bederson, B. B., & Hollan, J. D. (1997). KidPad: A design collaboration between children, technologists, and educators. Human Factors in Computing Systems: CHI 97 (pp. 463-470). ACM Press.
Druin, A. (1987). NOOBIE: The Animal Design Playstation. SIGCHI Bulletin, 20(1), 45-53.
Druin, A. (1996). A place called childhood. Interactions, 3(1), 17-22.
Druin, A. (1999). Cooperative inquiry: Developing new technologies for children with children. Human Factors in Computing Systems: CHI 99 (pp. 223-230). ACM Press.
Druin, A., Hendler, J., Monte, J., Boltman, A., McAlister, B., Plaisant, A., Sumida, L., Plaisant, T., Olsen, H., Smith, R., & Kowasky, A. (1999). Designing PETS: A Personal Electronic Teller of Stories. Human Factors in Computing Systems: CHI 99 ACM Press.
Druin, A., åkesson, K.P., Bayon, V., Bederson, B. B., Benford, S., Boltman, A., Cobb, S., Fast, C., Hamsson, P., Hourcade, J. P., Ingram, R., Kjellin, M., Neale, N., O’Malley, C., Simsarian, K. T., Stanton, D. Sundblad, Y., Svensson, S., Taxén, G., & Wilson, J. (Submitted). KidStory: An International, Interdisciplinary, Intergenerational, Research Partnership. Extended Abstracts of Human Factors in Computing Systems: CHI 2000. ACM Press.
Dwyer, T. (1980). The significance of solo-mode computing for curriculum design. R. P. Taylor (Ed.), The computer in the school: Tutor, tool, tutee (pp. 104-112). New York: Teachers College Press.
Fell, H. J., Ferrier, L. J., Delta, H., Peterson, R., Mooraj, Z., & Valleau, M. (1994). Using the Baby-Babble-Blanket for infants with motor problems: An empiracal study. First Annual ACM Conference on Assistive Technologies (pp. 77-84). ACM Press.
Feurzeig, W., & Papert, S. (1968). Programming languages as a conceptual framework for teaching mathematics. NATO Science Conference on Computers and Learning NATO.
Frye, D., & Soloway, E. (1987). Interface design: A neglected issue in educational software. Human Factors in Computing Systems: CHI+GI 87 (pp. 93-97). ACM Press.
Fulton, K. (1997). Learning in the digital age: Insights into the issues. Santa Monica, CA: Milken Exchange on Education Technology .
Goldberg, A. (1984). Smalltalk-80: The interactive programming environment. Reading, MA: Addison-Wesley.
Goldberg, A., & Suppes, P. (1972). A computer-assisted instruction program for exercises on finding axioms. Educational Studies in Mathematics, 4, 429-449.
Goldman-Segall, R. (1998). Points of viewing children's thinking. Mahwah, NJ: Lawrence Erlbaum Associates.
Gould, J. D., & Lewis, C. (1983). Designing for usability--Key principles and what designers think. Human Factors in Computing Systems: CHI 83 (pp. 50-53). ACM Press.
Greenbaum, J. (1993). A design of one's own: Toward participatory design in the United States. D. Schuler, & A. Namioka (Eds.), Participatory design: Principles and practices (pp. 27-37). Hillsdale, New Jersey: Lawrence Erlbaum.
Greenbaum, J., & Kyng, M. (1991). Design at work: Cooperative design of computer systems. Hillsdale, NJ: Lawrence Erlbaum.
Grudin, J. (1990). The computer reaches out: The historical continuity of interface design. Human Factors in Computing Systems: CHI 90 (pp. 261-268). ACM Press.
Halgren, L., Fernandes, T., & Thomas, D. (1995). Amazing Animation: Movie making for kids. Human Factors in Computing Systems: CHI 95 (pp. 519-524). ACM Press.
Heller, S. (1998 August). The meaning of children in culture becomes a focal point for scholars. The Chronicle of Higher Education, p. A14-A16.
Hoyles, C. (1985). Developing a context for Logo in school mathematics. Theoretical Papers: Logo85 (pp. 23-42). MIT .
Hunka, S. (1973). The computer-aided instruction activities of the Division of Educational Research Services at the University of Alberta. International Journal of Man-Machine Studies, 5(3), 329-336.
Inkpen, K., Booth, K. S., Klawe, M., & McGrenere, J. (1997). The effect of turn-taking protocols on children's learning in mouse-driven collaborative environments. Graphics Interface: GI 97 (pp. 138-145). Canadian Information Processing Society.
Jackson, S. L., Krajcik, J., & Soloway, E. (1998). The design of learner-adaptable scaffolding in interactive learning environments. Human Factors in Computing Systems: CHI 98 (pp. 197-194). ACM Press.
Johnson, J., Ehn, P., Grudin, J., & Nardi, B. T. K. (1990). Participatory design of computer systems. CHI 90 (pp. 141-144). ACM Press.
Koenemann, J., Carroll, J. M., Shaffer, C. A., Rosson, M., & Abrams, M. (1999). Designing collaborative applications for classroom use: The LiNC Project. A. Druin (Ed.), The design of children's technology. San Francisco, CA: Morgan Kaufmann.
Lepper, M. R. (1985). Microcomputers in education: Motivational and social issues. American Psychologist, 40(1), 1-18.
Lester, J. C., Converse, S. A., Kahler, S. H., Barlow, S. T., Stone, B. A., & Bhogal, R. (1997). The persona effect: Affective impact of animated pedagogical agents. Human Factors in Computing Systems: CHI 97 (pp. 359-366). ACM Press.
Loh, B., Radinsky, J., Russell, E., Gomez, L., Reiser, B. J., & Edelson, D. C. (1998). The progress portfolio: Designing reflective tools for a classroom context. Human Factors in Computing Systems: CHI 98 (pp. 627-634). ACM Press.
Malone, T. W. (1982). Heuristics for designing enjoyable user interfaces. Human Factors in Computing Systems: Gathersburg Conference (pp. 63-68). ACM Press.
McGilly, K. (1995). Classroom lessons: Integrating cognitive theory of classroom practice. Cambridge, MA: MIT Press.
Montford, S. J., Vertelney, L., Bauersfield, P., & Gomoll, K. (1990). Designers: Meet your users. Human Factors in Computing Systems: CHI 90 (pp. 439-442). ACM Press.
Moshell, J. M., & Hughes, C. E. (1996). The virtual academy: A simulated environment for constructionist learning. International Journal of Human-Computer Interaction, 8(1), 85-110.
Mumford, E., & Henshall, D. (1979). Designing participatively: A participative approach to computer systems design. UK: Manchester Business School.
Neal, A. S., & Simons, R. M. (1983). Playback: A method for evaluating the usability of software and its documentation. Human Factors in Computing Systems: CHI 83 (pp. 78-82). ACM Press.
Nicol, A. Interface design for hyperdata: Models, maps, and cues. Human Factors Society 32nd Annual Meeting (pp. 308-312).
Nielsen, J., & Molich, R. (1990). Heuristic evaluation of user interfaces. Human Factors in Computing Systems: CHI 90 (pp. 141-144). ACM Press.
Noirhomme-Fraiture, M., Charriere, C., Vanderdonckt, J. M., & Bernard, C. (1993). ERGOLAB: A screen usability evaluation tool for children with cerebral palsy. Extended Abstracts of Human Factors in Computing Systems: INTERCHI'93 ACM Press.
Oosterholt, R., Kusano, M., & de Vries, G. (1996). Interaction design and human factors support in the development of a personal communicator for children. Human Factors in Computing Systems: CHI 96 (pp. 450-457). ACM Press.
Papert, S. (1972). Teaching children thinking. Conference on Computers in Education: IFIPS (pp. 223-230).
Papert, S. (1977). A learning environment for children. R. J. Seidel, & M. Rubin (Eds.), Computers and communication: Implications for education (pp. 271-278). New York: Academic Press.
Papert, S. (1980). Mindstorms: Children, computers and powerful ideas. New York: Basic Books.
Pausch, R., Vogtle, L., & Conway, M. (1992). One dimensional motion tailoring for the disabled: A user study. Human Factors in Computing Systems: CHI 92 (pp. 405-411). ACM Press.
Pelegrino, J. W., Hickey, D., Heath, A., Rewey, K., Vye, N. J., & Cognition and Technology Group. (1991). Assessing the outcomes of an innovative instructional program: The 1990-1991 implementation of the Adventures of Jasper Woodbury. Vanderbilt University Technical Report, 90-1, Learning Technology Center.
Piaget, J. (1971). Psychology and Epistemology: Towards a theory of knowledge. New York: Viking Press.
Piaget, J. (1973). To understand is to invent: The future of education. New York: Grossman.
Piernot, P., Felciano, R. M., Stancel, R., Marsh, J., & Yvon, M. (1995). Designing the PenPal: Blending hardware and software in user interface for children. Human Factors in Computing Systems: CHI 95 (pp. 511-518). ACM Press.
Plowman, L. (1992). An ethnographic approach to analyzing navigation and task structure in interactive multimedia: Some design issues for group use. Conference on People and Computers: HCI'92 (pp. 271-287).
Plowman, L., Luckin, R., Laurillard, D., Stratfold, M., & Taylor, J. (1999). Designing multimedia for learning: Narrative guidance and narrative construction. Human Factors in Computing Systems: CHI 99 (pp. 310-317). ACM Press.
Rader, C., Brand, C., & Lewis, C. (1997). Degrees of comprehension: Children's understanding of a visual programming environment. Human Factors in Computing Systems: CHI 97 (pp. 351-358). ACM Press.
Rose, A., Ding, W., Marchionini, G., Beal, J., & Nolet, V. (1998). Building an electronic learning community: From design to implementation. Human Factors in Computing Systems: CHI 98 (pp. 203-210). ACM Press.
Salzman, M. C., Dede, C., & Loftin, R. B. (1999). VR's frames of reference: A visualization technique for mastering abstract multidimensional information. Human Factors in Computing Systems: CHI 99 (pp. 489-495). ACM Press.
Scaife, M., & Rogers, Y. (1999 Kids as informants: Telling us what we didn’t know or confirming what we knew already. A. Druin (Ed.), The design of children's technology . San Francisco, CA: Morgan Kaufmann.
Scaife, M., Rogers, Y., Aldrich, F., & Davies, M. (1997). Designing for or designing with? Informant design for interactive learning environments. Human Factors in Computing Systems: CHI 97 (pp. 343-350). ACM Press.
Schuler, D., & Namioka, A. (1993). Participatory design: Principles and practices. Hillsdale, NJ: Lawrence Erlbaum.
Searle, B., Lorton, P., & Suppes, P. (1974). Structural variables affecting CAI performance on arithmetic word problems of disadvantaged and deaf students . Educational Studies in Mathematics, 5, 371-384.
Smith, D. C., & Cypher, A. (1999). Making programming easier for children. A. Druin (Ed.), The design of children's technology . San Francisco, CA: Morgan Kaufmann.
Smith, B. K., & Reiser, B. J. (1998). National Geographic Unplugged: Classroom-centered design of interactive nature films. Human Factors in Computing Systems: CHI 98 (pp. 424-431). ACM Press.
Solomon, C. (1986). Computer environments for children. Cambridge, MA: MIT Press.
Solomon, G. (1979). Interaction of media, cognition, and learning: An exploration of how symbolic forms cultivate mental skills and affect knowledge acquisition. San Francisco: Jossey-Bass Publishes.
Steiner, K. E., & Moher, T. G. (1992). Graphic StoryWriter: An interactive environment for emergent storytelling. Human Factors in Computing Systems: CHI 92 (pp. 357-364). ACM Press.
Stewart, J., Bederson, B., & Druin, A. (1999). Single Display Groupware: A model for co-present collaboration. Human Factors in Computing Systems: CHI 99 (pp. 286-293). ACM Press.
Stodolsky, D. (1970). The computer as a psychotherapist. International Journal of Man-Machine Studies, 2(4), 327-350.
Strommen, E. (1994). Children's use of mouse-based interfaces to control virtual travel. Human Factors in Computing Systems: CHI 94 (pp. 405-410). ACM Press.
Strommen, E. (1998). When the interface is a talking dinosaur: Learning across media with Actimates Barney. Human Factors in Computing Systems: CHI 98 (pp. 288-295). ACM Press.
Sundblad, Y. (1987). Quality and interaction in computer-aided graphic design (Utopia Report #15). Stockholm: Arbetslivscentrum.
Super, D., Westrom, M., & Klawe, M. (1996). Design issues involving entertainment. Extended Abstracts of Human Factors in Computing Systems: CHI 96 (pp. 179-180). ACM Press.
Suppes, P. (1969). Computer technology and the future of education. R. Atkinson, & H. A. Wilson (Eds.), Computer-assisted instruction: A book of readings (pp. 41-47). New York: Academic Press.
Umaschi Bers, M., Ackerman, E., Cassell, J., Donegan, B., Gonzalez-Heydrich, J., DeMaso, R., Strohecker, C., Lualdi, S., Bromley, D., & Karlin, J. (1998). Interactive storytelling environments: Coping with cardiac illness at Boston's children's hospital. Human Factors in Computing Systems: CHI 98 (pp. 603-610). ACM Press.
Verburg, G., Field, D., St. Pierre, F., & Naumann, J. (1987). Toward universality of access: Interfacing physically disabled students to the ICON Educational Microcomputer. Human Factors in Computing Systems: CHI+GI 87 (pp. 81-87). ACM Press.
Wallace, R., Soloway, E., Krajcik, J., Bos, N., Hoffman, J., Hynter, H. E., Kiskis, D., Klann, E., Peters, G., Richardson, D., & Ronen, O. (1998). ARTRMIS: Learner-Centered Design of an information seeking environment for K-12 education. Human Factors in Computing Systems: CHI 98 (pp. 195-202). ACM Press.
Wilson, K. S. (1988). Palenque: An interactive multimedia digital video interactive prototype for children. Human Factors in Computing Systems: (pp. 275-279). ACM Press.
Wixon, D., & Holtzblatt, K. (1990). Contextual design: An emergent view of system design. Human Factors in Computing Systems: CHI 90 (pp. 329-336). ACM Press.