The Effectiveness of Worked Examples on a Game-Based Problem-Solving Task

Worked Examples 1

<Draft v.3 04/09/04>

The Effectiveness of Worked Examples on a Game-Based Problem-Solving Task

A Proposal

Submitted to: Dr. Harold O’Neil (Chair)

Dr. Edward Kazlauskas

Dr. Robert Rueda

by

Chun-Yi (Danny) Shen

University of Southern California

3401 S. Sentous Ave., # 148

West Covina, CA 91792

(626) 715-6130;

In Partial Fulfillment for Ed.D. in Learning and Instruction

Table of Contents

ABSTRACT…………………………………………………………….…………6

CHAPTER I: INTRODUCTION……………………………………..……….…..7

Background of the Problem…………………….………………………………….7

Purpose of the Study……………………………………………………………….9

Significance of the Study……………………………………………………………9

CHAPTER II: LITERATURE REVIEW….………………………………………..10

Relevant Studies…………………………………………………………………….10

Games and Simulations……………………………………..……………………..10

Theories of Games and Simulations…..…..……….……………………….10

Training Effectiveness of Games.…………..….…………………………..11

Promotion of Motivation…………………………………………..12

Enhancement of Thinking Skills………………………………….18

Facilitation of Metacognition……………………………………..23

Improvement of Knowledge………………………………………23

Building Attitudes…………………………………………………28

Design of Games and Simulations………………………………………..28

Summary………………………………..………………………………….29

Problem Solving……………….………………………………………………...... 29

Significance of Problem-Solving Skills……………………………………..32

Assessment of Problem Solving…………………………….……………..32

Measurement of Content Understanding……………………………32

Measurement of Problem Solving Strategies……………...... 33

Table of Contents (Cont.)

Measurement of Self-Regulation……………....…………………….42

Summary………………………………………………………..……….…..44

Worked Example……………………………………………………………………..45

Theories of Worked Examples………………………………………….…..47

Cognitive Load Theory……………………………………………..47

Schema……………………………………………………………..49

Scaffolding…………………………………………………………..50

ACT-R……………………………………………………………….51

Design of Worked Examples………………………………………………...52

Before vs. After………………………………………………………52

Part vs. Whole………………………………………………………53

Backward Fading vs. Forward Fading………………………………53

Text vs. Diagrams…………………………………………………..54

Visual vs. Verbal…………………………………………………..54

Steps vs. Subgoals………………………………………………….55

Summary……………………………………………………………………56

Summary of the Literature…………………………………………………………..56

CHAPTER III: METHODOLOGY……………………………………………….57

Research Design……………………………………………………………….….57

Research Hypotheses……………………………………………………………..57

Pilot Study…………………………………………..…………………………….57

Participants…………………………………………………….…………..58

Table of Contents (Cont.)

Puzzle-Solving Game…….……………………………….………………58

Knowledge Map…………….…..….…………………………………….61

Measures…………………………………………………………….…….63

Content Understanding Measure………………………………..…63

Domain-Specific Problem-Solving Strategies Measure………..…...65

Self-Regulation Questionnaire……………………………………66

Procedure..…..……………………………………………………………..67

Worked Examples…………………………………………………..67

Time Chart of the Study…………………………………….67

Data Analysis.…………….……..……………………………………………68

Main Study….…………………….…………………………………………………68

Method of the Main Study…………………………………….…………….68

Participants………………………………..…………….…………68

Game…….………..………………………………………………..68

Measures…………………………………………………………………….69

Knowledge Map…………………………………………..……….69

Domain-Specific Problem-Solving Strategies Measure…………... 69

Self-Regulation Questionnaire……………………………..……….69

Procedure..…..……………………………………………………………..70

Computer-Based Knowledge Map Training……….………………70

Game Playing……………………………………………..……….70

Worked Examples…………………………………………………..70

Table of Contents (Cont.)

Data Analysis………………………………………………………………70

REFERENCES ………………………………….……..…………………………72

Appendix AExpert Map………………………………………………….…90

AppendixBProblem Solving Strategy Questionnaire.………………..…..91

Appendix C Self-Regulation Questionnaire………………………………93

ABSTRACT

Training by computer games is one of the important activities in many industries (Adams, 1998; Chambers, Sherlock, & Kucik, 2002; Faria, 1998; Lane, 1995; O’Neil & Andrews, 2000; Ruben, 1999; Washbush & Gosen, 2001). Researchers point out that simulations and games are widely accepted as a powerful alternative to traditional ways of teaching and learning, with the merits of facilitating learning by doing(Mayer, Moutone, & Prothero, 2002; Rosenorn & Kofoed, 1998; Schank, 1993). Problem-solving skill may be effectively improved by computer games (Mayer, 2002). Problem solving is one of the most significant competencies whether in job settings or in schools, and, as a result, teaching and assessing problem solving become one of the most significant educational objectives (Mayer, 2002). In addition, according to previous studies (Cooper & Sweller, 1987; Sweller & Cooper, 1985),worked example can effectively facilitate problem solving skill by enhancing schema construction and automation, reduce cognitive load, and provide assistance during learning. Therefore the researcher plans to conduct anexperimentalstudy to examinethe effectiveness of worked examples on problem solving skill, including content understanding, domain-specific problem-solving strategies, and self-regulation, in a game-based environment.

In the first part of this proposal, the author will review the relevant literature on computer games and simulations, problem solving, and worked examples. The second part of this proposal will include a pilot and a main study. The pilot study will focus on a formative evaluation. The main study will examine the effectiveness of worked examples on a game-based problem-solving task.

CHAPTER I

INTRODUCTION

Background of the Problem

As Rieber (1996) pointed, play appears to be an important role in psychological, social, and intellectual development. A study by Betz (1995) shows that computer games facilitate learning by visualization, experimentation, and creativity of play. Also, Huntington (1984) indicates that computer games usually include problems that enhance critical thinking. Computer games have been used in many different industries, such as academic (Adams, 2000), business (Adams, 1998; Faria, 1998; Lane, 1995; Wabush & Gosen, 2001), military (O’Neil & Andrews, 2000; Chambers, Sherlock, & Kucik, 200), and medical (Ruben, 1999). Quinn (1991, 1996) indicates that computer games are effective tools for problem solving training, especially for adventure games. O’Neil and Fisher (2002) suggests that computer games have four major characteristics: (a) complex and diverse approaches to learning processes and outcomes; (b) interactivity; (c) skill to address cognitive as well as affective learning issues, and most importantly, (d) motivation for learning. As O’Neil and Fisher (2002) pointed, the effectiveness of instructional games has been categorized into five major aspects: promotion of motivation, enhancement of thinking skills, facilitation of metacognition, improvement of knowledge and skills, and improving attitudes.

The two major alternative techniques to learn problem solving that have been studies are the use of worked examples and goal-free problems (Owen & Sweller, 1985; Sweller, Mawer, & Ward, 1983; Sweller, 1990). In last 20 years, many researchers paid a considerable amount attention on worked examples and concluded that worked examples instruction is superior to the conventional problem solving instruction, especially in the field of music, chess, athletics, mathematics, computer programming, and physics (Carroll, 1994; Tarmizi & Sweller, 1988; Sweller, Chandler, Tierney, & Cooper, 1990; Chi, Bassok, Lewis, Reimann, & Glaser, 1989; Ward & Sweller, 1990; Renkl, Atkinson, Maier, & Staley, 2002; Reimann, 1997; VanLehn, 1996). A number of researchers investigated the efficacy of using worked examples in classroom instruction and provided evidence in favor of worked examples instruction rather than problem solving practice (Zhu & Simon, 1987; Carroll, 1994; Ward & Sweller, 1990; Cooper & Sweller, 1987). As Zhu and Simon (1987) pointed out, worked example can be an appropriate and acceptable substitute instructional method comparing to conventional classroom activity.

According to Gagne (1977), “educational programs have the important ultimate purpose of teaching students to solve problems – mathematical and physical problems, health problems, social problems, and problems of personal adjustment.”Some psychologists conclude that most human learning engages problem-solving activities (Anderson, 1993). Problem solving is one of the most significant competencies whether in job settings or in schools, and, as a result, teaching and assessing problem solving become one of the most significant educational objectives (Mayer, 2002). As Mayer (2002) pointed out, teaching problem-solving transfer has become one of the most critical educational. As O'Neil (1999) pointed out, problem-solving skill is a critical competency requirement of college students and employees. The National Center for Research on Evaluation, Standards, and Student Testing (CRESST) has conducted many researches on problem solving (O’Neil, 2002; O'Neil, Ni, Baker, Wittrock, 2002; O’Neil & Herl, 1998; O’Neil, Baker, & Fisher, 1998; Baker & O'Neil, 2002; Baker & Mayer, 1999). The CRESST adapts the problem-solving models of Baxter, Elder, and Glaser (1996), Glaser, Raghavan, and Baxter (1992), Mayer and Wittrock (1996), and Sugrue (1995). The CRESST model of problem-solving consists of three components: (a) content understanding, (b) problem-solving strategies, and (c) self-regulation (O’Neil, 1999; Baker & Mayer, 1999).

Purpose of the Study

The main purpose of this study is to examine the effectiveness of worked examples on problem solving in a game-based environment. The researcher will use the problem-solving assessment model developed by the National Center for Research on Evaluation, Standards, and Student Testing (CRESST) to measure the three components of problem solving skill, which are content understanding, problem solving strategy, and self-regulation (Herl, O’Neil, Chung, Schacter, 1999; Mayer, 2002; Baker & Mayer, 1999). The dissertation will consist of a pilot and a main study. The purpose of the pilot study is tomake sure about the feasibility of the computer game program, the format of worked example, and the assessment tools for problem solving skill.

Significance of the Study

There are very few studies comparing learning from worked examples only with learning from problem solving. Instead, Sweller and his colleagues (e.g., Mwangi & Sweller, 1998; Sweller & Cooper, 1985) have conducted several classic studies consisting of examples followed by similar problems (example-problem pairs). Studies on worked examples conducted by other researchers (e.g. Renkl, 1997) have focused on learning from examples only (Renkl, Atkinson, Maier, & Staley, 2002). In addition, there are many worked examples studies in the field of mathematics, computer programming, and physics (Tarmizi & Sweller, 1988; Sweller, Chandler, Tierney, & Cooper, 1990; Chi, Bassok, Lewis, Reimann, & Glaser, 1989; Ward & Sweller, 1990), but there is no study that investigates the effectiveness of worked examples in game-based problem solving task.

CHAPTER II

LITERATURE REVIEW

Relevant Studies

Games and Simulations

As Gredler (1996) defined that “game consist of rules that describe allowable player moves, game constraints and privileges (such as ways of earning extra turns), and penalties for illegal (nonpermissable) actions. Further, the rules may be imaginative in that they need not relate to real-world events.”

As Christopher and Smith (1999) pointed, a simulation game has four major elements: (a) settings, (b) roles, (c) rules, and (d) scoring, recording, or monitoring. Driskell and Dwyer (1984) define a game as a rule-governed, goal-focused, microworld-driven activity incorporating principles of gaming and computer-assisted instruction. On the other hand, Gredler(1996)describes a game as an environment with allowable player moves, constraints and privileges, and penalties for illegal actions. In addition, the rules of games do not have to obey those in real-life and can be imaginative.

Theories of Games and Simulations

In learning by doing, in which students work on realistic tasks, a major instructional issue within simulation environments is concerning the proper type of guidance (Mayer, Moutone, & Prothero, 2002). Researchers point out that simulations and games are widely accepted as a powerful alternative to traditional ways of teaching and learning, with the merits of facilitating learning by doing (Mayer, et al.; Rosenorn & Kofoed, 1998; Schank, 1999). In addition, problem-solving skill may be effectively improved by computer games (Mayer, 2002). The potential of learning by doing, such as using simulations and games, has been pointed by educational reformers as an alternative to learning by being told, in which students listen to what teachers have to say (Schank, 1999).

Adams (1998) points out that a game satisfies learners’ visual and auditory sensors and provides flexibility in learning, which makes it an attractive tool for teaching and learning, based on the perspectives of constructivism (Amory, 2001) and dual-coding theory (Mayer & Sims, 1998). However, students perform differently when different scaffolding is provided in simulations and games (Mayer, 2002). For example, students learn better from a computer-based geology simulation when they are given some support about how to visualize geological features. The worst performing group was the group that received the least amount of support beyond basic instructions and the best performing group was the group that received the most support (Mayer et al., 2002).

Training Effectiveness of Games

As Rieber (1996) pointed out, play appears to be an important role in psychological, social, and intellectual development. Play is not opposite to work and seems to be an acceptable method of learning (Blanchard & Cheska, 1985). A study by Betz (1995) shows that computer games facilitate learning by visualization, experimentation, and creativity of play. Also, Huntington (1984) indicates that computer games usually include problems that enhance critical thinking. This comes from the analysis and evaluation of information in computer games in order to decide logical steps toward a goal (Huntington, 1984). Computer games have been used in many different industries, such as academic (Adams, 2000), business (Adams, 1998; Faria, 1998; Lane, 1995; Wabush & Gosen, 2001), military (O’Neil & Andrews, 2000; Chambers, Sherlock, & Kucik, 200), and medical (Ruben, 1999).

Quinn (1991, 1996) indicates that computer games are effective tools for problem solving training, especially for adventure games. O’Neil and Fisher (2002) suggests that computer games have four major characteristics: (a) complex and diverse approaches to learning processes and outcomes; (b) interactivity; (c) skill to address cognitive as well as affective learning issues, and most importantly, (d) motivation for learning (p.6).

As O’Neil and Fisher (2002) pointed, the effectiveness of instructional games has been categorized into five major aspects: promotion of motivation, enhancement of thinking skills, facilitation of metacognition, improvement of knowledge and skills, and improving attitudes.

Promotion of Motivation

Play performs important roles in psychological, social, and intellectual development (Quinn, 1994; Rieber, 1996). As Quinn (1994) defined, play is a voluntary activity that is intrinsically motivating. Internal promotion of motivation can be viewed from two aspects, which are intrinsic and extrinsic. Intrinsic motivation refers to behaviors that are involved in for their own internal reasons, such as joy and satisfaction (Woolfolk, 2001). On the other hand, extrinsic motivation refers to behaviors that engaged in for external reasons, such as obligation and reward (Deci, Vallerand, Pelletier, & Ryan, 1991). Although Mckee (1992) and Billen (1993) argue that games affect cognitive functions, motivation and take players away from the real world, Thomas and Macredie (1994) indicates that games appear to motivate players intrinsically by evoking curiosity. This motivation may be due to the challenge, elements of fantasy, novelty, and complexity of games (Malone, 1980, 1981, 1984; Carroll, 1982; Malone and Lepper, 1987; Rivers, 1990).

As Woolfolk (2001) defined, motivation is usually as an internal state that arouses, directs, and maintains person’s behavior. Quinn (1994, 1997) indicates that games need to combine fun factors with aspects of instructional design and system that include motivational, learning, and interactive elements to benefit educational practice and learning. In addition, learning which is fun seems to be more effective (Lepper and Cordova, 1992). As Bisson and Luckner (1996) say:

The role that fun plays with regard to intrinsic motivation in education is twofold. First intrinsic motivation promotes the desire for recurrence of the experience….Second, fun can motivate learners to engage themselves in activities with which they have little or no previous experience.

Amory, Niacker, Vincent, and Adams (1999) indicate that playing computer games can intrinsically motivate the college students participating in their study. Further more, they concludes that for educators development of learning materials based on adventure game could be a superior tools to attract learners into environments where knowledge is obtained with intrinsic motivation (Amory, Niacker, Vincent, & Adams, 1999). Ehman & Glenn (1991) indicates that simulations could increase students’ motivation, intellectual curiosity,sense of personal control, and perseverance. A study conducted by Ricci, Salas, and Cannon-Bowers (1996) shows a significant positive correlation between the level of students’ enjoyment during training with computer games and their test scores.

Malone (1981) points that intrinsic motivation is an important factor for problem solving. According to Dawes & Dumbleton (2001), computer games as an instructional tool facilitate both intrinsic and extrinsic motivation of students.

There are many empirical evidences supporting that motivation has a positive influence on performance (Clark, 1998; Emmons, 2000; Ponsford & Lapadat, 2001; Rieber, 1996; Urdan & Midhley, 2001; Ziegler & Heller, 2000; Berson, 1996). Computer games have been used in many different industries, such as military (O’Neil & Andrews, 2000), and schools (Adams, 1998; Amory, 2001, Amory, Naicker, Vincent, & Admas, 1999; Barnett, Vitaglione, Harper, Quackenbush, Steadman, & Valdez, 1997), for instruction purpose because computer games could provide diversity, interactivity, and motivation for learning (O’Neil & Fisher, 2002).

It is a common belief that motivation affects performance. In school, especially, motivation has been considered basic for making various pedagogical decisions, and using rewards to motivate school children has become a common practice. Children's interest in exploration and learning starts at an early age but begins to fade as these children progress through the grades. According to Lepper and Hodell (1988), "Motivation can become a problem for many students inour educational system." They addressed the inskill of school systems to enhance and maintain motivation. It is a normal practice for schoolchildren to be extrinsically rewarded with stars and ribbons, with forms of public recognition(stapling good work on bulletin boards), and with a variety of other interventions primarily aimed at controlling behavior. There was a study conducted by Westrom and Shaban (1992) examining the intrinsic motivation in computer games. Intrinsic motivational effects of an instructional computer game (Mission:Algebra) were compared with those of a non-instructional computer game (Lode Runner). The study examined Challenge, Curiosity, Control, and Fantasy aspects of the two games as factors of intrinsic motivation. It also examined differences in intrinsic motivation between boys and girls, among players with different levels of Perceived Creativity (Williams, 1980), and among subgroups formed by these factors. In the outset, motivation for the non-instructional game was higher than for the instructional game and consisted mainly of Challenge and Curiosity. But it dropped significantly as students gained experience. Motivation for the instructional game on the other hand, did not drop with experience but increased marginally and consisted mainly of the factor labeled Control. There were no significant differences between boys and girls in their level of motivation. There were no significant interactions with levels of Perceived Creativity. Each of the Challenge, Curiosity, Control, and Fantasy factors varied in ways that seemed reasonable and contributed to students' overall expression of motivation.

Anderson and Lawton (1992b) have documented that 92.5% of instructors usingtotal enterprise (TE) simulations in college capstone courses grade on simulation performance.It seems axiomatic that people who perform best in a game have learned whatthe game has to teach in addition to being able to apply previously gained businessknowledge to the situation posed by the simulation. However, alternative explanationsfor performance are possible. Ifresearchers can discoverwhy some learn more than others from the simulation experience,teachers could use the information to enhance or supplement their learning environmentswith other materials or pedagogical methods.

Washbush and Gosen (2001) conducted a series of exploratory studies dealing with learning in total enterprise simulations.These studies had three purposes: (a) to examine the validity of simulations as learning tools, (b) to measureany relationships between learning about the simulation and economic performance in the game, and (c) to discoverif some players learn more than others from the same business gaming experience.For this research, they hypothesized that seven types of variablesmight help explain why some students learn more than others: (a) academic skill, (b) attitudes toward the simulation, (c) cohesion, (d) goals, (e) motivation, (f) organization, and (h) struggle. These variables were selected for a variety of reasons, including common sense;because educational, management, or simulation scholars have suggested they influencelearning; because of observations by the researchers themselves; and because ofprevious research. In some cases, variables were chosen because they have either beenpredicted or found to influence performance in a simulation and might also be significantpredictors of learning in the simulation.Motivationwas chosen because it has been found to affect performance in academic(Sjoberg, 1984) and in simulation (Gosenpud & Washbush, 1996b) environments.To pursue the research purposes, 11 studies were conducted between the spring of1992 and the fall of 1997.All participantswere undergraduate students enrolled in a required administrative policy course at theUniversity of Wisconsin–Whitewater. The simulation used was MICROMATIC(1992), a moderately complex top management game. With but two exceptions, studentsplayed the simulation in teams of 2 to 4 members, with the vast majority in3-member teams.Learning was measuredusing parallel forms of a multiple-choice, short-essay examination. The results showed that learningoccurred from simulation play but did not vary with performance.There were 24 indices of motivation measured twice a data set in three data sets. The results reveal that eight individual measures of motivation predicted learning, but sixof these relationships were negative. Therefore, they concluded that there is no relation between simulation-based learning and motivation.