Multitasker, Part 1

In this post, I’ll describe the progress we’ve made on a second game for the York College summer research program. We only have six weeks to design the games, collect data, and present our results at a local conference. There might be time to shower and eat.

Sometimes you just get lucky. A student comes to you with an interest that turns into a pithy concept that’s easy to implement as a game. One of my students expressed an interest in multitasking. This issue has received a lot of attention in recent years due to the rapid proliferation of the Internet and mobile technology. Most people, particularly my college students who text during lectures, operate under the illusion that they can multitask. The illusion of multitasking is convincing because they we can accomplish more than one goal at a time by rapidly switching between tasks. People who operate under this illusion are not entirely misguided. We have an enormous capacity to process large amounts of sensory information from various modalities (e.g., sight and hearing) at the same time. And we have the ability to execute multiple motor commands at the same time (e.g., walking and chewing gum). However, we are particularly terrible at making more than one decision at a time. In attention research, this phenomenon is referred to as a “bottleneck.” As a demonstration, try to read a book while listening to the news. At some point, if you are absorbing the reading, you will miss some critical information on the news. Hal Pashler’s laboratory at UCSD has done revealing experiments on multitasking. They found that when subjects were attending to a stimulus, decisions made in response to a second stimulus were delayed until after a decision about the first stimulus was made. Our educational objective for Multitasker was to demonstrate to students that performance suffers when you attempt to multitask. We predicted that students who played the game would have a different opinion of multitasking relative to students who did not play the game.

Because we have very little time to make this game, we opted to make a board game. Another advantage of making a board game is that players can be challenged with very physical tasks, which we hope will make the lesson more evident. The core mechanic of the game revolves around trying to complete up to four tasks at the same time. A timer will be used to insure players perform each task for a sufficient period of time. We decided to adopt a few of the mini-games in Cranium (i.e., drawing and sculpting). However, some of the tasks will be modified so they can be performed simultaneously. Additionally, we might have to find tasks where the fail state is obvious. For example, it’s obvious when you drop a ball during juggling, but it might not be obvious when a person stops drawing or sculpting. Also, we are still looking for two tasks that can be accomplished with either a foot or using the voice.

To insure that players will not be overwhelmed immediately by performing four tasks at once, the number of possible tasks on any given round of play will be determined in advance in a series of levels. In level 1, the role of a four-sided die will be used to assign the one of the tasks to the player. In level 2, the role of the die, even or odd, will be used to pick two tasks. In level 3, the role of the die will be used to pick two tasks, but the player can choose the third task. In level 4, all four tasks must be performed. If a player successfully performs three challenges in a row, then they advance a level. However, if the player fails a given trial, they are moved back a level. Thus, flow is maintained by introducing and removing tasks. It’s worth noting that this method of using a 3-up/1-down staircase is standard in psychophysics. The object of the game is to be the first to complete Level 4, performing all four tasks three times in a row.

While the core mechanic of the game, objective, and reward/punishment schemes are designed, we are still looking for a fun method of providing feedback. In a previous post, I provided arguments for starting the design process with an educational objective and a game mechanic before designing the user interface. However, as I also mentioned, you can get surprising results from developing the three in parallel. Even though there are still details to complete for the game mechanic, my students also designing a toy that will serve as the centerpiece of the game. They toy will have several functions: (1) It will act as a repository for the game materials; (2) It will act as a method of keeping score and ranking the players; (3) And it will hopefully convey a message about student life. Raph Koster and Jessie Schell both indicate that user interface should be a fun toy. It’s an invitation to play, and I’m hoping my students can come up with some fun ideas that go beyond the traditional game board (a.k.a. “Game Bored”).

Decision Maker, Part 1

Now that I’ve explained our lab’s method of game development and rapid prototyping, I’m going to briefly explain the rationale for some of the games in development and describe the progress we’ve made in the past two weeks. Keep in mind that we only have six weeks to design the games, collect data, and present our results at a local conference. It’s a sprint.

Decision Maker is designed to teach students about decision-making and to help them make better decisions under uncertain conditions. Decision-making was thoroughly studied by Danniel Kahneman and Amos Tversky, who are considered the fathers of behavioral economics. Kahneman received a Nobel Prize for their work in 2002 (after the passing of Tversky in 1996). Their model of decision-making was called Prospect Theory and, in the classic paradigm, subjects must choose between a sure bet (e.g., $100) and the prospect of winning a lottery (e.g., 50% chance of winning $214). The utility of each prospect is defined as the probability multiplied by the value. In our example, the prospect would be the wisest choice because the overall utility of the bet, $107, is higher than the utility of the sure bet, which is $100.  What Kahneman and Tversky discovered, however, was that people often behaved irrationally when presented with bets that had extreme probabilities or values. The astronomical prize money and the misperception of extremely low probabilities explain why people would pay $5 for a state Lotto ticket when the odds of winning are overwhelmingly against the player. While Prospect Theory has been around for years, there have been few efforts to shape behavior given this knowledge. Merely telling people how to make decisions is not enough to alter their behavior. Consequently, Decision Maker is designed to train students to improve their decision-making skills for extreme probabilities and values.

Designing games that are fun, educational, and able to collect data for scientific purposes is challenging. After the educational objectives of our games are established, we develop experimental protocols that will allow us to assess the behavior we wish to shape. Then, we develop game mechanics that compliment those experimental protocols. Our lab uses Tracy’s Fullerton’s Game Design Workshop and Jessie Schell’s Game Design: A Book of Lenses to ensure we address the most critical elements of game design. While it’s not necessary to adhere to this particular order, I recommend starting with a good scientific experiment. However, it is sometimes useful to develop the experiment and game mechanics in separate “sandboxes.” I often ask students to simultaneously design an experiment, a game mechanic, and a fun toy/interface to play with. Combining the results of independent endeavors often produces interesting and unexpected surprises.

Fortunately for Decision Maker, there are several experimental paradigms from behavioral economics to consider. We adopted a method of adjustment paradigm where players indicate the sure bet they would accept in lieu of a given prospect. Players will be presented with positive and negative prospects, and prospects will vary widely in probability and value. The set of probabilities and values were randomly generated to make the mental computation of utility difficult (e.g., 0.2% chance of winning $10,147). Using the staircase method typically employed in psychophysical experiments, the difficulty of the decisions will increase when players make more correct decisions, and the difficulty will decrease if they make errors. The staircase allows us to find the threshold where decisions become less reliable, and it keeps players in a state of flow (where the task is optimally challenging without being too frustrating or too boring). While multiple ascending and descending staircases can be interleaved, we decided to go with a simple descending staircase with consistent step sizes between trials. Our prediction is that subjects who play our game will preform better on a post-test of decision-making relative to control subjects who received equivalent practice in decision making without using games.

After the experiment was designed, we developed the game mechanics. Some of these mechanics are more clearly defined in Tracy’s book, but I’ll briefly define them here. Objectives describe what the player is trying to achieve in the long run. They can describe intermediate goals or the ultimate win/fail states of the game. Resources are items in the game that you are either acquire or get rid of to achieve your objective (e.g., the pieces in Chess or the money in Monopoly). Feedback mechanisms are implemented in games to inform the player about their performance (e.g., point totals or badges). Reward/Punishment Contingencies describe how and at what rate the game will react to a player’s decisions. Different contingencies can have dramatically different effects on behavior. For example, more work is typically elicited from pigeons if rewards are intermittent rather than consistent. With effective contingencies and feedback mechanisms, behavior can be shaped quickly to help the player achieve their objective. Of course, games don’t always have to be friendly. Unreliable feedback mechanisms can be used to achieve a different effect. Flow has been defined in detail elsewhere. Students should focus intensely on how to use all the other mechanics and standard psychophysical procedures to elicit a state of flow in the players. The staircase method should be the starting point when considering flow. Finally, boundaries are rules that prevent players from acting in a particular way. While limiting player behavior might appear to be a fun killer, boundaries often have the opposite effect. For example, soccer is really only fun because players are not allowed to use their hands. This game mechanic is best employed when you are not making progress on a design. If a student is functionally fixed on a particular design that is not working, introduce a boundary to change the designer’s frame of reference.

The core game mechanic for Decision Maker rests on how prospects are presented to the player and how the player evaluates those prospects. Prospects will be presented on playing cards along with scenarios related to student life. Players must write down the sure bet they would accept in lieu of the prospect. The correct utility of the prospect will appear on the back of the card. Players are rewarded for correct decisions by receiving a card and they are punished by not receiving a card. The objective of the game is to accrue more cards than your opponent before the deck of cards is exhausted. After a choice is made, the player spins the dial of a spinner and watches the gamble play out in real time. It’s important to note that, just like the state Lotto, players can win on rare occasions even if the choice to gamble is incorrect. Immediate feedback is implicit when the player flips the card to see the correct answer, but feedback is also available by comparing how many cards each player has collected relative to the opponents. We didn’t really feel the need to impose boundaries in this game because the behavior is fairly controlled and we didn’t want to further limit our players. Because we are developing this game as a board game, implementing a psychophysical staircase procedure was a little trickier. We decided to introduce levels into the game. Each level will have it’s own spinner and deck of cards. During early levels, the player will evaluate relatively easy prospects (i.e., within the range of reliable decision making). If the player successfully answers a certain number of questions, they are advanced to the next level where more difficult decisions have to be made. If a player doesn’t accrue a significant number of cards at the end of a level, they must go back and replay that level. Thus, student must improve on their evaluation of difficult prospects in order to win the game. Spinners for the early levels will be marked to indicate probabilities from 1 to 100%. Spins for the advanced levels will represent extreme probabilities by requiring the player to make several spins in a row within a target zone (e.g., the bet for a 0.25% prospect pays off if the player gets the needle to land between 0 and 5 twice in a row). The physical task of spinning the spinner several times for low probabilities will hopefully reinforce the notion that it’s unwise to bet on rare outcomes! Players will keep track of their own score. Players must maximize their winnings and finish with the most number of cards to win the game. If a player finishes the game without having the largest total on their scorecard, all players purge their cards and the final level will be repeated until there is a winner.

To make the game more fun, we plan to add a number of physical tasks that also must be completed before advancing to the next level (e.g., stack all the blocks that come with the game Jenga). Players will have a limited time to complete mini-games, physical feats, or puzzles to pass to the next level. The addition of these puzzles will add variety to the game and allow players to take a break from performing mental calculations. Additionally, some game cards will introduce “windfalls” or “calamities” into the mix. Windfalls might spontaneously grant the player an extra turn, extra cards, or allow them to circumvent the physical task. Calamities might require the player to loose a turn, give up cards, or move back a level. To maintain balance between competitors, players with fewer cards are more susceptible to windfalls and players with more cards are more susceptible to calamities.

Our plan is to have a working prototype of this game in the next week and play test all our games so that we can collect data the following week. Subsequent posts will describe our other games and the progress made in this game.

York College Summer Research Program

Welcome to the first in a series of posts related to rapid prototyping and game development with students. York College hosts the CUNY Summer Undergraduate Research Program (C-SURP) for high school students and undergraduates. The program is competitive, pays a stipend, and culminates in paper and poster presentations that can be applied toward college admissions or scholarship applications. I currently have nine students in my lab this summer, six of which are participating in C-SURP. All of the students are developing games of their own, but each student works on every game in the lab. The design process is collaborative, but students serve as project manger for their own game. To support this collaborative environment, I have adopted the principles described in the Valve Handbook for New Employees, which encourages collaboration without authoritative leadership. My goal is to create a sandbox where students can develop ideas without fear of criticism. I view my role as a facilitator who only intervenes to ask critical questions if a student is veering toward hazardous territory. If things are going well, I shouldn’t be talking at all.

The primary research objective of the lab is to study the efficacy of game-based learning in various disciplines. We incorporate principles from cognitive psychology, neuroscience, education, and game design to create effective learning systems. To that end, each student’s project is a study of learning in itself. Students are both the creators of a learning experience and students of the design process. My goal is to engage students in the subject matter of their choosing by involving them in design. By designing games, they must master the material related to their subject and then create a tool to teach those lessons to other students.

There is clearly no single best practice for educational game design. Even so, game designers have agreed that a certain number of elements are commonly found in successful games, and successful games are more likely to result from certain design processes. I’ve found that it’s critical to have frequent (but not necessarily daily) scheduled meetings and then allow students to break into groups as they see fit. Weekly meetings are not enough because students typically forget what they were supposed to accomplish by the time they get around to working. Cognitive psychology informs us that spaced learning is most effective, and professionals in the creative arts will attest that having a regular work regimen sets the stage for those “Eureka” moments in the creative process. Fortunately, for educators, this process is easy to implement in a class that meets two or three times a week.

During our group meetings, I ask students to review any progress they made or problems they encountered since the previous session. This practice is common to lab meetings. However, there is one critical difference. When solutions to problems are explored, it is important not to criticize any proposed solutions. Every member must be able to freely express any wacky idea that comes to mind if the appropriate solution is going to find it’s way to the surface. Reserve critical review of the game until after it has reached a stage where it can be prototyped and tested. As a facilitator, you run the risk of discouraging student participation if you start dropping logic bombs on brainstorming sessions. If you’re looking for a fun way to get this idea across to your students, play a round of “yes-and,” an improv game where students take turns building a story using the phrase “Yes, and” while avoiding the phrase “Yes, but.” It’s harder than it seems.

As a facilitator your primary job in the brainstorming sessions is to make sure the discussion is focused on the problem while remaining impartial. Having a dry-erase board in the room is critical for keeping track of all the ideas that emerge during the session and, more importantly, for making associative links between seemingly unrelated ideas. Student participation will run the gamut from incredibly-insightful-but-shy to extremely-loquacious-but-unfocused. The dry-erase board is a wonderful tool for focusing the unfocused. Ideas can be rapidly sketched as they come to light without the fear of running off the rails. Asking biographical questions is a great way of drawing out the shy students. Even the most reluctant students I’ve had in the lab will respond to personal questions about their interests, opinions, or experiences. You can use the answers to those questions to start a dialog about how to apply those attitudes toward the design of their game. At the end of each session, each student agrees on what needs to be accomplished before the next meeting. I’ve found that taking a picture of the dry-erase board is a quick and dirty means of backing up the day’s efforts without having to take additional notes.

There are a couple of challenges you will undoubtedly face when working with students in game development. First, students will come to you with an idea, together you will work to refine that idea into something tenable, and then they will come back to you the next day with a completely different topic. Many students are not academically resilient. When they encounter a problem that results in an appreciable amount of effort, it is easier for them to choose a new problem to solve. This is not a failure! This is a fine example of lateral thinking. Normally, I would encourage students to explore all the possibilities. However, most projects are under a time constraint so, you must help them not to exceed their “time budget.” Students switch from topic to topic because they are blissfully unaware of the vast amount of information they must absorb to complete the project (aren’t we all?). In my experience, when left alone, students who can’t settle on an idea will switch topics until they run out of time (sometimes this lasts an entire semester). As a facilitator, you have to intervene. I recommend introducing game elements to keep them on track. Give them a task list so they can monitor their own progress. Introduce a boundary that will hasten their work and make it more game-like. In game design, boundaries are used to limit a particular behavior, but it’s the act of limiting that behavior that makes the game fun (e.g., not using your hands in soccer). For example, to keep my students from spending too much time on an idea, I might give them a time limit for each stage of the process. As an alternative, you can also introduce friendly competition between students in the form of a race, or provide rewards for completing stages on time.

Students will also get frustrated at the difficulty of the readings and the amount of time it takes to sift through a seemingly endless pile of literature. Do you remember what it was like to read your first peer-reviewed journal as an undergraduate? Imagine being given a stack of readings like this as a high school student! As a facilitator, one of your other important jobs is to cull the reading list. However, it’s critical that you let the students discover information on their own. No matter how well we curate and scaffold the reading list, students do not like being handed a pile of papers. It makes them feel like they are being forced to do work. Let them start with searches on the Internet, even if it leads to dubious sources. When they find sources they like, augment those readings with a highly refined list of your own. Explain outright that you are giving them this list to save them time. Now that they have done some work on their own, they will appreciate the list. I typically prescribe a chapter from an undergraduate text, a good review article, and less than three peer-reviewed articles that are directly related to the subject matter.

Finally, the students might not understand or identify with the logic of game-based education. Fortunately, this is an easy lesson to teach. Start by having students talk about the games they’ve played, what games they like, what games they didn’t like, and what they may have learned from the games. You’ll probably have to step in and stop the discussion! Then, have them play some state-of-the art educational games at Explain that these games may or may not be the best examples of how to teach a lesson. They should have fun playing the games, but also encourage them to review the games with a critical eye for improvements. Follow the game playing session with a crash course in game design. Explain the critical elements like objectives, win-loose states, resources, boundaries, and reward-punishment contingencies. I try to explain these elements from the perspective of behavioral psychology because, ultimately, all behavior in games can be explained using well-tested models of behavior from psychology. Try to keep in mind that we are using games as tools to shape behavior. If we do our job, our games will produce measureable changes in behavior that can be quantified. When you have finished, introduce a few readings on game design, particularly Tracy Fullerton’s Game Design Workshop or Jessie Schell’s The Art of Game Design: A Book of Lenses.

Now that I’ve introduced my method of introducing game-based learning to students, I’ll follow up with a few posts on the games we’re working on and the day-to-day progress on those games as they develop.


Learning by design

Need help with the Commons? Visit our
help page
Send us a message
Skip to toolbar