I recently participated in panel about integrating primary literature into the classroom. Faculty from various departments exposed their motivations, tricks, successes and failures in attempting to introduce difficult reading to undergraduates at my college. In my class, students participate in two research projects of their own design. I make the assumption that they will be referring to web sites that are not peer-reviewed, and consequently I encourage them to start off with those sites and follow the trail to the primary literature. In that sense, I embrace the chaos of what I know will happen anyway. Students will consult Google, Wikipedia, and Ask.com before they consult the Journal of Theoretical Whatchamawhosit. For me, one of the critical elements of game-based learning is to acknowledge the true motivations and behaviors that are beyond your control in the classroom. Students have personal motivations for attending college, taking your class, and completing this particular assignment. Some of them will latch onto an artificial motivation because they are “good boys and girls,” and others will recognize their inner motivations, rebel agains the assignment, or align the assignment with their intrinsic motivation. Accordingly, all of the exams in my online classes are open book. I can’t prevent students from cheating during an online exam, and thus all of the questions demand extrapolating knowledge from the course to new situations. The answer won’t be in the book, but the book may be used as a reference to help solve the problem. This process could be improved upon. Our dreaded exams might actually become a mystery to be solved, a puzzle, or a game. Recently, a professor at UCLA was recognized for allowing cheating on his exam. Peter Nonacs allows students to complete the test in his Behavioral Ecology class using any means available. The exam focuses on game theory for natural selection, and he approaches the assessment process in the same way. Can you imagine students looking forward to an exam?
On Thursday, April 11, 2013 the Center for Excellence in Teaching and Learning sponsored a presentation by TGI members Sarah Kate Gillespie and Tom Zlabinger from the Department of Performing & Fine Arts. The talk was titled “Multiplayer Classroom: Team-Teaching the Art & Sound of Video Games Across Two Disciplines,” and the speakers described their experiences team-teaching a course on the art and sound of video games. The course was offered as a pilot course, and it is a rare example of interdisciplinary teaching at York. Now a 40-year-old medium, video games are incredibly complex, and providing multiple perspectives on the art form requires multiple instructors or courses. The pair described their experiences coordinating their teaching methods and attitudes, and we look forward to seeing great work come from their students in the future.
Anthony Salcito, the Vice President of Worldwide Education at Microsoft, has listed the Transformative Games Initiative as global hero in education on his blog Daily Edventures. Salcito’s blog interviews a new educator every day with the intent of describing the current landscape in education. His goal is to optimize the application of technology and maximize student achievement. The interviews include many perspectives on education and the use of technology in the classroom. Interviewees include educators of various disciplines from all over the world. Even Bill Gates contributes to the conversation!
Rasha Alsaidi and Robert Duncan from the York College Transformative Games Initiative presented a paper at the York College Research Conversations series. The talk was sponsored in part by the Office of Undergraduate Research and the TENSOR Scholar’s Program in Mathematics and Computer Sciences. Alsaidi presented data that addressed decision making in college freshmen. According to Alsaidi, freshmen tend to ignore the source of information when the content is difficult to understand. Duncan presented follow-up data from lab member Xin Lin, who demonstrated high school students could learn to improve their decision making for difficult choices if they were presented using classic game mechanics.
Contemporary practices in game development do not always serve the scientific method. As the merits of game-based learning are now recognized by higher education, we must work to reconcile the two practices to generate research programs that are both scientifically sound and adaptable to rapidly changing environments. Traditional implementation of the scientific method involves creating a falsifiable hypothesis and rigorous testing. Rejection of the hypothesis results in valuable information and a new set of hypotheses that alter the course of the research program. Planning, development, implementation, analysis, and reporting typically occur in discrete steps. In the world of software development, this approach is called the waterfall method. Software developers consider this approach outdated because the technological landscape changes too rapidly. Products that are developed using the waterfall approach can be out of date by the time they reach market. Consequently, software developers have adopted a method called Scrum that allows them to bring a product to market very quickly while making adjustments along the way.
Scrum gets its name from rugby, where players take turns advancing the ball to the goal by any means necessary. The process is built around rapid sprints, iterative development cycles that include design, implementation, prototyping, and testing. While a development team might start with a clear plan, they don’t know what problems will emerge along the way. These agile development cycles provide room for compensation in the event of failure. Scrum development teams are small (3-7) and the iteration cycles are fixed and short (1 day to 1 month). At the end of a sprint, a vertical slice is taken to determine the status of all processes involved in product development. If a feature is not completed, it gets placed into a backlog where it may be integrated back into development depending on the team’s priorities and abilities. Backlogged features can even be added to a product after the launch date in the form of software updates.
Traditional laboratory practices and educational game development conflict with each other because they operate according to radically different time scales. Scientific research programs typically have a long delay between conception and analysis, which provides little room for adjustment. Changing multiple variables during the course of an experiment is recognized as bad science because it complicates interpretation of the data. On the other hand, game designers don’t know what changes will need to occur during development to meet the demands of the players. Adopting the waterfall approach can hinder the creative process by limiting a designer’s ability to adapt to player feedback.
Traditional scientific practices can also hinder learning outcomes. The long delay between conception and analysis means that junior scientists have relatively few opportunities to test their skills. For example, there was a four-year delay between the design of my PhD thesis and the final analysis. Conversely, Scrum sprints offer several opportunities for learning. While this iterative process is possible in some scientific research programs, it is not common to most. Few students design a perfect project on their first attempt, and thus the more opportunities a student has to learn from failure the better. By encouraging multiple quick iterations, Scrum offers more opportunities to learn. Consequently, we might be doing our students a disservice by sticking to traditional scientific methods.
Fortunately, we don’t have to reinvent the wheel to reconcile these two radically different approaches. In brief, we adopt Scrum within a larger, slower moving research plan. The precedent for this approach comes from a philosophical dialog between Karl Popper, Thomas Kuhn, and Imre Lakatos. Popper is considered by many to have made a critical contribution to the scientific method by canonizing the process of falsification. According to Popper, scientific hypotheses gain little strength from positive evidence. Rather, hypotheses can only be falsified by negative evidence. For example, a person who believes all ducks are white must revise their schema the moment they witness a black duck, and this revision occurs regardless of how many white ducks they have seen in the past. Kuhn and his contemporaries rebutted against naïve falsification by asserting that no single experiment could be used to prove or deny a hypothesis because hypotheses exist within a larger framework created by the current dogma of the day. To wit, medical students are often told “Half of what we are teaching you is wrong, but we don’t know which half.” Kuhn posits that no theory fits the data perfectly, and thus all theories must be rejected unless naïve falsification is relaxed to include a modicum of probability. Lakatos was a student of Popper, and he reconciled the two arguments by insisting that major research programs are not necessarily subject to naïve falsification but individual experiments are. Thus, a research program does not need to be abandoned just because one experiment doesn’t support the hypothesis. Rather, individually falsified experiments can provide valuable clues that lead to revised research programs or major paradigm shifts in the field.
Most science operates within the framework proposed by Lakatos. Major research programs are proposed in grants and experiments are conceived to test predictions made by the program. Research results are shared publicly, and the field gravitates toward the most likely explanation for a phenomenon. A reliable body of knowledge is constructed by generating novel predictions that challenge the status quo. The good news is that this approach also suits educational game development! Each game developed in a lab is an experiment that can be used to test the hypotheses of a slower moving research program. By this standard, games that radically challenge standard educational practices are likely to bare the most fruit. While many games will fail to contribute to education, a few games will have the potential to alter the course of education forever. Thus, traditional scientific practices can be fortified by the rapid development methods of Scrum. Additionally, students can work quickly on several games and have more opportunities to learn. From the student’s perspective, this model is certainly better than waiting six years for data to trickle in!
York College recently hosted a panel to discuss the potential of using social media in the classroom, including social games. The panel discussion was sponsored by the Center for Excellence in Teaching and Learning at York College. Panel members included Nicholas Grosskopf from Health and Physical Education, Wenying Huang-Stolte from Academic Computing / Educational Technology, Michael Smith from Performing and Fine Arts, and myself. The panel description follows:
“Beyond Blackboard: Using Social Media In and Out of the Classroom”
With the recent explosion of mobile-based technologies, today’s higher education faculty often struggle with ways to keep students engaged in course content. Instead of competing with these technologies, there are ways in which they can be harnessed and used to enhanced instruction. The purpose of this CETL workshop is to introduce participants to a variety of social media tools (e.g., blogs, wikis, Twitter, Facebook, etc.) which can be implemented both in and out of the “classroom” to use students’ language and experiences as vehicles for learning. Workshop facilitators and participants will engage in dialogue about best practices when using social media in the teaching and earning process. The workshop will also introduce participants to current research on using social media, specifically the benefits of openness (visibility) to students and faculty, risks, challenges, and how instructors can address these issues effectively.
Attached are the slides from the social gaming portion of the panel discussion:
It’s been a while since the blog has been updated, but there is much to report. First, I am excited to say that I will be acting as the new Director of Undergraduate Research at York College starting in the Fall of 2013. I will be succeeding Dr. Rishi Nath from Mathematics and Computer Sciences, who did an outstanding job of launching the program and increasing the student participation in our annual research day to nearly 300. Undergraduate research is really taking off at York College, and those involved are looking forward to attracting attention to the college, pursuing new funding mechanisms to support this work, developing productive student-faculty relationships, and learning as much as we can from each other.
Second, Transformative Games has a new website in development that offers many features for students and faculty. TransformativeGames.org has been launched and content is being added rapidly to provide students and faculty at York College with the latest in games-based learning. The site also hosts a games gallery, a mirror of this blog, opportunities for student-faculty collaborations, and a discussion board for members to debate topics and provide feedback. An external site was developed primarily to host the games themselves, which is easier than hosting the site at the official York College website or the CUNY Academic Commons. While the website currently resides at a private URL, the blog will still be hosted in the Academic Commons. There are also plans to mirror the essential components of this site on the official York College website.
Finally, there are many dates coming up that I’m going to discuss in future posts:
November 6th – I will be attending the 2012 CUNY Honors Opportunities Conference at the Macaulay Honors College. This year’s theme is “Undergraduate Research: The Opportunities and the Challenges.” I hope to report many interesting developments that will affect undergraduate research and games-based learning.
November 13th – Former student Rasha Alsaidi and I will be presenting at the York College Research Conversations lecture series. Our talk, “Transformative Games: Undergraduate Research by Design,” will describe the process of creating game-based learning systems for undergraduates. The conversation will focus on productive mentor-mentee relationships and building a framework to accommodate students with widely varying research interests.
November 29th and 30th – I will be presenting two talks at the CUNY IT Conference. The first talk will be with the CUNY Games Network, and the second will be with the York College Center for Interdisciplinary Health Practice. Both talks will discuss games-based learning and are described in detail in previous posts.
Members of the York College Center for Interdisciplinary Health Practice will present at the 11th annual CUNY IT Conference November 29th and 30th. The talk will describe recent work to create simulations for students in the School of Health and Behavioral Sciences. The project utilized Second Life to create scenarios for the students to role-play. Digital technology allows students to simulate activities that would otherwise be expensive or dangerous to perform in real life. Students could play the role of doctor, patient, caregiver, occupational therapist, and more. The simulations could also be viewed passively to occasion a discussion in the classroom. Vignettes were designed to be useful to students in a variety of disciplines including but not limited to nursing, physician’s assistant, occupational therapy, social work, and psychology.
Members of the CUNY Games Network will be presenting during the annual IT Conference at CUNY on November 29th and 30th. The advisory board will be presenting “Gaming Across the Curriculum,” which will provide an introduction to game-based learning along with examples of games designed and tested by CUNY faculty. The presentation will include “What’s Your Game Plan?,” a game designed by faculty member Joe Bisz. The game is designed to help educators create games for any classroom. The hands-on workshop will provide an opportunity to explore the fundamentals of game-based learning for novices and experts alike.
A second talk will feature a review of a new learning management system at BMCC called College Quest. The project, led by Joe Bisz and Francesco Crocco in conjunction with Neuronic Games, is designed to incorporate game mechanics into a college-wide LMS. Students design avatars, earn points and badges, work through levels, collaborate in a social network, receive push-notifications for deadlines, and much more.
This is the last in series of posts about rapid prototyping for game development with high school students. I will use one of our games, Teen Angst, as a case study for what to do if things don’t go according to plan.
Teen Angst had the broadest scope of any of our games. The plan was to use game mechanics to shape decision making about three topics that most interested teenagers: relationships, substance abuse, and nutrition. Players answered a series of questions based on scenarios that were presented via PowerPoint (Figure 1). Points for four resources (Health, IQ, Friends, and Money) were gained or lost depending on the most likely consequences of the decision.
There were several design flaws in the game. First, there was no means of controlling flow. Task difficulty was not adjusted based on performance, which typically results in either boredom or frustration for the player. Second, we adopted a linear narrative that limited choices to only a few options. While the psychology literature indicates that too many choices can paralyze a person with indecision, the game world is full of examples where presenting players with more choices increases their engagement with and enjoyment of the game. In the game world, having a choice means the player is in control of the outcome, and thus they are more likely to engage with the system. As educators we should capitalize on this phenomenon and reconcile it with what we already know – ownership of the learning experience is critical to learning outcomes. Third, the game was unbalanced. In a perfectly balanced game, all the probable outcomes have an equal likelihood of occurring. A perfectly balanced game (e.g., “tic-tac-toe” or “rock-paper-scissors”) is also called a zero-sum game because opponents have an equal opportunity to win. The reward-punishment contingencies in a game, the player resources, or other factors that affect the final outcome can also be out of balance. In our game, the reward-punishment contingencies were not evenly distributed across resources. Even though the number of questions pertaining to the topics and resources were balanced, the point allocation for each topic-resource combination was not balanced. For example, players had more opportunities to gather resources for Health than the other resources.
We were aware of all of these design issues going into data collection, but we didn’t realize they would have such a strong impact on the data. Six high school students participated in the experiment, but the data for one was removed because the instructions weren’t followed properly. Subjects played the game and provided responses on answer sheets. As I mentioned in a previous post, there was an error in creating the answer sheets, which made it difficult to relate individual answers to their corresponding questions. However, we were able to compare the points earned from the first half of the game to points from the second half. The prediction was that players would earn more points in the second half because of practice effects.
Interestingly, performance exceeded chance levels during the first half of the game (Figure 3), which suggests that subjects were attentive and understood the rules. Nevertheless, contrary to our expectations, performance decreased during the second half of the game (Figure 2). Data were combined across all subjects and all categories (i.e., Relationships, Drugs, and Nutrition). For each resource (i.e., Health, Relationships, IQ and Money), performance during the second half of the game was worse than for the first half of the game, c2 (1, N = 157) = 80.067, p < 0.0001). Similarly, performance during the second half of the game was worse when data were combined across resources (Figure 3), c2 (1, N = 157) = 25.28, p < 0.0001). This decrease in performance might be attributed to fatigue. However, it is more likely that this effect is the result of an imperfect game design. The data were consistent with post-game interviews where the players reported being bored.
After six weeks of hard work, a result like this could be devastating to a student. At worst, the student might doubt the scientific method and loose interest in science. It is critical to spend as much time with the student as possible to confirm that they understand the value of impartiality, learn from failure, and persist in their quest for truth. I found it useful to recount my personal experiences with failed experiments as well as examples from famous scientists. Shifting the focus to improving the game was also helpful. However, it was particularly interesting to find that the student found some solace in knowing that her results were important because they provided evidence for the lab’s overarching hypothesis, namely, that properly employed game mechanics are useful for education. In her case, an imperfect design resulted in a baseline to which future iterations of the game will be compared. We both learned a lot from each other, and the student is sure to benefit from this experience in the future.