Rasha Alsaidi and Robert Duncan from the York College Transformative Games Initiative presented a paper at the York College Research Conversations series. The talk was sponsored in part by the Office of Undergraduate Research and  the TENSOR Scholar’s Program in Mathematics and Computer Sciences. Alsaidi presented data that addressed decision making in college freshmen. According to Alsaidi, freshmen tend to ignore the source of information when the content is difficult to understand. Duncan presented follow-up data from lab member Xin Lin, who demonstrated high school students could learn to improve their decision making for difficult choices if they were presented using classic game mechanics.

Alsaidi-Duncan_Abridged Slides

Contemporary practices in game development do not always serve the scientific method. As the merits of game-based learning are now recognized by higher education, we must work to reconcile the two practices to generate research programs that are both scientifically sound and adaptable to rapidly changing environments. Traditional implementation of the scientific method involves creating a falsifiable hypothesis and rigorous testing. Rejection of the hypothesis results in valuable information and a new set of hypotheses that alter the course of the research program. Planning, development, implementation, analysis, and reporting typically occur in discrete steps. In the world of software development, this approach is called the waterfall method. Software developers consider this approach outdated because the technological landscape changes too rapidly. Products that are developed using the waterfall approach can be out of date by the time they reach market. Consequently, software developers have adopted a method called Scrum that allows them to bring a product to market very quickly while making adjustments along the way.

Traditional laboratory practices and educational game development conflict with each other because they operate according to radically different time scales. Scientific research programs typically have a long delay between conception and analysis, which provides little room for adjustment.

Scrum gets its name from rugby, where players take turns advancing the ball to the goal by any means necessary. The process is built around rapid sprints, iterative development cycles that include design, implementation, prototyping, and testing. While a development team might start with a clear plan, they don’t know what problems will emerge along the way. These agile development cycles provide room for compensation in the event of failure. Scrum development teams are small (3-7) and the iteration cycles are fixed and short (1 day to 1 month). At the end of a sprint, a vertical slice is taken to determine the status of all processes involved in product development. If a feature is not completed, it gets placed into a backlog where it may be integrated back into development depending on the team’s priorities and abilities. Backlogged features can even be added to a product after the launch date in the form of software updates.

Traditional laboratory practices and educational game development conflict with each other because they operate according to radically different time scales. Scientific research programs typically have a long delay between conception and analysis, which provides little room for adjustment. Changing multiple variables during the course of an experiment is recognized as bad science because it complicates interpretation of the data. On the other hand, game designers don’t know what changes will need to occur during development to meet the demands of the players. Adopting the waterfall approach can hinder the creative process by limiting a designer’s ability to adapt to player feedback.

Traditional scientific practices can also hinder learning outcomes. The long delay between conception and analysis means that junior scientists have relatively few opportunities to test their skills. For example, there was a four-year delay between the design of my PhD thesis and the final analysis. Conversely, Scrum sprints offer several opportunities for learning. While this iterative process is possible in some scientific research programs, it is not common to most. Few students design a perfect project on their first attempt, and thus the more opportunities a student has to learn from failure the better. By encouraging multiple quick iterations, Scrum offers more opportunities to learn. Consequently, we might be doing our students a disservice by sticking to traditional scientific methods.

Fortunately, we don’t have to reinvent the wheel to reconcile these two radically different approaches. In brief, we adopt Scrum within a larger, slower moving research plan. The precedent for this approach comes from a philosophical dialog between Karl Popper, Thomas Kuhn, and Imre Lakatos. Popper is considered by many to have made a critical contribution to the scientific method by canonizing the process of falsification. According to Popper, scientific hypotheses gain little strength from positive evidence. Rather, hypotheses can only be falsified by negative evidence. For example, a person who believes all ducks are white must revise their schema the moment they witness a black duck, and this revision occurs regardless of how many white ducks they have seen in the past. Kuhn and his contemporaries rebutted against naïve falsification by asserting that no single experiment could be used to prove or deny a hypothesis because hypotheses exist within a larger framework created by the current dogma of the day. To wit, medical students are often told “Half of what we are teaching you is wrong, but we don’t know which half.” Kuhn posits that no theory fits the data perfectly, and thus all theories must be rejected unless naïve falsification is relaxed to include a modicum of probability. Lakatos was a student of Popper, and he reconciled the two arguments by insisting that major research programs are not necessarily subject to naïve falsification but individual experiments are. Thus, a research program does not need to be abandoned just because one experiment doesn’t support the hypothesis. Rather, individually falsified experiments can provide valuable clues that lead to revised research programs or major paradigm shifts in the field.

Most science operates within the framework proposed by Lakatos. Major research programs are proposed in grants and experiments are conceived to test predictions made by the program. Research results are shared publicly, and the field gravitates toward the most likely explanation for a phenomenon. A reliable body of knowledge is constructed by generating novel predictions that challenge the status quo. The good news is that this approach also suits educational game development! Each game developed in a lab is an experiment that can be used to test the hypotheses of a slower moving research program. By this standard, games that radically challenge standard educational practices are likely to bare the most fruit. While many games will fail to contribute to education, a few games will have the potential to alter the course of education forever. Thus, traditional scientific practices can be fortified by the rapid development methods of Scrum. Additionally, students can work quickly on several games and have more opportunities to learn. From the student’s perspective, this model is certainly better than waiting six years for data to trickle in!

York College recently hosted a panel to discuss the potential of using social media in the classroom, including social games. The panel discussion was sponsored by the Center for Excellence in Teaching and Learning at York College. Panel members included Nicholas Grosskopf from Health and Physical Education, Wenying Huang-Stolte from Academic Computing / Educational Technology, Michael Smith from Performing and Fine Arts, and myself. The panel description follows:

“Beyond Blackboard: Using Social Media In and Out of the Classroom”

With the recent explosion of mobile-based technologies, today’s higher education faculty often struggle with ways to keep students engaged in course content.  Instead of competing with these technologies, there are ways in which they can be harnessed and used to enhanced instruction. The purpose of this CETL workshop is to introduce participants to a variety of social media tools (e.g., blogs, wikis, Twitter, Facebook, etc.) which can be implemented both in and out of the “classroom” to use students’ language and experiences as vehicles for learning. Workshop facilitators and participants will engage in dialogue about best practices when using social media in the teaching and earning process.  The workshop will also introduce participants to current research on using social media, specifically the benefits of openness (visibility) to students and faculty, risks, challenges, and how instructors can address these issues effectively.

Attached are the slides from the social gaming portion of the panel discussion:

PDF version (2.9 MB)