Why should I care about game testing?

It took me a little longer than I would have liked to finish off this series. I was encouraged to do so after one of our friends, Amy Jussel, Executive Director and Founder of Shaping Youth, said she found it useful.

Now, for the finale! So what does all this game evaluation stuff mean to you? Well, it means that we know what kids liked, what they understood, and what we can change to make Playnormous games better. After our research partner, Dr. Cynthia Phelps of The University of Texas Health Science Center Houston, finished her formative and summative evaluations, she learned several things.

The most significant change in knowledge involved foods in the “slow” category, i.e. foods that are kind of healthy and have some fat and sugar. After playing Food Fury, a scientifically significant number of kids learned the correct category for these foods:

• bacon (slow to whoa)
• peanut butter (go to slow)
• 2% milk (go to slow)
• pancakes (whoa to slow)
• yogurt (go to slow)

Changing a game after results obtained from an evaluation.

The goal of our game testing is to be able to answer three questions:

1. Is the game usable?
2. Is the game fun to play?
3. Is the game teaching what we want to teach?

After the evaluation process, if the researcher can say “yes” to all of these evaluation questions, then we stick with the current game design. If not, we make changes to the game.

1. Not Usable
   1. Were kids getting stuck and unable to play the game?
   2. For example, testing of Food Fury revealed that kids did not understand how to submit chains of foods, either by double clicking on the last tile selected or pressing the enter button.
   3. The solution: make instructions interactive and show specifically how to submit chains in a step-by-step fashion.
2. Not Fun
   1. Did the kids enjoy what they were doing? Would they play the game outside of school?
   2. For example, testing of Food Fury revealed that some of the kids found the game too slow when they played it.
   3. The solution: internal programming tweaks to make the game faster and levels that require more chains in less time.
3. Not Teaching
   1. Were the kids actually mastering the learning objectives?
   2. For example, testing of Food Fury revealed that initially kids didn’t understand that “Whoa” foods are foods that are not very healthy and should only be eaten once in a while. They were thinking “whoa” as in “whoa…cool!” and that “whoa” foods were the “awesome and fun” foods.
   3. The solution: add an introductory reel that described “whoa.”

The future.

Evaluation doesn’t stop there. Dr. Phelps hopes to continue testing Food Fury and discover even more about how it benefits kids.

• Does knowledge transfer from icon-to-icon. Can kids use the information they learned about foods in the Food Fury game and apply it to other foods? For example, kids learn in Food Fury that broccoli is a “go” food and broccoli with a topping like cheese is a “slow” food. Does this mean that kids would understand that another vegetable, like cauliflower is a “go” food and cauliflower with cheese is a “slow” food?
• How about gameplay strategy? Dr. Phelps also hopes to look at what strategies the kids were using to categorize the foods. Did kids just assume that all green foods were in the “go” category? What happens if we were to add something like a green jello with fruit icon?
• Does knowledge transfer to the real world? Can playing Food Fury help kids pick out 10 go, slow, and whoa foods at the grocery store?

Well, that concludes my series on how game evaluations work from a research perspective. Any questions?