Why videogames may not be the sturdiest research tools
Psychology and videogames are tightly bound together on various levels —one of them being how scientific data gathered from videogame-related studies are manipulated and interpreted by the videogame industry to create and market psychologically appealing games. But Stephen Fairclough, blogger for the very mindful psychologicalcomputing.net, advises scientists, game-makers, and everyone in between to be careful with the delicate, volatile and highly variable conclusions such studies tend to draw:
So we have four games to be compared, which are similar in theme but also different in terms of mechanics this is one big problem for evaluation using bespoke commercial products, the lack of systematic control across different software titles. The alternative is to compare different versions of the game world that may be constructed from scratch or to use a SDK to create a systematic variation in the game world. Also the number of participants in this study is very low, especially as the group of six players varied considerably with respect to age and were further divided into casual vs. core gamers. What this does is simply increase the level of unsystematic “noise” in the data, although achieving statistical significance is perhaps not a pressing issue for this kind of commercial work, it does make it very hard to find consistency in the data at all, except by considering data from one individual.
While Fairclough emphasizes the importance of strict, ethical testing policies and the discouragement of oversimplified interpretations of test conclusions, one has to wonder what tricks the videogame industry has already picked up from these studies—and what it fundamentally lacks.
On an unrelated note: one also must wonder if test subjects are offered cake for participating in these studies. And if the cake is, in fact, a lie.
-Lana Polansky
