The critical effect : evaluating the effects and use of video game reviews

Abstract

Game reviews play an important role in both the culture and business of games – the words of a reviewer can have an influential effect on the commercial success of a video game. While reviews are currently used by game developers to aid in important decisions such as project financing and employee bonuses, the effect of game reviews on players is not known. Additionally, the use of game reviews to improve evaluation techniques has received little attention. In this thesis we investigate the effect of game reviews on player experience and perceptions of quality. We show that negative reviews cause a significant effect on how players perceive their in-game experience, and that this effect is a post-play cognitive rationalization of the play experience with the previously-read review text. To address this effect we designed and deployed a new heuristic evaluation technique that specifically uses game reviews to create a fine-grained prioritized list of usability problems based on the frequency, impact, and persistence of each problem. By using our technique we are able to address the most common usability problems identified by game reviews, thus reducing the overall level of negativity found within the review text. Our approach helps to control and eliminate the snowballing effect that can be produced by players reading reviews and subsequently posting their own reviews, and thus improve the commercial success of a game.