Repository logo

Towards Reliable Online Feedback : The Impact of User Preference and Visual Cues in Rating Scales and User Ratings



Journal Title

Journal ISSN

Volume Title






Degree Level



With the rise of dependency on online shopping and service providers, consumer ratings and reviews help users decide between good and bad options. Reliable and useful ratings can ensure better consumer service, product sales, brand management. Any underlying bias or external factors affecting users emotional stability can corrupt the credibility of user feedback. Prior studies suggest that the visual representation and design elements provided with a rating scale can affect the user's responses, specially if the rating scales have visual labels that generate an emotional response in users. Since there are a number of rating scale designs used in online e-commerce sites and recommender systems, it is also important that users get a say in which rating scale they are comfortable in using. Online marketplace still does not provide a platform to consider user's own choice in this matter. This preferential choice of scales can make users more involved in the rating process and help get the best response from them. Earlier research have already proved that users have specific personalized preferences when it comes to using rating scales to give feedback online. Further emphasis on how this preference and visual cues together can elicit more reliable online feedback mechanism is required in this area. This thesis aims to investigate whether the preference of users in rating scales influences the reliability and authenticity of user's ratings. It also explores the user's reaction to certain visual cues in rating scales, and how user's preferences of rating scale are influenced by such visual elements. A within-subject study ($n$ = 187) was conducted to collect user ratings of popular products with six different rating scale designs, using two types of visual icons (stars and emojis) and colour-metaphors (using a warm-cool and a traffic-light metaphors). Statistical analysis from the survey shows that users prefer the scale with most visually informative design (traffic-light metaphor colours with emoji icons). It also shows that users tend to give their true ratings on scales they prefer most, rather than the scale design they are most familiar with. The rating score analysis also demonstrates a positive shift and better consistency in the ratings given on more visually rich scales. Based on these results, it can be concluded that user involvement is desirable in selecting the rating scale designs, and meaningful visual cues can contribute in getting more accurate (truthful) rating scores from users. The proposed approach of user preference based rating system has novelty because I elicited the user's own opinion on what their accurate or ``true" rating is; rather than only relying on analysing the data received from the rating scores. This work can offer insights for online rating scale designs to improve the rating decision quality of users and help online business platforms obtain more credible feedback from customers which can significantly improve their services and user satisfaction.



User Ratings, Frequent Pattern Mining, UX Design, Visual Metaphor



Master of Science (M.Sc.)


Computer Science


Computer Science


Part Of