The belief online user ratings offer good indicators of product quality is largely an illusion, according to a new study conducted by researchers at the University of Colorado in Boulder.
Still, most retailers provide user ratings on their Web sites and many consumers rely on the information when making purchase decisions, according to the results of the study, published in the Journal of Consumer Research.
For the study, researchers examined user ratings for 1,272 products across 120 categories. Their analyses found a low correspondence between average user ratings of products on Amazon.com and product ratings based on objective tests found in consumer reports.
“The likelihood that an item with a higher user rating performs objectively better than an item with a lower user rating is only 57 percent,” said Bart de Langhe, author of the study and professor of marketing at the Leeds School of Business at CU. “A correspondence of 50 percent would be random, so user ratings provide very little insight about objective product performance.”
In addition, user ratings don’t predict the resale value of used products, the study found. “Products with better reliability and performance retain more of their value over time,” de Langhe said. “If average user ratings reflect objective quality, they should correlate positively with resale values. The fact that they don’t casts more doubt on the validity of user ratings.”
Philip Fernbach and Donald Lichtestein, professors of marketing at the Leeds School, co-authored the study, which also examined what information consumers rely on when judging the quality of products on Amazon.com.
Consumers rely heavily on the average user rating, which is presented front and center, according to the paper. They do this regardless of whether the average rating is based on a small or large sample of consumers.
“This is a mistake,” de Langhe said. “Oftentimes, there are just not enough ratings for a product or there is too much disagreement among reviewers. In this case, consumers should not trust the average very much, but they do nonetheless.”
In light of their findings, the authors recommend consumers remain cautious when they make inferences about product quality based on user ratings.
“Accurately evaluating product performance is not an easy task,” said de Langhe. “Different alternatives need to be evaluated side by side under the same conditions using objective measurement instruments. You can’t assume that people follow such a scientific approach before they rate products online.”
What are the implications for marketing and branding in the age of user-generated content? Because consumers comparing two products with the same objective qualities tend to give higher ratings to the one that is more expensive or from a brand with a premium reputation, marketing still counts.
“Brand managers should not fear for their jobs just yet,” de Langhe said. “User ratings do not shield consumers from the influence of good old-fashioned advertising and price-signaling.”