Incentivized product reviews: Positive to a fault?

ITHACA, N.Y. – It stands to reason that the more one is compensated for performing a task, the greater the incentive to do a good job and the better one feels about doing it.

But what if the task is writing an objective review of a company or service? Does the compensation blur the lines of objectivity?

Kaitlin Woolley, assistant professor of marketing in the Samuel Curtis Johnson Graduate School of Management, Cornell SC Johnson College of Business, wondered the same thing.

“You often receive emails after a purchase, offering you a chance to win a gift card to the company in exchange for writing a review,” she said. “That seemed problematic to me – you’re buying reviews from your customers. I was interested in how incentives affect what customers write in their reviews.”

Woolley found that offering direct compensation for posting written reviews results in a greater proportion of positive versus negative emotion across a variety of product and service experiences, which she tested using two natural language-processing software systems and human judges. Whether overly glowing reviews are always good for a company, however, is another question.

Woolley is the lead author of “Incentives Increase Relative Positivity of Review Content and Enjoyment of Review Writing,” which published April 23 in the Journal of Marketing Research. Woolley’s co-author, Marissa Sharif, is an assistant professor of marketing at the Wharton School of the University of Pennsylvania.

Online reviews are critical for modern businesses; prior research has found that upward of 90% of consumers consult such reviews at least occasionally before making new purchases.

Statistically these reviews, Woolley said, create what’s known as a “J-shaped distribution,” with some bad reviews on one side, a considerable number of good reviews at the other, and a large number of silent customers in the middle. Incentivizing reviews, Woolley said, is an attempt to get that silent middle to speak up – in a positive manner.

“The idea is that if companies pay them, maybe that silent group will have more motivation to actually write a review,” Woolley said. “Incentives are generally great at increasing motivation and can increase the volume of people writing reviews. But I was curious how it might bias what people write.”

Woolley and Sharif attempted to answer this question through a series of seven controlled experiments. The first four experiments all confirmed the hypothesis that incentivizing review-writing increases customers’ enjoyment of the review writing process and the relative positivity of review content, using different products and services (e.g., video streaming service; recent fast-food experience). Additionally, the fourth showed that the effect is lessened when the incentive is less directly tied to the actual process of review-writing.

In the fifth experiment, participants (some incentivized, some not) were asked to write a review for a popular breakfast cereal, with some asked to read negative information about the company first. Overall, Woolley said, positivity increased in proportion to both the awareness and the type (monetary vs. nonmonetary) of the compensation. But compensation did not produce positive reviews for a company viewed in a negative light.

For the final two experiments, the researchers recruited Cornell students to complete a review of their current on-campus dining experience or of their spring semester. Half were told in advance that they’d be compensated for their review and others received compensation after the fact. “We found that when you paid students for their dining hall review, it increased positivity by 55%,” Woolley said.

Incentivizing reviews can have a positive effect on a company’s bottom line, of course, but the investment comes with risks, Woolley said.

“There could be welfare implications for consumers if they are exposed to information that might be overly positive, especially if it doesn’t live up to the experience that they’re expecting,” she said. “I don’t know that marketers should just say, ‘Let’s use incentives to boost review positivity,’ because there may be unforeseen negative consequences, too.”

This work was supported by a grant from the Cornell Center for Social Sciences.

For additional information, see this Cornell Chronicle story.

-30-