DeftMerch
DeftMerch answers

How should merchandising treat products with inflated review scores?

Updated October 10, 2026 ยท DeftMerch answers

Short answer: Merchandising should treat review scores as one noisy signal among several, discounting them when the review distribution looks manipulated or unrepresentative. Ranking that leans heavily on raw star averages promotes whatever has the most aggressive review operation, not the best product. The fix is to weight reviews by verified purchase status, review velocity patterns, and their agreement with behavioral signals like repeat purchase rate.

Why review scores go wrong

Review scores fail in three distinct ways. Some are gamed outright: incentivized reviews, review farms, and selective solicitation that harvests only happy customers. Some are stale: a product reformulated two years ago still coasts on reviews of the original. And some are unrepresentative: a product with eleven reviews averaging 4.9 stars is not better than one with four thousand reviews at 4.6, but a naive sort says it is.

The merchandising damage is real. When ranking trusts inflated scores, the products that win are the ones with the best review operations, which correlates with marketing spend, not quality. Shoppers learn that the top-ranked items disappoint, trust in the ranking erodes, and the whole personalization system pays for a signal it should never have trusted blindly.

Reading the distribution, not the average

The average star rating is the least informative number on the page. The distribution tells the story: a healthy product shows a bell curve with a positive skew, while a manipulated one shows a bimodal split or an implausible wall of five-star reviews with no critical middle. Review velocity matters too; a product that collected two hundred reviews in a week after months of silence was running a campaign, not earning love.

Text adds the second layer. Genuine critical reviews mention specific, varied complaints. Fake or incentivized positives repeat the same phrases and praise generic attributes. None of this requires exotic NLP; simple measures like review text uniqueness and the ratio of verified to unverified reviews separate the honest catalogs from the noisy ones well enough to rank on.

A healthier way to use reviews in ranking

Down-weight, do not discard. Reviews still carry signal, especially the critical ones, which shoppers read carefully and which predict returns. The ranking formula should use a Bayesian average that pulls thin-review products toward the category mean, require a minimum review count before the score influences rank at all, and discount reviews that fail verification checks rather than counting them fully.

Then cross-check against behavior. A product with a 4.9 average and a terrible repeat purchase rate is telling you the reviews are wrong and the behavior is right. A product with modest reviews and excellent retention is telling you the opposite. When review signals and behavioral signals disagree, behavior should win, because it is harder to fake and closer to what the business actually wants.

Fixing the catalog, not just the formula

Ranking can only discount bad signals; the catalog team has to remove them. That means review policies with teeth: no selective solicitation, verified-purchase badging that actually means something, and periodic purges of review sets that fail statistical checks. It also means treating review quality as a merchandising input, with dashboards that flag suspicious distributions the way they flag out-of-stock items.

The long game is diversifying the trust signals. Photo and video reviews, which are harder to fake at scale, deserve more weight. So do post-purchase surveys tied to actual orders. A ranking system fed by three independent honest signals beats one fed by a single gameable number, and shoppers can feel the difference even if they never see the math.

Should we hide products with suspicious reviews?

Rarely. Hiding products punishes potentially innocent items and creates merchandising gaps. Discount the suspicious signal in ranking and flag the product for review-team attention; removal is for confirmed fraud, not statistical suspicion.

Do verified purchase badges solve this?

They help but do not solve it, because incentivized reviewers are often real purchasers. Verification answers whether the reviewer bought the product, not whether the review is honest. Layer behavioral checks on top.

How do we explain ranking changes to brands we carry?

With the data, not the accusation. Show the brand that their product's review distribution triggered quality filters and what a healthy distribution looks like. Most brands would rather fix their review practices than argue with the math, once they see it.