How to Test a Discount Strategy

To test a discount strategy, discount one group of products and hold the discount back from a similar group over the same weeks, then compare the profit of the two. Sales during the sale are not the answer on their own, because they include orders you would have had anyway. The difference between the two groups is closer to what the discount added.

Why is revenue during a sale not the result?

Because part of it was coming anyway. Harald van Heerde, Sachin Gupta and Dick Wittink re-examined earlier studies of supermarket promotions (Journal of Marketing Research, 2003). Only about a third of the extra units sold during a promotion came from shoppers switching away from other brands. The other two thirds was growth in the category itself: shoppers buying more, or buying sooner than they otherwise would have.

For a store, an order that would have come next month at full price, placed now at 20% off, is the same order at a lower price.

What does a discount cost before it earns anything?

Work out the break-even before you test. Take a product that sells for $100 and costs you $50:

At a 50% margin, 20% off needs about 67% more units just to make the same profit. If the test does not get near that, the discount cost more than it brought in.

How do you set up a discount test?

Eric Anderson and Duncan Simester describe the basic business experiment as one action for one group, a different action or none for a control group, and a comparison of the results (Harvard Business Review, 2011). For a Shopify store:

1. Pick two groups of products that sell alike. Similar prices, similar sales in recent weeks, similar season. Two halves of a collection, split by tag, work well.
2. Discount one group and leave the other at full price. In Discounty, create a Bulk price editor campaign on the first group's tag or collection.
3. Run both over the same weeks. The season, the weather and your advertising are then the same for both.
4. Compare profit per group, using the break-even above. Read both groups' sales and costs from Shopify's own reports, so they are measured the same way. Discounty's Analytics covers only the discounted group.

How do you test which discount percentage works best?

Run two depths at the same time on two similar groups, for example 15% on one and 25% on the other, with a third group at full price. Compare each group's profit with the full-price group. The deeper discount usually sells more units; the test tells you whether it sold enough more to pay for the extra 10 points.

How do you test discount types by product?

Run the same idea on different product groups. A lower price per item for buying more suits consumables and things people buy in multiples; a spend threshold suits a store where most orders are one item. Test one type per group, with a held-back group of similar products, and change one thing at a time. Compare the types on profit from Shopify's reports, since each campaign type counts its revenue a little differently.

How long should a discount test run?

At least one full buying cycle for your products. For a consumable that is a few weeks; for furniture it can be months. Then look at the weeks after the test too: if sales of the discounted group dip below normal, shoppers probably bought ahead.

What can you read in Discounty?

For Bulk price editor campaigns, the Analytics page shows Discount campaign revenue, Orders and AOV, with a Sales by product table and a table of the orders it was used in. Revenue counts only the campaign items in each order, at the price paid. The Campaigns page shows Revenue and Orders for every campaign type.

Two things to know. The numbers cover the campaign's whole life, with no date filter. Refunds and cancellations made after an order is placed do not change them, so use Shopify's own reports for profit and returns.

To end a test, deactivate the campaign. It shows as Reverting while Discounty puts the original prices back.

What are the criteria for choosing a discount strategy?

Ask what the discount is up against. Stock that is not selling calls for clearance. A small basket calls for a threshold or a quantity break. A customer who has not ordered yet calls for a first-order offer. Our guide to discount strategy in retail sets out each pattern. Then test the one you chose against a held-back group.

How we counted

The van Heerde, Gupta and Wittink finding and the Anderson and Simester method come from their published work (Journal of Marketing Research, 2003; Harvard Business Review, 2011), not from Discounty data. The break-even table is arithmetic on an example price.

Questions

How do you test whether a discount worked?

Run it on some products and hold it back from similar ones over the same weeks, then compare profit. A before-and-after comparison changes too many things at once.

How do you test discount percentage effectiveness?

Run two depths at the same time on two similar product groups, with a third at full price, and compare each group's profit with the full-price group.

What should you measure in a discount test?

Profit per group, against the break-even for that discount. Revenue alone counts orders you would have had anyway.

How long should a discount test run?

At least one full buying cycle for your products, then a few weeks after to see whether shoppers bought ahead.

Can I A/B test discounts on Shopify?

Yes, by product rather than by shopper. A Bulk price editor sale changes a product's price for every shopper, so hold the discount back from a similar group of products and compare.