Loyalty campaigns: how to tell if yours did anything

Every page that ranks for this phrase answers a different question. Search loyalty campaign and you get seven results about loyalty programmes — the permanent structure, the points, the tiers. A campaign is the thing you run for two weeks in November. They are not the same object and they fail in different ways.

Loyalty campaigns are not loyalty programmes

A programme is standing infrastructure. A campaign is a time-boxed activation inside it, aimed at a segment, with a start date and an end date.

The distinction matters because the second failure is invisible to the way almost everyone measures. That is the rest of this article.

Loyalty campaign ideas, and what each one costs

Eight that are actually runnable, with the cost mechanism rather than a promise:

That last row is the only one on the list that does not spend gross margin. In this product the mechanic is a GATED benefit rather than a DISCOUNT one, and it is consistently the least-used of the two. Access costs you nothing and cannot be stacked with a promo code.

Your members outspend everyone. Here is why that proves nothing.

The standard proof that a loyalty campaign worked is a comparison between members and non-members. It is the number every vendor in this category publishes, and it cannot distinguish a programme that transformed behaviour from one that did nothing at all.

Here is a store where the campaign effect is exactly zero, by construction.

A thousand customers, ten equal groups. Group n places n orders a year — group 1 buys once, group 10 buys ten times. Average order value is identical for everyone. One rule: customers with three or more prior orders enrol. That is the only assumption doing any work, and it is the realistic one, because heavier buyers say yes at checkout more often.

Now run a campaign that changes nobody's behaviour. Nothing. Then measure it the normal way:

Orders per customer per year, in a store where the campaign did nothing

Members average 6.5 orders a year against 1.5 for non-members — 4.3 times more often. Member revenue is 1,040,000 of 1,100,000 total, so 94.5% of revenue comes from members.

Both statements are true. Both are worthless. The campaign did nothing, and the headline figures the industry quotes appeared anyway, because enrolment selected on the behaviour being measured.

This is not a quirk of retail. It is the reason advertising measurement moved to experiments: comparing exposed to unexposed groups, however carefully matched, does not recover what an experiment finds. Fifteen field experiments at Facebook covering 500 million user-experiment observations concluded that "the observational methods often fail to produce the same effects as the randomized experiments, even after conditioning on extensive demographic and behavioral variables."

How to run a loyalty campaign you can measure

Five steps. The only one people skip is the second, and skipping it costs you the answer.

1. Define eligibility as a rule you can paste. "Members with at least one purchase

in 180 days, excluding anyone who bought in the last 14." Write it down before you look at anyone.

1. Hold back a random tenth. Use hash(customer_id) mod 10 == 0 rather than a

shuffle — it is auditable, and it reproduces months later when someone asks how you picked.

1. Anchor on enrolment date, so spending from before the campaign cannot leak into

the treated group.

1. Observe for the campaign window plus one full purchase cycle. One cycle tells

you whether you moved a purchase forward. Two tells you whether you created one.

1. Freeze the analysis before launch. Metric, window, cut-off. Deciding what counts

after seeing the data is how a null result becomes a positive one.

Then the arithmetic is one line:

incremental revenue = (rev per head treated − rev per head held out) × treated heads

Subtract the reward cost at issuance, multiply by gross margin, and you have incremental contribution. That is the number. Everything else is a description.

If your list is too small to hold one out, run pre/post on matched weeks instead — but say out loud that it cannot remove seasonality, concurrent promotions or base drift, and therefore licenses a much weaker claim.

How big does the list have to be?

Large enough that the difference you care about is bigger than the noise. This is a formula, not a benchmark:

n per arm = (z(α/2) + z(β))² × [p₁(1−p₁) + p₂(1−p₂)] / (p₁ − p₂)²

At 80% power and α = 0.05, the multiplier is 7.849. Find your row:

Read that against your actual list. Two thousand customers with a 10% holdout gives you two hundred in the control group — enough to detect a ten-point swing and nothing finer. That is a legitimate position to be in. What is not legitimate is running the campaign, seeing a three-point move, and calling it a win.

What a double-points weekend actually costs

Multipliers are widely described as creating urgency without touching your economics. They are a discount with a delay.

A 2× multiplier on a 5% base earn issues 10% of basket value as points. What that costs depends on how many are redeemed:

At 60% redemption, on a 200 basket, here is what it does to contribution margin:

The thinner your margin, the larger the share of it a multiplier consumes. At 30% you are giving away a fifth of the contribution on every order in the window.

One more piece of arithmetic nobody runs: reward threshold ÷ (AOV × earn rate) is how many orders a member needs to reach a reward. A 50 reward at a 5% earn rate on a 200 basket takes five orders. If your customers place two orders a year, the reward is two and a half years away and the points are decoration. Unredeemed points are also a liability, not revenue — a promise you have made and not yet paid.

When to stop

Kill criteria, written before launch. Every signal here is a comparison you run, not a threshold anyone can hand you:

And the comparison that decides the budget: cost per incremental order from the holdout, set against your blended cost of acquiring a new customer. If keeping costs more than acquiring, the points budget belongs in ads. Most merchants have the acquisition half of that comparison and not the retention half, which is precisely the gap this method closes.

For the wider picture of what these mechanics do to repeat purchasing, see retention measured rather than assumed. For the other side of the margin question, how to increase average order value works through what a threshold does to profit.

Common questions