S1 Ep 31
Benchmarking E-Commerce Loss - Towards a Common Language and Set of KPIS
This week, Colin Peacock is joined by Professor Michael Townsley to reflect on ECR Retail Loss’s first-ever meeting focused on benchmarking KPIs for e-commerce loss — an increasingly vital yet under-defined area of retail loss prevention.
Drawing on insights from six global retailers, they explore how different teams are measuring key metrics like acceptance rates, chargebacks, returns disputes, and fraud — often using different language to describe the same problems.
Their discussion also highlights the gap between malicious and non-malicious losses, and the importance of creating a unified framework for tracking both.
With the goal of building clear benchmarks for the industry, they highlight the need for deeper collaboration, better data, and more interventions to tackle e-commerce loss.
Podcast summary
In this episode, host Colin Peacock speaks with Professor Michael Townsley about ecommerce fraud KPIs and how retailers are currently benchmarking e-commerce loss and fraud. They discuss the key KPIs being used, such as acceptance and block rates, chargeback rates, and delivery/returns disputes, and highlight how inconsistent terminology makes comparison difficult. The conversation explores the balance between preventing payment fraud and maximising sales, the shift of fraud into returns, and the relative neglect of non-malicious loss. Their insights matter for retailers and loss prevention teams seeking to build robust, comparable e-commerce loss metrics and an end-to-end view of online risk.
Cleaned transcript
[Intro & Purpose]
HOST (COLIN PEACOCK):
Hello, my name is Colin Peacock.
In this session, I’m joined by Professor Michael Townsley.
We are looking at the challenge around benchmarking e-commerce loss KPIs and exploring what some of the key metrics might be to understand the levels of loss that e-commerce trading may be experiencing.
Good evening then, I should say. It’s soon time to put another prawn on the barbie, yeah?
GUEST (PROF. MICHAEL TOWNSLEY):
Yeah, yeah, something like that.
HOST:
The reason for jumping on this call is to talk through what we took away from our first ever meeting with retailers on how they are KPI-ing the problem of e-commerce loss.
So, what are the KPIs and what are some of the numbers that they’re looking at?
This is with a view, in the long term, to perhaps getting out there and publishing the equivalent of a shrink number for an e-commerce loss operation.
That would help us know where we should be fishing and looking for new interventions.
We had six retailers on the call. Some were based in Europe, some in America.
A lot of them had global operations, so they were familiar with operations in, for example, Mexico, Australia, Canada, Europe, the UK and beyond.
Really fascinating meeting. What were your takeaways?
Early Takeaways: Language & Terminology
GUEST:
Yes, I thought it was fascinating. Lots of information. High information ratio.
One of the first things that struck me was that by the time we got into the second or third retailer, they were talking about the same thing but often using different terms.
HOST:
Confusing, right?
GUEST:
Yeah.
We often had to say, “When you say that, what does that actually mean?”
Some people were talking about delivery or disputes about deliveries.
Another retailer was talking about errors to do with outgoings.
Then the returns were incomings, and so that was acceptance rates and block sales.
There was constant translation.
So the first thing I was thinking about was: how are we going to establish some sort of common terminology?
Common KPIs Emerging
GUEST:
When we stepped back and thought about all the different retailers and the ways they were measuring things that were important to them, they did seem to revolve around two or three things.
There was the acceptance rate or the block rate.
A few retailers talked about the percentage of transactions that were flagged as suspicious but then were ultimately approved after some sort of manual intervention.
Everyone talked about a chargeback rate.
Then there was some variation with just about everybody in deliveries and returns: disputes around deliveries or return errors.
All of those indicators were expressed as percentages.
There were some dollar values, but largely it was percentages, percentage of sales.
That makes it really easy to compare business to business.
Obviously, different categories are probably going to have different levels, but percentages are just a rate, so they enable valid comparisons.
The other thing was that just about everyone talked about their percentage this year compared to last year.
So it was either year-to-date or year-on-year.
That’s great, because with all these indicators you get magnitude and direction.
You get direction from comparisons across multiple years or time periods, and the magnitude is the score itself, the percentage.
Those were the commonalities for me. That was quite consistent across the board.
Malicious vs Non‑Malicious Loss
GUEST:
Stepping back, it did seem that a lot of the conversation was around malicious loss and fraud particularly.
There didn’t seem to be too much around non-malicious loss, or it was certainly a less prominent aspect of the conversation.
If people did mention different problems, it wasn’t the majority. It was a minority of retailers talking about non-malicious loss.
Adrian has spoken many times about known loss and unknown loss.
In the known bucket, there is malicious and non-malicious.
There are lots of examples where non-malicious loss can actually be far greater in magnitude than malicious loss.
So I thought that was quite interesting, and it showed the potential for broadening our scope of what is loss in the e-com space.
Non-malicious is there, and arguably might have better data, or at least easier-to-capture data.
It may be easier to measure those sorts of problems compared to the malicious space.
HOST:
I think your point is well made.
The cohorts and retailer representatives we had were really focused on payment fraud, perhaps returns fraud, and less on the mistakes that happen every day with couriers and others.
But again, where there’s an opportunity to make an innocent mistake, there’s an opportunity for fraud.
At ECR we’ve encouraged professionals to look at the whole.
There should be someone who oversees the whole lot, so that you have an end-to-end view.
It wasn’t obvious that everyone on the call had an end-to-end view, to your point.
Payment Fraud: Levels and Risk Appetite
HOST:
Let’s dig into some of the common areas we found where we got quite a few data points, particularly on the whole payment fraud piece.
I was shocked.
The lowest number was 0.0026, and the highest was about 0.05.
So that’s five basis points.
From five basis points to less than one basis point.
It feels like that’s an area where, certainly for the retailers in this sample, payment fraud is now difficult for bad actors.
Is that what you’re seeing?
GUEST:
I think I read a report, admittedly pre-pandemic, that card-not-present fraud globally had peaked and had started to diminish.
It’s largely around analysis at scale from some of the big players.
So I think some of the payment fraud is becoming more manageable.
In other calls, not this one, we’ve had retailers talk about how they think they’ve got the payment bit organised now.
What they’ve seen is a displacement to the returns process.
Bad actors are shifting, going where the opportunity is.
If you close off an easy opportunity, then they’ll go to the next easiest opportunity.
Acceptance vs Block Rates
HOST:
Clearly there’s a trade-off between payments fraud and all sorts of loss in terms of your acceptance rate.
We saw the acceptance rates.
The retailer that said they had the most number of blocks had 0.95% of all orders blocked.
Another retailer was as low as 0.1%.
This comes back to risk appetite.
How much do you want to do?
One retailer talked about how the value they deliver as a department was around lowering the acceptance or block rates to release extra sales.
They hope that the extra, slightly riskier sales they take do not lead to huge levels of losses on payment fraud or other types of fraud.
It’s a fascinating balance in terms of exploring risk appetite for retailers.
GUEST:
Absolutely.
If this is what you’re referring to, there did seem to be a bit of variation with some of the solution providers that different retailers are using.
This was not exactly a service, but they seemed to have different traction depending on what solution they had adopted.
There’s a really interesting implication if those statistics bear fruit.
Returns, Disputes and Data Gaps
HOST:
The area very close to you is returns fraud.
I didn’t really hear many data points or many KPIs linked to returns fraud.
There were some people talking about figures like 0.39% of goods lost in transit or disputed claims.
We heard a bit about that: 0.3%, 0.4%, 0.2%.
But I didn’t hear people having a good sense of what percentage of returns were fraud.
I guess that must be a really difficult number to get hold of.
GUEST:
Some retailers outsource the returns processing, which I think makes it even more difficult.
They’re even further removed from the data.
More surprisingly, some of the research – I think it’s Lisa Jack’s work for ECR Loss – suggests that retailers don’t have good data on their return rate, let alone return fraud.
So I think maybe we start with understanding, and having a very good understanding of, what the return rate is before we get to fraud.
Because if you don’t even know what the legitimate returns are that are coming in, I don’t know how you quantify fraud.
HOST:
Yeah, you’re right. That’s a good build.
