Facial Recognition - Developing the Guard Rails for Increased Biometrics

In this episode, host Colin Peacock and Professor Emmeline Taylor explore how facial recognition (or “biometric”) technology is being used in retail, focusing on watch lists of repeat offenders and access control applications.

They discuss strong anecdotal results reported by several international retailers, including significant shrink reductions, faster investigations, and additional benefits such as identifying wanted offenders, missing persons and dishonest employees.

Alongside this, they unpack the facial recognition guardrails. The legal, ethical and reputational risks: customer discomfort, terminology (“facial recognition” vs “biometric technology”), accuracy thresholds, and concerns about bias and wrongful identification. Both conclude that facial recognition is likely to be a powerful loss prevention tool, but argue the industry urgently needs clear codes of practice, independent accuracy standards and robust staff training before use becomes widespread.

Cleaned transcript

Introductions and context

COLIN:

Welcome again to the ECR Retail Loss podcast. I’m joined again by Professor Emmeline Taylor, and this week we’re discussing the use of facial recognition technology in retail.

We’re turning our attention today to the world of facial recognition – although people call it biometrics, face matching, person recognition, all sorts of things. At ECR we’ve been studying this for at least 10 years. We’ve had regular meetings – at least one a year, sometimes two – where retailers come to our group and share what they’re doing on facial recognition.

About six years ago we had a really great presentation from Rite Aid. They shared what they were doing in their stores, how they’d been able to identify prolific offenders. Then Reuters wrote a report, the legal team got worried, and everything was pulled overnight. So we know it’s a very sensitive area – a bit of a “Big Brother is watching you” feel.

In this group, we tend to look at the use case around a watch list, but there are other uses. We saw it the other day in a security operations centre I visited – they were using it to let people in and out. In distribution warehouses, access control is getting more use. Some people are also looking at using facial recognition on a retrospective basis.

But the main use case we focus on is the watch list: someone does something bad in the store, it’s recorded and put into a database, and when that person comes back into the store, the system flags them and there is a response from the store. We’ll get into what that response looks like.

We’re getting to a stage now where quite a few retailers have scaled it and are getting results. On our recent call we had three or four retailers sharing their work from around the world – tests in Australia and New Zealand, work from America, and work being done in the UK.

We’re now seeing some retailers who have really scaled it, aligned top management to what they’re trying to do, and put clear aims, objectives and guardrails in place. One of them reported a 25% reduction in their shrinkage number, which is a pretty impressive result and obviously very positive feedback. Another retailer reported a smaller number – maybe 10 or 15% – but showed how they were solving lots of cases.

It seemed that all the crimes in that particular part of America were suddenly being solved. They’d never had so many cases presented in such a short time – bang, bang, bang. Things happening in the car park, people coming in around some sort of gift card trial, employees and third-party vendors involved – all sorts of things they uncovered.

So it’s a very controversial area, with lots of discussion about where it goes next. The retailers we spoke to on the call are, in part, evangelists. They’ve seen this as potentially the biggest game changer for them in terms of loss prevention, and they evangelise it. It’s great for us to listen to that.

But it is controversial. Emmeline, you’ve been heavily involved in this for a long time. I think you’ve got a research project you want to do in the future. How are you thinking about this space?

Current use of facial recognition and early results

EMMELINE:

I’m fascinated by facial recognition. Even though it’s been around for several years, I don’t think there are many topics quite as “hot” as this one at the moment.

On the one hand, we’ve got anecdotal evidence that it’s hugely successful on a range of metrics – I’ll come to those in a moment. On the other hand, there’s huge nervousness and reticence to use it. People are uncomfortable about whether they’ll be able to adhere to the legislative framework in the country they’re operating in. They worry about whether they’ll put customers off. There are still a lot of questions businesses feel haven’t yet been satisfactorily answered.

We had several presentations in that meeting, from four different countries – America, the UK, Australia and New Zealand – and they all presented quite different use cases. We tend to think of facial recognition as being one thing, but actually the way it’s being used is really different. Some of that is because of what you can and can’t do in those jurisdictions, but it’s all around trialling and testing where this is going to be most effective.

What really struck me is that we primarily think of facial recognition as being about catching the bad guys – getting that ping that a bad actor is coming back in, repeatedly, perhaps putting a cap on to try to evade detection. There were some brilliant stats presented on that, as you mentioned: a 25% reduction in shrink, and the time saving. One of the presenters said what typically would have been a three-month investigation took three hours, because they were able to pull up that they had somebody travelling the country doing the same MO. They’d been under the radar, and suddenly, they had every single incident in front of them.

But there were peripheral benefits as well. One retailer had the police coming to them, searching for wanted felons – very unsavoury criminals. There were cases involving murders, sexual violence – all sorts of serious individuals being caught through the retail facial recognition system. There were missing persons being found.

Of real interest to me was the number of internal employees that were pinging on their system – employees going to a different store where they didn’t work. You can’t make it up, can you? The case they gave was really compelling for multiple reasons.

But as an academic, I always take it with a pinch of salt in terms of: give me the hard evidence, the evaluation. I think that’s where we need to move towards.

Language, terminology and customer tolerance

EMMELINE:

You mentioned terminology right at the beginning, and I think that’s really interesting because it links to customer tolerance.

It seems to me “facial recognition” as a term is perhaps the most contentious or triggering for people. In the USA use case we heard, they said they’d had no public backlash at all. Their signage said they were using “biometric technology” – no mention of cameras, no mention of faces.

Do you think there’s something in that, about how we term this?

COLIN:

Interestingly, I’ve noticed that some UK retail stores have switched to using the word “biometric” as well. Perhaps it’s a better way of describing this, given the bad publicity around the words “face recognition”.

There’s nothing more sensitive than your face, is there? It’s probably the most personal thing. So maybe if we move away from “facial recognition” towards “biometrics” – which I would have thought would be a scarier word, but maybe it isn’t – that might help. Perhaps there should be some testing with customers around what makes them feel more or less comfortable.

As you said, this particular retailer doesn’t advertise and toot their horn that they’ve got facial recognition. They use the word “biometric” and they’ve had no complaints. It’s incredible, isn’t it, the power of a word?

EMMELINE:

Yes, it really is very powerful.

Technology deployment: cameras and accuracy

COLIN:

One of the interesting things was where you put the cameras. We saw two examples.

What was fascinating about the American example is that they put the cameras very high up. The ceiling was about 100 feet, and they put the camera up there with a big zoom lens. They said that really helped them pick up people who tried to dip their heads, pull caps down, things like that. They were still able to capture their faces. I was really impressed by the technology – how it can zoom in.

Then we have the question of accuracy. My sense is that beyond, say, 95% – if you set a threshold where the system is 95% confident it’s a match – that’s possibly when you should give that alert to the store. Then the store can make up its mind as to whether it is a good match.

If it’s only a 50% match, then you’re going to get a lot of potential for error – even though, arguably, the worst that might happen is good customer service. But there’s still a risk.

There’s also the question of compliance. Stores can sometimes be tempted to go over the line. If there’s a person they think they recognise, the message from head office might be, “Give great customer service and talk to them.” But I’m sure some stores would be tempted to ask them to leave, which is where accuracy becomes a really big problem.

EMMELINE:

Absolutely. We’ve already seen some terrible cases of people being marched out of stores after a completely inaccurate match. Sadly, that seems to confirm, for those critical of facial recognition, that there might be racial and gender bias within it.

For me, accuracy is really key. I’d like to see an industry benchmark – a standard threshold of accuracy. I think all suppliers should have to publicly demonstrate, independently verified, what their accuracy rate is. Then it’s up to retailers to set their threshold for what they think an acceptable match is. We’ve heard thresholds like 80% or 90% confidence levels.

But as you know, it’s really about how people respond to it. Best practice, for me, is around having maybe two people independently check – humans checking the technology – before intervening.

I think customer service is probably the best way to go, rather than just flinging people out of the store. But what if the individual is pinging because they’re violent, and that’s the reason they are on the system? In that case there has to be a way of saying, “Do not approach this person, because last time they were approached they assaulted somebody.”

So we need a really clear understanding: how accurate is it? Let’s verify that. Should they be approached? If they shouldn’t, what do we do next? That’s where training of store associates is going to determine the success or not of facial recognition – and whether customers will put up with it or even desire it.

Governance, guidelines and independent oversight

COLIN:

I certainly believe the industry should get its act together and come up with a set of guidelines on how to use this before regulations get in the way. I’ve been saying that for 10 years, so maybe someone will do something about it now.

Emmeline, you’re actually very well placed to help with that.

I also think you’re absolutely right about the accuracy piece – having independent verification, rather than schoolboys marking their own homework. Those are really good points.

This technology does seem to be growing in relevance for retailers, especially if it does what it says on the tin. It appears to have a real impact on behaviours. At the very least, it gives the store manager and the security guard advanced warning.

One thing I remember from the call was that this particular technology was described as your most reliable loss prevention person in the whole store. They’re not on their phones, they’re not distracted. It was the “perfect employee”, wasn’t it? Apologies to all the real employees listening, but humans can be distracted, turn up late to work, be off sick, go on lunch breaks. The system is there all the time, protecting you 24/7.

Another point you made earlier is that some technologies, when the original offence is recorded, ask the user to classify whether it was a violent case or just stealing a Mars bar, or whatever. So when a face comes up as a match, there is a flag indicating whether they’ve been violent or not.

EMMELINE:

I think that’s really important in deciding how best to proceed. There can’t be a “one size fits all” response.

This has to be part of the answer. We know we’ve got a real issue with prolific repeat offenders, and just being able to piece together details on people who take advantage of there being lots of police jurisdictions is crucial. So long as they move between jurisdictions, at the moment they don’t necessarily attract the attention they should.

We’re seeing a lot of retrospective facial recognition already, and I think there’s much bigger tolerance for that at the moment. Using stills from body-worn or CCTV cameras and checking them against police databases to see whether that is a wanted offender.

It’s live facial recognition that we haven’t quite got comfortable with yet, in terms of retailers feeling OK about it.

Learning from CCTV and the need for a code of practice

EMMELINE:

I can remember the days of CCTV being introduced. What was pivotal was the code of practice issued by the Information Commissioner’s Office in the UK. Suddenly retailers said, “Fantastic, we’ve essentially got a tick list here. If we do this, we’re compliant. We know exactly how to operate them, and we’ve got nudges towards best practice and effectiveness.”

What’s missing now, for facial recognition, is that kind of code of practice for deployment. That could really spearhead its use.

COLIN:

I think you’re absolutely right on that.

EMMELINE:

I’m glad this has been recorded, Colin!

COLIN:

Absolutely right – it’s much needed to advance this.

As you say, the “biometrics” positioning might be something that helps move us on as well.

We’re going to be coming back to this again in a year’s time – we’ve already scheduled the meeting. No doubt we’ll have more cases as this all evolves, and things will definitely change. Adoption rates, I think, will grow – especially if there’s a clear code of practice.

Fingers crossed.

Thank you very much indeed for your time, and we’ll see you next time.

EMMELINE:

Thank you very much. Thanks, Colin. Bye-bye.

COLIN:

Bye.