Improving the Product Look Up Menu at SCO – Meeting Recap

The latest discussion in our self-checkout working group, attended by over 120 participants, revealed ground-breaking advancements in retail technology and improvements in the Product Lookup Menu (PLU) process.

As retailers move towards sustainability and an increased offering of loose products, the importance of scales and a PLU menu becomes more important than ever. This shift not only addresses environmental concerns but also brings unique challenges in inventory management and customer experience.

One of the key insights discussed at the meeting was the significant role AI is starting to play in enhancing the accuracy of product identification. Retailers are adopting AI-driven systems at self-checkout stations and weigh stations in the aisle which are proving increasingly effective.

Some of these systems have demonstrated remarkable accuracy rates, as high as 99.96%, in recognising products like fruits and vegetables.

Such technological prowess not only reduces the likelihood of mislabelling, said to be 4% by one retailer, but also streamlines the customer’s shopping experience. However, discussion at the meeting also highlighted the ongoing issue of product misrepresentation. One retailer noted selling more carrots than they actually bought into the business!

Some customers are clearly mis-identifying products at weigh-stations and at SCOs, which underscores the necessity for more sophisticated recognition systems. The discussion also touched on the challenge of distinguishing between organic and non-organic produce, suggesting the use of stickers for easier identification. As the industry moves towards more sustainable practices including using brown paper bags, the technology must adapt to maintain its effectiveness.

The video below recaps the key findings, with the transcript below.

Colin: Well, this week we had our self-checkout group. I think we had over 120 participants registered for a discussion around the product lookup menu.

Increasingly, retailers, in order to move away from plastic, are looking to use loose products, perhaps, fruits and vegetables, bakery in other areas. And that requires increased use of the product lookup menu, which is integrated into SCO, but also, we know it’s also in the aisle as well.

And as we’ve documented, this is an area of vulnerability for retailers in terms of mistakes and errors.

But we also know it’s a huge consumption of time and effort.

And we have also learned with our online grocery team, that when online pickers look to pick loose items from customers online for their online orders, again, they have to use the product lookup menu, so it’s also a significant point of friction for online grocery pickers in the stores too.

So, we’ve got all that going on.

It really was a topic of very high interest, hence the number of people registered to attend and who participated.

And we had some excellent speakers sharing what they’re doing to try to at least detect some of the problems that might be arising and helping to make the whole process more easy and faster.

What did you take as your key notes?

Adrian: Yes, it was a very popular session. And I think the reason for that, Colin, is it is a real and present issue, I think, for all the grocers who are using self-checkout.

The vast majority of whom expect their shoppers to select loose fruit and veg.

It’s one of their attractive propositions of groceries – you can choose your own. Isn’t it?

So, it’s been a long part of the use of self-checkout.

How do you get the customer to weigh the products that they want to purchase, but critically also to identify what they are, so they can get the right price.

It’s now almost become legion, hasn’t it, in terms of arguably one of the most common ways in which people abuse SCO is to misrepresent one piece of fruit for another, or one piece of vegetable for another.

Colin: [joking] Yes, everything looks like a brown onion.

Adrian: Exactly, and I think a number of retailers have shared over time now just the extent to which this is potentially being abused.

There was one retailer who came out with a startling statistic, that they sell more carrots than they actually bring into their business. Which was highly indicative that perhaps people are choosing carrots when they may not have been carrots.

We know it’s an issue, but it’s not only an issue in terms of abuse.

As you rightly say, it can be complicated for customers to use these.

When you have potentially, some readers could have upwards of 50, 60 different types of fruit and vegetables in their stores, try to organize a product lookup screen that enables you to get to that papaya or whatever it is, that sweet potato that you purchased.

It can take time.

There can be multiple menus, can’t there, for people to work through, which isn’t easy because nobody’s ever trained us to use these technologies.

There’s that issue around just how difficult this can be for customers to do that, and there’s the potential abuse as well of, as I say, the older carrots for grapes trick.

Then the third element that you mentioned was this productivity piece, which actually came up in the session, and it wasn’t something we’d really thought about very much in terms of that audience benefiting from this.

Of course, what a lot of the interventions and solutions focused upon this are to do with product recognition, aren’t they?

They’ve got to be able to say, that’s a banana, and I can choose that for you on your behalf from the product lookup. That is the critical element.

How well can these video analytics systems identify fruit and vegetables accurately to be able to deal with this issue.

And so we heard from three retailers who shared their experiences.

The first one was slightly different in that it was mainly focused upon the weighing of fruit and vegetables away from self-checkout. So it was related, but it was different. It was in their in-aisle weigh stations, wasn’t it?

And they were using a very simple system, which was a camera above the weigh station.

And it had AI linked to it. And it had a learning algorithm in there. And they were using it  simply to look at what was in the weigh tray and say to the consumer, they’re bananas. Is that what you want to buy?

And they claimed they were getting 99.96% accuracy of the system, recognising the things that were put into that weigh scale.

Colin: So that was a remarkable statistic. What was also fantastic about what they did, I thought, was they were able to put it into silent mode and actually work out how many people were mislabelling, which was incredible.

You know, so I think the statistic was around 4% were being mislabeled, which is a lot of carrots or a lot of, not carrots. Or whatever it is.

Adrian: Yeah, that’s right. I like the surveillance mode ideas you can have with these things, because what surveillance mode is really good at this silent mode is it really helps you to get a measure of the size of the prize, doesn’t it?

Because you can see – and retailers often don’t like it, or at least don’t like having it on for too long – because they can see the loss. Visibly across the screen.

But it’s a great insight into just how much is happening. They were recognising something like 4% of the products that were put into that weigh scale were incorrectly chosen from the product lookup.

So that’s a really big win just in terms of reducing that.

But they also talked about the reduction in customer time. They reckoned it would reduce customer time by as much as 80%. Just because if you put the bananas in it, it just says bananas. And all you have to do is say yes. And that’s so much quicker than going… bananas, need to find bananas, click on bananas, and then print.

So that was really good.

And then the other two were looking more specifically at how they were using it at self-checkout. So it was integrated more into their fixed SCUs. And that was a little bit more checkered, wasn’t it?

I think one had gone through a number of iterations of this technology, trying to get it to be sufficiently accurate to offer value.

And then the other one was a pretty archaic system, really, which it wasn’t very good at recognizing things other than if it was perfectly square or had square edges.

So it’s trying to identify where people were putting things on the weigh scale that were patently not fruit and veg.

There were very few fruit and vegetables that were perfectly straight and have got square edges. And so that system was trying to say, you’ve put a bottle of wine on there. And you’re claiming it’s bananas. We don’t think that’s the case.

We saw different ways of doing this.

But there were lots of other retailers on the call who were clearly very interested in this and trialing it out as well.

Because I think everybody recognises there’s much work to be done here. And a lot of benefit can be derived from these much improved product recognition systems that are coming to market.

Colin: Yeah, as you say, I think it really was an area where there will be much more. And that’s why I think we said we would anniversary this meeting.

And the cloud did seem to make a big difference. I think we heard a couple of times if you can actually have all this intelligence. Every machine learns at the same rate.

Yeah. And perhaps that prize of, you know, recognition, and speed, might trump some of the other reasons we might want to do this around the sort of malicious types, you know, putting Lego on the scales. I don’t know how often that happens, you know.

Adrian: No, but a couple of other issues came up.

One is: what do you do if you want customers to put things into bags?

Obviously, there’s a drive to move away from C2 plastic for sustainability issues. So if you’re saying to customers, we want you to put them into brown paper bags, the best product recognition system in the world is going to struggle to look through a brown paper bag and see what’s in there.

So that’s an issue that needs to be addressed. It’s not the end of it, but it needs to be thought about.

And then of course, there’s a thorny old issue that we always talk about, which is organic versus non-organic vegetables and fruit. And how can these systems try and possibly identify the difference between an apple that is organic and an apple that’s non-organic?

And I think there was basic recognition that you need to help the system in that place by having some sort of sticker on there. So it can actually hone in on the sticker and say, I can see that that’s organic because I recognise that particular sticker.

Otherwise, I doubt whether there’s going to be any systems because the idea that there are subtly different shades of green between them is asking a lot. I think it’s much easier if you just put a sticker on these things.

Colin: Yeah, hopefully you can just grow the sticker on there as well. Yeah, it grows with the sticker on it. But this seems to be the solution for one retailer, although I guess putting a sticker on it is an extra cost.

That’s one way round it, but the brown paper bags, again, are a different thing.

Well, thanks for that, Adrian. We’re next together on December the 13th. There should be a QR code somewhere.

And on December the 13th, we’re looking at self-checkouts in non-groceries. So we’ve got a couple of retailers who are saying, yes, it’s working. Here’s some differences. And a couple saying, well, it didn’t really work for us you know and for these reasons.

So it’ll be very interesting to hear the different travails of retailers as they have experimented with self-checkouts outside of the grocery. So looking forward to that.

Thank you again and have a great weekend. Thanks Colin, bye bye.

Adrian: Bye bye.