The Utilisation of Video Analytics: A Survey of Retailers

This is ECR Retail Loss’ second report focussed upon the extent to which retailers have deployed or are thinking about using a range of video analytics across their supply chains. It is based upon a survey of global retailers and looks at nearly 50 types of analytic across five retail environments. We very much hope that the findings will offer value to the retail industry, certainly at a time when the capability of video-based technologies continues to grow at pace. While the results show that the most deployed video analytics remain primarily focussed upon helping to deliver safe and secure retail environments, it is also clear that there is growing interest in their use to improve customer service and streamline retail operations and practices. It is undoubtedly a fast moving technology and so the results from this study will, I’m sure, be of interest across a range of retail functions. We would like to thank both Professor Beck for carrying out this study and the retailers who so willingly agreed to help him with this survey. As always, ECR Retail Loss very much appreciates your commitment to contributing to the development of new thinking and ideas. Finally, can I encourage you to not only read and share this study, but also take part in the work of ECR Retail Loss – further details can be found at: www.ecrloss.com.

The use of video technologies and in particular, ‘ video analytics’ is growing considerably across the retail industry. The 2020 ECR Retail Loss report on this topic summarised the various ways in which video systems were deployed and for what reasons, including a few examples of video analytics that were being used or trialled. These spanned both security and non-security settings including burglar alarm monitoring and staff member not present scenarios. In 2022 ECR Retail Loss carried out a short survey of retailers to more fully understand the types of video analytics that retailers had already deployed, had on trial, or were planning to use in the next 12 months. Given the ever developing nature of the video analytics industry it was decided to repeat this study two years later, and presented in this report are the findings from the 2024 survey.

The move from analogue to digital video systems brought about a profound change in the way in which these technologies could be utilised. In effect they went from very ‘dumb’ systems – merely recording ‘images’ that could be viewed in real time or later, to ones that were ‘smart’, generating ‘data’ that could be searched and analysed in ways that were simply not possible with an analogue system. When it comes to defining what is meant by the term ‘video analytic’, currently there is no clearcut definition and there are areas of ambiguity within the industry. The following discussion is an attempt to begin to try and draw out what a workable definition might be. A good starting point is the following: ‘the capacity of computers to automatically interpret digital images to provide insights of value to the user ’. There are three elements to note here, first, the process is done by computers, secondly it is done automatically – no human input is required, and thirdly, it provides valuable insights. While the first two elements are relatively straightforward to understand and agree, the third element is perhaps overly broad – arguably, any video footage in the right circumstances could provide insights that are of value.

When reviewing the range of ‘analytics’ currently in the marketplace, what seems clear is that they are largely designed to either generate an alert when something happens outside of an agreed parameter, or they enable an action to proceed if an agreed parameter has been met. So, for example, a person is ‘identified’ entering a restricted area and the system sends an alert to a designated guardian. A product is ‘identified’ that has not been registered by a shopper on a selfcheckout till and an alert is generated. A fire exit door is opened when a fire alarm has not been activated and an alert is sent to a designated guardian. A member of staff is allowed to enter a restricted space through a system recognising their face. A truck is allowed to enter a distribution centre because its vehicle licence plate is recognised as being on an approved list. These would all seem to be examples of where the purpose of the video analytic has been clearly defined and there is evidence of an actionable, often real time outcome that brings tangible benefits to the user. So far so straightforward, but what about a system which, for instance, counts the number of people in a defined area , or only records video images when there is any form of movement within the field of view of the camera to preserve storage space? Are these forms of video analytics as well? I would argue that the former could be, while the latter is probably not. If the purpose of counting people within a given area is for a clearly defined and actionable purpose, then it is a video analytic. For example, if the system is counting people entering a store in order to trigger an alert to make a certain number of checkouts or checkout staff available to reduce queue times in the near future, that would be a analytic. Equally, if monitoring the number of people in a store was used to control how many more could enter (such as the traffic light system used in the COVID Pandemic period), then that also would be as a video analytic. However, if the system is simply counting the number of people purely for footfall analysis at a later date, then that is arguably not a video analytic but more a routine (video) data collection exercise, not dissimilar to data collected from say point-of-sale systems

Let’s now look at the second example – a system designed to only store video images when there is movement within the field of view of the camera to save on storage space. I would argue that this is not a video analytic but instead an example of a ‘smart’ video optimisation system. It likely meets the first two criteria of the video analytic definition described above (automatic and computerised) but there is much less evidence to say that it is generating clearly defined actionable outcomes. That’s not to say that it is not an extremely helpful tool for saving space and time, but it is arguable whether it is a video analytic. Given all of that, perhaps a ‘ work in progress’ definition of a video analytic might be:

A video system that uses the capacity of computers to automatically interpret digital images in order to generate clearly defined and actionable outcomes

As with most of the surveys carried out on the retail industry, they are largely based upon data provided by the willing – getting a representative sample can be very challenging, especially when the survey is interested in what might regarded as ‘company sensitive’ information. ECR Retail Loss has now built up a strong reputation within the global retail community for undertaking independent and useful research across a wide range of topics and making the results freely available to all. As such, it can often call upon its ‘affiliates’ to participate in new research and the current study is no exception. The purpose of the study was very straightforward – understand which of 47 pre-defined video analytics were retailers either currently deploying or planning to deploy soon. These 47 analytics were grouped in to six discrete parts of the retail supply chain:

• Store Environment 5 Analytics
• Store Entrance 4 Analytics
• • Shopfloor 13 Analytics
• Checkout 9 Analytics
• Store Backroom 6 Analytics
• Retail Distribution 10 Analytics

A full list of each of the Video Analytics used in this survey is available in Appendix 1. For each analytic respondents were asked to score them using a 5-point scale:

• Not Using
• Plan to Trial/Use in the Next 12 months
• Currently Trialling in a Few Stores
• Partially Deployed (less than 50% of all locations)
• Fully Deployed (more than 50% of all locations)

Details of the survey were sent to as many retailers as possible (mainly those that had attended previous ECR Retail Loss events) and information about it was also posted on Linked-in. Few details were collected about the companies taking part although it was possible to differentiate between Grocery and Non-grocery respondents. In total 70 retailers responded – 43 Grocery and 27 Non-grocery. The largest proportion of retailers were from Mainland Europe (34%), followed by North America (31%), UK (26%), and relatively small numbers from Australia and New Zealand (6%), South America (1%) and the Middle East (1%). As such, caution needs to be exercised when reviewing and interpreting the data in this report. In addition, while a short description was provided for each of the video analytics included, respondents may have interpreted their purpose and operation in different ways. It was hoped that statistically valid comparisons could be drawn between the 2022 and 2024 surveys but unfortunately, this has not been possible due to the relatively small sample sizes and the number of analytics.

The findings have been broken down in to three main sections. The first section offers a summary of the extent to which video analytics are being considered across the various areas of the retail supply chain. The second section provides a summary of the most deployed video analytics, first for all respondents and then by type of retailer. This is followed by those which are most commonly being trialled or planned for future use. The third section then provides a more detailed breakdown of the extent to which all of the 47 video analytics are either being used or trialled/planned, this time broken down by where they are used across the retail supply chain.

To understand the extent to which video analytics are either being deployed or trialled in different parts of the retail supply chain, it is important to take in to account the number of analytics under consideration and the number of retailers that responded to the survey. For instance, respondents were asked about 9 types of video analytic in the Store Checkout Area while in the Store Entrance area they were asked to consider only 4. To take account of this variability and enable valid comparisons to be drawn, a Video Analytic Utilisation Rate has been calculated. This is based on a scale from 0 to 100, where a score of 100 would represent ALL respondents using ALL of the analytics, while a score of 0 would represent a situation where NONE of the respondents were using ANY of the analytics. This score provides a way of both comparing the relative use of video analytics between different retail locations and differences between current deployment rates and planned/trialled use. As can be seen in Table 1, the overall deployment of all types of video analytics is currently low, a utilisation rate of 10%. The area where they are most frequently deployed is in Retail Distribution (19%), around the store environment (15%), and the Store Checkout (10%). Three areas within the store currently have the least deployment – the Shopfloor (4%), the Backroom area (7%) and the Store Entrance (7%).

Table 1 Utilisation Rate by Type of Retail Location

However, the data also shows that respondents have plans for considering future utilisation of video analytics. Overall, the planned/trialling utilisation rate is 18%, which equates to a 77% increase on current rates of deployment. Of particular note is the Store Checkout area , which has an 177% increase in its possible utilisation rate (10% deployed – 29% planned/trialled). Equally, the shopfloor is another area where respondents are thinking about expanding their use of video analytics – a 295% increase in potential utilisation. Overall, the data suggest that while current rates of use are low, retailers are beginning to both recognise and prioritise the growing use of video analytics within their organisations

The following section summarises the data on the five video analytics that were most frequently deployed across all forms of retail location, firstly for all respondents and then broken down between Grocery and Non-grocery retailers (Tables 2, 3, and 4).

As can be seen, all five of the most deployed video analytics are focused explicitly on security/safety issues. The most deployed is the use of an analytic to generate an alert when an intruder is attempting to gain access to a retail store (29%), followed by the same analytic but deployed in retail distribution (21%) and the identification of non-scanning at self-checkouts (21%).

Table 2 Top Five Currently Deployed Video Analy tics: All Respondents

In addition, 18% of respondents had deployed a video analytic to provide an alert when there was an unauthorised opening of a door at a distribution site. Finally, the fifth most deployed analytic was to identify movement in a designated area outside a retail store (17%).

It was possible to differentiate the results between two broad types of respondents – those that were Grocery retailers and those that were Non-grocery retailers. Detailed in Table 3 are the top five most deployed video analytics for Grocery retailers only.

Table 3 Top Five Currently Deployed Video Analy tics: Grocery Retailers

Here you can see the dominance of the checkout area for the deployment of video analytics in Grocery respondents – four of the top five are focussed upon this area, with non-scanning at self-checkouts top of the list (29%). As can be seen in Table 4, for Non-retailers, the picture was very different, with three of the top five being focussed upon distribution premises and the remaining two on the Store Environment. The most deployed was an intruder detection analytic for retail stores (30%) followed by the same analytic for distribution sites (28%)

Table 4 Top Five Currently Deployed Video Analy tics: Non-grocery Retailers

A very different picture emerges when the top five video analytics currently being trialled or planned are reviewed. Here you can see the dominance of the Store Checkout area as a place of considerable interest (Table 5).

Table 5 Top Five Currently Planned/Trialling Video Analy tics: All Retailers

Perhaps not surprisingly, given the inexorable rise in the use of self-checkout systems, nearly onehalf (49%) of all respondents said that they were currently either planning to trial or trialling a video analytic focussed upon the identification of non-scanning at self-checkouts. This was followed by the identification of mis-scanning events (45%) and non-payment alerts (39%) in the same space. The remaining two prospective analytics were shelf-edge alerts for suspicious behaviour (37%) and automatic number plate recognition at retail stores (31%).

Perhaps not surprisingly, given the rapid growth of self-scan checkout technology in Grocery retailing, four of the top five analytics were all in the checkout area (Table 6).

Table 6 Top Five Currently Planned/Trialling Video Analy tics: Grocery Retailers

Here you can see the extent to which the majority of all Grocers were investing in these video analytics – nearly 56% planning or trialling non-product scanning alert analytics at self-scan, with 48% looking at mis-scanning in the same area . In third place was non-payment alert analytics (49%). In joint fourth place was suspicious behaviour at the shelf edge (43%) and product identification, again at the selfcheckout (43%). While the use of self-checkout systems is still relatively infrequent outside of Grocery, it was interesting to see the top two slots being taken by analytics designed to manage problems in this space (Table 7).

Table 7 Top Five Currently Planned/Trialling Video Analy tics: Non-Grocery Retailers

A significant minority (41%) stated that they were planning/trialling the use of mis-scan analytics, with over one-third (37%) stating the same for non-scanning analytics in the self-scan area . In joint third was in-aisle suspicious behaviour monitoring (37%) and intruder alarm alerting (37%), with automatic number plate recognition at retail stores the last of the top five analytics (33%).

This third section provides a more detailed breakdown of all the video analytics covered by this survey, organised by utilisation within the retail supply chain and comparing rates of deployment against rates of trialled/planned use

Store Environment The survey asked respondents to consider their use of five video analytics in the Store Environment – this refers specifically to the space immediately surrounding a store, such as car parks/parking lots, storage spaces and goods loading areas (Table 8). The percentages refer to the actual number of respondents who stated they had either deployed or were planning/trialling the video analytic.

Table 8 Use of Video Analy tics in the Store Environment

Nearly one-third of all respondents stated that they had deployed a video analytic that provided an alert when unauthorised people entered or tried to enter one of their stores (29%). The next most deployed analytic operated in a very similar manner – providing an alert when there was movement in a designated high-risk area , such as an outside storage facility (17%). The remaining three analytics were much less frequently deployed, with only 3% of respondents stating they had deployed a system that could provide alerts when persons were moving in the opposite direction to agreed flow patterns. Across all five analytics, there was evidence of a good degree of trialling/planning to use them in the near future, with particular interest in automatic number/licence place recognition (31%) and intruder alerting (27%). However, those showing the most potential change in deployment were reverse flow/ movement alerts and the automation of number/licence plate recognition.

With regards to the store entrance, just four analytics were proposed (Table 9).

Table 9 Use of Video Analy tics at the Store Entrance

As can be seen, rates of adoption in this space are relatively low and it is unclear the extent to which the most frequently deployed analytic – Trolley Pushout Theft Alert – was correctly interpreted by respondents as a video analytic as defined at the start of this report. There are currently two main providers of a technology that addresses the issue of trolley pushout thefts, both of which primarily rely upon a non-video-based trigger system to activate a wheel lock on suspect trolleys. There are other providers that do rely upon a video-based system to identify suspect trolleys, but it would seem surprising that 16% of all respondents to this survey had currently deployed such a relatively niche technology. It is also interesting to note that only 4% of respondents stated that they had currently deployed any form of facial recognition technology, although five times as many did state that they had plans/trials underway (21%). Those showing the most potential change in deployment were analytics designed to offer alerts when checkout capacity had been reached and facial recognition for identifying those previously involved in offending.

The store shopfloor is an area that has seen a considerable amount of interest when it comes to the development of video analytics, both for security and safety purposes and to improve business productivity. A total of 13 video analytics were considered by respondents in this space (Table 10).

Table 10 Use of Video Analy tics on the Store Shopfloor

By far and away the most frequently deployed video analytic in this space was the generation of an alert when an unauthorised door was opened, such as a fire exit (16%). A common tactic adopted by professional thieves is to avoid leaving stores via the traditional entrance/exit, where security staff are often stationed, and instead force open a fire exit door to make their escape with stolen goods. The other 11 analytics were much more rarely deployed thus far – in-aisle suspicious behaviour alerts (9%), product recognition at in-aisle weight stations (7%) and fire exit blockage alerts (6%).

However, there was evidence of considerable trialling and planning of future use of a range of video analytics in this space. Of note was shelf-edge suspicious activity alerts (37%), in-aisle suspicious behaviour alerts (29%) and analytics to monitor stock levels on shelves (25%). It would certainly seem from this data that the shopfloor is likely to be an area of considerable video analytic testing and deployment soon. Those showing the most potential change in deployment were alerts to identify low/out of stock situations, in-store health and safety incidents, shelf planogram compliance.

Like the shopfloor, the retail checkout space is an area where there is a growing number of video analytics in operation, not least because of the rising use of self-checkout technologies which have presented retailers with new loss control challenges (Table 11).

Table 11 Use of Video Analy tics at the Store Checkout

Respondents were presented with nine types of video analytic, and as shown, overall, current deployment rates are relatively low although it was the area with some of the highest rates of planning/ trialling underway. In terms of current deployment, the most deployed video analytic thus far was for the identification of non-scanning at self-checkouts (21%), followed by similar technology but focussed on mis-scanning (16%), and the use of a video analytic to identify non-scanning at staffed-checkouts (15%).

Concerns about losses associated with self-checkout and the potential of video analytics to help control them was very evident when looking at the extent to which they were being planned/trialled. Nearly one-half of respondents stated they were planning/trialling video analytics to identify non scanning at self-checkouts (49%), with almost as many for identifying mis-scanning (45%). Those showing the most potential change in deployment were queue alerts, self-checkout exit controls, and non-payment alerts.

The penultimate area under consideration in this study was the store backroom area. Here, respondents were asked to consider the use of six video analytics (Table 12).

Table 12 Use of Video Analy tics in the Store Backroom

While overall levels of deployment were again relatively low across the six analytics, two were much more likely to be deployed than the others – alerting of movement in designated high risk areas (16%) and unauthorised door opening alerts (15%). Along with these two, analytics for identifying fire exit blockages were the three more commonly planned/trialled technologies in this part of the retail estate. Those showing the most potential change in deployment were reverse flow/movement alerts, good receiving accuracy alerts, and fire exit blockage alerts.

The final area of interest was the use of video analytics in retail distribution spaces, where respondents were asked to consider 10 analytics (Table 13).

Table 13 Use of Video Analy tics in Retail Distribution

The most commonly deployed analytic thus far was for the generation of alerts when intruders were trying to penetrate buildings (21%), followed by unauthorised door opening alerts (18%). Those least deployed thus far were focused upon generating alerts when a reverse flow/movement event was identified, such as staff passing in the wrong direction through doorways (3%) and the use of facial recognition (2%). Areas drawing attention for trialling/future planning were intruder alarm alerts (21%), automatic number/ licence plate recognition for access control (20%) and suspicious vehicle alerts (17%). Those showing the most potential change in deployment were facial recognition for access control, health and safety alerts, and automatic number/licence plate recognition of suspicious vehicles.

While the use of video technologies in retailing is nothing new – some of the earliest forms of ‘closed circuit television’ (CCTV) were used in a retail setting, until recently it remained a largely passive technology, often installed to offer a form of security blanket. In part this was a function of the technology itself – analogue video, in data terms at least, offered little analytical capability. The advent of ‘digital’ video has transformed the capability landscape, enabling ‘ video data’ to be collected, analysed, and acted upon in the same way other forms of data have been utilised in the past. This has led to the growth of ‘ video analytics’ – computer-based systems that can generate value from digital video images. The current survey offers a snapshot of the extent to which the retail industry is now engaging with the development of video analytics. What it found was that to date, the adoption of video analytics is currently relatively low – an overall utilisation rate of just 10%. However, the data also identified a clear trend towards future interest in and use of video analytics – it would seem to be an area on the cusp of much greater utilisation across a range of retail settings.

In particular, retail store checkouts and in-aisle seem to be areas of considerable testing and future development. Moreover, for Grocery retailers, the interest in video analytics seems particularly pronounced, driven in large part by their growing concerns around the use and control of self-scan technologies. The data suggests that video technologies in general, and video analytics in particular, could be one of the major growth areas in retail technologies in the near future

This is the second survey of video analytics carried out by ECR Retail Loss and it is hoped that it brings value in terms of providing insight into how retailers are currently using video analytics and what the likely areas of future development might be. It would seem an exciting and rapidly developing area , but, like any emerging new technology, it will be important to ensure that it offers value-added deliverables. Most video analytics operating in a retail environment are highly influenced by the complexity of the environment within which they are tasked to operate – the greater the complexity, the greater the challenge to get them to work as designed. It will be important, therefore, that retailers ensure that any given video analytic genuinely delivers value rather than simply becoming an ongoing and unwelcome distraction for those tasked with meeting the core goals of the business. To help with this process, a checklist has been reproduced from the ECR Retail Loss Report titled Assessing the Use of Video Technologies in Retailing, which provides guidance on how a prospective video analytic can be assessed within a given retail context (Appendix IV). Finally, ECR Retail Loss will carry out this survey again in the future to review how the use of video analytics is evolving across retail supply chains.