Video Analytics for Anomaly Detection

Video Analytics towards Anomaly Detection on the Edge for Smart Cities

Camera’s are widely used in the smart city domain to monitor and supervise environments such as road traffic, office buildings, smart homes, etc. However, most commercial (off-the-shelf) camera systems can only detect a few sets of predefined objects (e.g., person and vehicles) and behaviours. Most of these camera systems are designed for streaming the video to the cloud. In a limited number of systems, cameras themselves may do minor edge processing tasks such as detecting people and vehicles. Such primitive capabilities are not sufficient to facilitate more complex use-cases below. Further sending video streams to the cloud without processing may not be useful and require significant network bandwidth, especially when the systems need to be scaled for thousands of cameras. Further, not all video frames are worth processing in-depth.

Farms in Monmouthshire want to prevent/detect crime and safeguard lone workers. The objective is to prevent thefts of machinery and livestock and monitor farmers to ensure their safety, particularly whilst working alone at remote locations on the farm. Raglan Castle in Monmouthshire wants to detect vandalism and ensure children’s safety from monitoring any children climbing walls or performing any dangerous activities so that the local staff can intervene in a timely manner. Blaenau Gwent wants to monitor their car parks to understand how they are being used as well as how to better incentify the use of public transport (e.g., monitor how many people get off from a vehicle). Another important aspect is to detect anti-social behaviour using bus stop cameras. All the use cases require some level of anomaly detection capabilities beyond what off the shelf systems can provide.

This project combines pre-trained object detection and computer vision models to detect complex anomaly behaviours using cameras. Each pre-trained model plays a crucial role in a particular scene to extract information and actions to be incorporated together to detect different types of anomalies. Moreover, this project is not focused on processing a full video in real-time. It aims to pick up signals of potential anomalies through lightweight edge processing (e.g., a farm animal moving towards an unusual area). Once the signals are detected, systems will conduct in-depth analysis using their full capabilities by feeding the selected frames into several different pre-trained computer vision models. The objective of this project is as follows:

• Conduct a literature review on anomaly detection from camera feeds image/video and explore the types of anomalies.
• Measure trade-offs of processing videos fully and partially on edge and in the cloud.
• Explore the factors that impact anomaly detection performance in terms of environment and in-the-wild challenges, such as lighting, angle, camera resolution and other factors.
• Develop a system that can automatically adapt and decide which anomalies should be monitored on edge to pick the early signals related to a given use case (i.e., deployment) to improve overall performance and accuracy.
• Develop a technique to detect anomalies using off-the-shelf cameras by applying existing deep learning and computer vision techniques.

Team

Partners

Outcomes