As we have seen in the rapidly expanding field of AI, the technology opens the door to a host of applications. When it comes to retail security, the “smart vision” applications of AI enable retailers to leverage their existing networks of security cameras. Many of today’s video security systems feature 4K HD video to help human operators monitor and react better – but also to provide the best possible raw video data for the AI and machine learning programs that enhance operations continuously.

Advanced analytics based on AI is becoming more prevalent, and one of the factors of its success lies in properly processing the images collected by these cameras.  The cameras are moving beyond pixels, with multi-sensor cameras increasing and image processing split between the cloud and “the edge” (where the cameras are installed).  Essentially, they are pushing the amount of video information beyond what the human eye can see, and the brain can process.

Therefore, new AI processing technologies, such as Graph Streaming Processing (GSP), are needed to meet AI-based analytic goals such as 95% or better detection rates and a false positive ratio of only 1 in 25,000.  Furthermore, GSP can enable retailers to avoid the cloud entirely and enable 100% edge processing performance at low power, which will help retailers to optimize security, inventory, placement, monitoring, and customer experience.

In a typical use case, the customer wants to develop a complete AI edge security application deployed in a retail establishment to detect people and objects in real-time. This application requires object detection, new AI models, and the pre-processing of the camera data using image signal processing.  The complete end-to-end AI application development workflow using GSP and open and code-free software platforms now enables a visual and code-free process to achieve results much more quickly than other AI development tools.  GSP used in this manner can build flexible and configurable application flows, deploy and open standard packages, and then manage and monitor these complete edge AI applications deployed in the retail store with edge MLOps.

This end-to-end application typically requires integrating various disparate tools, which can take months to develop and completely encode.  However, the objective can now be completed in days under “code-free” techniques.  A notable example concerns the edge AI application development workflow to solve a target use case using image signal processing (ISP) flow.

AI edge applications are more than just neural network models, and they often include a preprocessing image signal processing step, a post-processing tracker, or a “sense of fusion” step.  And AI developers have a big challenge integrating these disparate tools and workflows to build these more complex edge AI applications. This example shows that GSP and code-free platforms can make a substantial difference in time to market and ROI.

As part of smart vision, edge AI applications, pre-processing is done using ISP to clean up images before sending them to the neural network model for inference. A sample workflow utilizes ISP to manipulate input bayer images into a user-viewable RGB format.  The developer then starts by selecting a dataset that needs pre-processing. A dataset containing bayer or raw color format images is then transformed into RGB color space.

The ISP flow editor in the software platform would contain a few predefined ISP modules. A developer can quickly build the new desired flow by clicking the modules. The flow may be constructed by connecting a sequence of modules.  The flow may also be configured by changing a module’s properties based on the dataset’s needs or the use case. Once reached the desired level of performance, an application deployment package can be generated from the main platform interface.

As these packages are open and flexible, the developer can deploy the solutions to edge hardware from multiple OEMs or export to open formats such as OpenVX.  The hardware takes the ISP flow configured in the design stage and renders the output into human-viewable RGB color space, and additional processing has been applied to the images.  The images may now be sent to a model for inferencing with the neural network model and post-processing, such as tracker and sensor fusion.

With almost $112.1 billion¹ worth of products stolen from retailers in 2022, the introduction of legislative bills attempts to address organized crime hurting retailers and endangering public safety. While the legislation helps, theft of this magnitude will persist. So retailers must adapt quickly and continue processing massive amounts of data generated at the edge while maintaining security system uptime. That means keeping existing models in operation and implementing a comprehensive future-proof hardware platform that utilizes no-code/low-code software to deploy AI quickly without requiring expensive Data Scientists to develop complex models.

About Blaize

Blaize is a leading provider of a proprietary purpose-built, full-stack hardware architecture and low-code/no-code software platform that enables edge AI processing solutions at the network’s edge for computing in rapidly growing markets — automotive, mobility, retail, security, industrial automation, and many others. Blaize’s novel Graph Streaming Processor solution solves the technical problem that edge AI processing requires across those verticals — very low latency and high thermal and power efficiency — which previously relied on retrofitting sub-optimized AI solutions designed more for data centers and the cloud. Visit www.blaize.com to learn more.

¹ Retailers have a crime problem. It’s in the numbers