Following our recent presentation at Open Source Summit North America, "Living on the Edge: Pure Open Source AI Stack with Panfrost, GStreamer and Tensorflow Lite" (video available here), many showed interest in learning more about solving real-world problems with computer vision. With that in mind, here is a new blog series, on computer vision, object detection, and building a system on the edge.
 

An active area in the field of computer vision is object detection, where the goal is to not only localize objects of interest within an image but also assign a label to each of these objects of interest. Considerable recent successes in the area of object detection stem from modern advances in deep learning, particularly leveraging deep convolutional neural networks. Much of the initial focus was on improving accuracy, leading to increasingly more complex object detection networks such as SSD, R-CNN, Mask R-CNN, and other extended variants of these networks. While such networks demonstrated state-of-the-art object detection performance, they were very challenging, if not impossible, to deploy on edge and mobile devices due to computational and memory constraints. This greatly limits the widespread adoption for a wide range of applications such as robotics, video surveillance, autonomous driving where local embedded processing is required.

Automotive

▾

AR & VR

▾

Retail

▾

Robotics

▾

As an example, suppose we want to predict when cars are about to cut into a lane. In order to acquire labeled data, we can observe when a car changes from a neighboring lane into the main lane and then rewind the video feed to label that a car is about to cut into the lane.

3. Process input through the network. In the context of object detection, the input is a collection of images, as defined in the data collectin step.
4. Compute the loss, how far is the output from being correct. In the context of object detection, the loss measures how good the network is at detecting the objects in an image.
5. Propagate gradients back into the network’s parameters. The gradient is just made up of the derivatives of all the inputs (images) concatenated in a vector.
6. Update the weights of the network, e.g. using a simple update rule: weight = weight - learningrate * gradient. Using the defined weights from the first step and calculated gradients from step 5. The learningrate parameter is a gradient descent specific value that has to be tailored to the task at hand.

Object detection is the task of image classification with localization, although an image may contain multiple objects that require localization and classification. In recent years, the performance of object detection has dramatically improved due to the emergence of object detection networks that utilize the structure of CNNs.

Some examples of object detection include:

• Drawing a bounding box and labeling each object in a street scene.
• Drawing a bounding box and labeling each object in a VR environment.
• Drawing a bounding box and labeling each object in a factory.

→ In the next part, we will go over some of the recent developments in the object detection domain and explain their details.