YOLOv7: Making YOLO Great Again

Just a few weeks ago, YOLO v7 came into the limelight by beating all the existing object detection models to date. Anyone who has worked in Object detection has heard about YOLO. It’s been here for a while now, and to date, we have seen a lot of YOLO versions. YOLO is not a single architecture but a flexible research framework written in low-level languages. The framework has three main components: the head, neck, and backbone. Different sets of components and architecture are associated with the above-mentioned three components giving rise to different YOLO versions. For a detailed breakdown of YOLO architecture, read the given blog that discusses each part of YOLO architecture in great detail. In this blog, we are mainly going to focus on what’s new in this version of YOLO.

There are many complex things presented in the YOLO v7 paper, so, without further ado, let’s dive deep into the details of this incredible architecture.

Model re-parameterization

The idea behind model re-parameterization is that it merges multiple computational modules into one at the inference stage, thus giving us better inference time. The model re-parameterization techniques are divided into two main categories: model-level ensemble and module-level ensemble. The former is to train multiple identical models with different training data and then average the weights of multiple trained models. The latter is to perform a weighted average of the weights of models at different iteration numbers. In recent years module-level re-parameterization has gained a lot of traction. Also, some of the re-parameterization techniques are not architecture agnostic; meaning they can be used only with some architectures. YOLO v7 introduces a new kind of re-parameterization that take care of previous methods' drawback.

Model Scaling

Model scaling is a way to scale up or down an already designed model and make it fit any computing device. This gives the designed architecture great flexibility. There are different types of scaling like resolution scaling (size of the input image), depth scaling (number of layers), width scaling (number of channels), and stage scaling (number of feature pyramids). Model scaling basically gives us the trade-off between speed and accuracy.

You can find more about model scaling here:

Network architecture search (NAS) is a commonly used model scaling method. NAS can automatically search for suitable scaling factors from search space without defining too complicated rules. The disadvantage of NAS is that it requires costly computation to complete the search for model scaling factors. The major problem with NAS is that scaling factors are analyzed independently. YOLO v7 introduces a new scaling method to overcome the problem of independent analyses.

Model Architecture

Extended efficient layer aggregation networks

The design of an efficient architecture primarily focuses on the number of parameters, amount of computations, and computational density of a model. The VovNet model goes a bit further and analyses the influence of the input/output channel ratio, the number of branches of the architecture, and the element-wise operation on the network inference speed. The subsequent prominent development in architecture search is called ELAN and YOLO v7 extends that and calls it E-ELAN. The conclusions drawn from the ELAN paper were that by controlling the shortest longest gradient path, a deeper network can learn and converge effectively.

Regardless of the gradient path length and the stacking number of computational blocks in large-scale ELAN, it has reached a stable state. If more computational blocks are stacked unlimitedly, this stable state may be destroyed, and the parameter utilization rate will decrease. E-ELAN uses expand, shuffle, and merge cardinality to achieve the ability to continuously enhance the learning ability of the network without destroying the original gradient path. E-ELAN only changes the architecture in the computational block, while the architecture of the transition layer is entirely unchanged. E-ELAN strategy is to use group convolution to expand the channel and cardinality of computational blocks. It applies the same group parameter and channel multiplier to all the computational blocks of a computational layer. Then, the feature map calculated by each computational block is shuffled into g groups according to the set group parameter g, and then concatenated together. At this time, the number of channels in each group of feature maps will be the same as the number of channels in the original architecture. Finally, add g groups of feature maps to perform merge cardinality. In addition to maintaining the original ELAN design architecture, E-ELAN can also guide different groups of computational blocks to learn more diverse features.

Model scaling for concatenation-based models

The primary purpose of model scaling is to adjust some attributes of the model and generate models of different scales to meet the needs of different inference speeds. In Google’s famous architecture called EfficeintNet, they scale between width, depth, and resolution. But later on, researchers tried to see the effect of vanilla convolution and group convolution on the amount of parameter and computation when performing scale.

Click here to know more about different types of Convolutions.

The strategy used by EfficientNet is not suitable for concatenation-based architecture, when scaling up or scaling down is performed on depth, the in-degree of a translation layer which is immediately after a concatenation-based computational block will decrease or increase. For instance, scaling-up depth will cause a ratio change between the input channel and output channel of a transition layer, which may lead to a decrease in the hardware usage of the model. When the depth factor of a computational block is scaled, the proposed method should also calculate the change of the output channel of that block. Then perform width factor scaling with the same amount of change on the transition layers, and the result is shown in the below figure. YOLO v7 compound scaling method can maintain the properties that the model had at the initial design and maintains the optimal structure.

Trainable bag-of-freebies

Planned re-parameterized convolution

Before we discuss this in detail, let’s look at another type of convolutional block called RepConv. RepConv is similar to what we see in Resnet but apart from one identity connection it also has one more connection with 1x1filter in between.

YOLOv7 researchers used gradient flow propagation paths to analyze how re-parameterized convolution should be combined with different networks. The below diagram shows in what way the conv blocks should be placed. Out of 8 combinations, 4 work out great (marked with a green tick in the below image).

Coarse for auxiliary and fine for lead loss

Deep supervision is a technique that is often used in training deep networks. Its central concept is to add an extra auxiliary head in the middle layers of the network, and the shallow network weights with assistant loss as the guide. Even for architectures such as ResNet and DenseNet which usually converge well, deep supervision can still significantly improve the model's performance on many tasks. The below figure shows the object detector architecture “without” and “with” deep supervision. In YOLOv7 architecture, the head responsible for the final output is called the lead head, and the head used to assist in training is called the auxiliary head.

In the past, in the training of the deep network, label assignment usually refers directly to the ground truth and generates hard labels according to the given rules. However, in recent years, if we take object detection as an example, researchers often use the quality and distribution of prediction output by the network and then consider together with the ground truth to use some calculation and optimization methods to generate a reliable soft label.

But here’s a question: “How to assign a soft label to the auxiliary head and lead head ?”. YOLOv7 use lead head prediction as guidance to generate coarse-to-fine hierarchical labels, which are used for auxiliary head and lead head learning, respectively. The two proposed deep supervision label assignment strategies are shown in the above figure.

Lead head guided label assigned: By letting the shallower auxiliary head directly learn the information that the lead head has learned, the lead head will be more able to focus on learning residual information that has not yet been learned.

Coarse-to-fine lead head guided label assigned: This mechanism allows the importance of fine label and coarse label to be dynamically adjusted during the learning process and makes the optimizable upper bound of the fine label consistently higher than the coarse label.

YOLOv7 surpasses all known object detectors in both speed and accuracy in the range from 5 FPS to 160 FPS and has the highest accuracy 56.8% AP among all known real-time object detectors with 30 FPS or higher on GPU V100. YOLOv7-E6 object detector (56 FPS V100, 55.9% AP) outperforms both transformer-based detector SWINL Cascade-Mask R-CNN (9.2 FPS A100, 53.9% AP) by 509% in speed and 2% in accuracy, and convolutionalbased detector ConvNeXt-XL Cascade-Mask R-CNN (8.6 FPS A100, 55.2% AP) by 551% in speed and 0.7% AP in accuracy

Original Paper: link

Thanks for giving your time and if you think this blog added something to your knowledge base, please consider following the AIGuys Blog. If you are interested in becoming a writer at AI Guys, you can follow this link.