Summary

The undefined website presents an overview and analysis of the U²-Net architecture, a variant of the U-Net model designed for salient object detection in images, which utilizes nested U-structures and Residual U-blocks to capture multi-scale contextual information efficiently.

Abstract

The U²-Net architecture, as discussed on the undefined website, is an advanced deep learning model for salient object detection (SOD). It builds upon the popular U-Net framework by incorporating a two-level nested U-structure, which allows for the capture of rich contextual information at various scales. The core of U²-Net is the Residual U-block (RSU), which is designed to extract both local and global features without significantly increasing computational costs. The RSU blocks are configured with varying heights (L) to adapt to different spatial resolutions of feature maps, ensuring a balance between capturing large-scale information and preserving fine details. The U²-Net model generates multiple side output saliency probability maps, which are then fused to produce the final saliency map. The website highlights the model's ability to outperform other networks in SOD tasks, as evidenced by comparative results.

Opinions

The author of the website content expresses that U-Net has maintained its popularity since its release in 2018 and continues to be a foundational model for various segmentation tasks.
The U²-Net model is praised for its innovative design that captures more contextual information and increases network depth without a substantial increase in computational cost.
The use of dilated convolutions in the RSU blocks is presented as a strategic choice to enlarge the receptive field and capture global information while keeping memory and computation requirements in check.
The website content suggests that the U²-Net architecture is superior to other models for SOD, as demonstrated by the comparative results shown in the figures.
The author encourages readers to engage further with the topic by reading the original paper and following the AIGuys blog for more insights in the field of AI.

Paper Review: U2-Net Going Deeper with Nested U-Structure for Salient Object Detection

Segmenting different objects in a given image has been a pretty well-known task in the field of computer vision. Over the years we have seen autoencoders to crazy deep learning models like Deeplab being used for semantic segmentation. In the deep ocean of all the models, one name still remains at the top, and it's called U-Net. U-Net was released in 2018 and since then it has gained huge popularity and been used in some form or the other for several different tasks related to segmentation. In this blog, we are going to cover one variant of U-net called U²-Net or U-squared Net. U²-Net is basically a U-Net made of U-Net.

So, without further ado, let’s jump into this awesome paper. U²-Net was designed for the purpose of saliency object detection or SOD. For those who don’t know, saliency object detection is basically detecting the most important or the main object in a given image.

Key developments

The architecture of our U2 -Net is a two-level nested U-structure. The design has the following advantages:

It is able to capture more contextual information from different scales thanks to the mixture of receptive fields of different sizes in our proposed ReSidual U-blocks (RSU).
It increases the depth of the whole architecture without significantly increasing the computational cost because of the pooling operations used in these RSU blocks.

Proposed network

Residual U-Block: Both local and global contextual information are very important for salient object detection and other segmentation tasks. To keep the memory and computation down, networks like VGG16, ResNet, or DenseNet use a smaller size filter typically 1x1 or 3x3. In the below image, we can see what different networks look like. The output feature maps of shallow layers only contain local features because the receptive field of 1×1 or 3×3 filters are too small to capture global information. To achieve more global information at high-resolution feature maps from shallow layers, the most direct idea is to enlarge the receptive field but that comes at the cost of extra computation.

The above figure(d), shows an Inception-like block, which tries to extract both local and non-local features by enlarging the receptive fields using dilated convolutions. However, conducting multiple dilated convolutions on the input feature map (especially in the early stage) with original resolution requires too much computation and memory resources. Inspired by U-Net, ReSidual Ublock or RSU, captures intra-stage multi-scale features. The structure of RSU-L (C_in, M, C_out) is shown in the above figure. Here L is the number of layers in the encoder, C_in, C_out denote input and output channels, and M denotes the number of channels in the internal layers of RSU.

RSU mainly consists of three components:

An input convolution layer, which transforms the input feature map x (H×W ×C_in) to an intermediate map F1(x) with the channel of C_out. This is a plain convolutional layer for local feature extraction.
A U-Net-like symmetric encoder-decoder structure with a height of L that takes the intermediate feature map F1(x) as input and learns to extract and encode the multi-scale contextual information U(F1(x)). Larger L leads to deeper residual U-block (RSU), more pooling operations, a larger range of receptive fields, and richer local and global features. Configuring L enables the extraction of multi-scale features from input feature maps with arbitrary spatial resolutions. This process mitigates the loss of fine details caused by direct upsampling with large scales.
A residual connection that fuses local features and the multi-scale features by the summation: F1(x) +U(F1(x)).

U²-Net Architecture: In encoder stages, En_1, En_2, En_3, and En_4, residual U-blocks RSU-7, RSU-6, RSU-5, and RSU-4 are used respectively. As mentioned before, “7”, “6”, “5”, and “4” denote the heights (L) of RSU blocks. The L is usually configured according to the spatial resolution of the input feature maps. For feature maps with large height and width, greater L values are used to capture more large-scale information. The resolution of feature maps in En_5 and En_6 is relatively low, further downsampling of these feature maps leads to loss of useful context. Hence, in both En_5 and En_6 stages, RSU-4F are used, where “F” means that the RSU is a dilated version, in this stage the pooling and upsampling operations are replaced with dilated convolutions. That means all intermediate feature maps of RSU-4F have the same resolution as its input feature maps.

The decoder stages have similar structures to their symmetrical encoder stages with respect to En_6. De_5 also uses the dilated version residual U-block RSU-4F which is similar to that used in the encoder stages En_5 and En_6. Each decoder stage takes the concatenation of the upsampled feature maps from its previous stage and those from its symmetrical encoder stage as the input.

The last part is the saliency map fusion module which is used to generate saliency probability maps. U²-Net first generates six side output saliency probability maps S (6) side, S (5) side, S (4) side, S (3) side, S (2) side, S (1) side from stages En_6, De_5, De_4, De_3, De_2 and De_1 by a 3 × 3 convolution layer and a sigmoid function. Then, it upsamples the logits (convolution outputs before sigmoid functions) of the side output saliency maps to the input image size and fuses them with a concatenation operation followed by a 1×1 convolution layer and a sigmoid function to generate the final saliency probability map S_fuse.

Here are a few results of U²-Net compared with other networks.

That’s it for this blog, the goal of this blog post is to present the original paper in a simple and easy-to-read format and also give you the key takeaways.

For more detail, you can read the original paper.

Link to: Original Paper

Thanks for giving your time and if you think that this blog added something to your knowledge base, please consider following the AIGuys Blog, and if you are interested to become a writer at AI guys you can follow this link.