avatarDavid Cochard

Summary

MobilenetSSD is a fast and efficient machine learning model for object detection, optimized for mobile devices and supported by the ailia SDK for easy integration into AI applications.

Abstract

The MobilenetSSD model is a Single Shot MultiBox Detector (SSD) that utilizes Mobilenet as its backbone architecture for feature extraction. This model is designed to perform quick object detection by computing bounding boxes and categories from input images. It is particularly tailored for mobile and embedded vision applications, achieving a balance between accuracy and computational efficiency. The ailia SDK facilitates the use of MobilenetSSD in AI applications, with additional support from a collection of ready-to-use ailia MODELS. Users can also train the model on custom datasets, following a specific data format and using resources like pytorch-ssd for transfer learning. The model's architecture processes images to output bounding boxes and scores, which are then refined using non-maximum suppression (nms) to filter out the most probable detections.

Opinions

  • The MobilenetSSD model is praised for its efficiency and speed in object detection tasks, making it suitable for mobile and embedded devices.
  • The integration of MobilenetSSD with the ailia SDK is highlighted as a significant advantage for developers looking to deploy AI applications across platforms with GPU acceleration.
  • The use of SSDSpec and Extra Feature Layers in the model's architecture is emphasized for improving the detection process by considering various aspect ratios and default box sizes.
  • The article suggests that training MobilenetSSD on custom data using pytorch-ssd can be a straightforward process, leveraging transfer learning from pre-trained models.
  • The need for specific system requirements, such as Mac or Linux for certain training scripts, is acknowledged, which may imply limitations for users on other operating systems like Windows.
  • The recommendation of ax Inc.'s AI services, including consulting, model creation, and application development, indicates a positive endorsement of their expertise and capabilities in the AI domain.

MobilenetSSD : A Machine Learning Model for Fast Object Detection

This is an introduction to「MobilenetSSD」, a machine learning model that can be used with ailia SDK. You can easily use this model to create AI applications using ailia SDK as well as many other ready-to-use ailia MODELS.

Overview

MobilenetSSD is an object detection model that computes the bounding box and category of an object from an input image. This Single Shot Detector (SSD) object detection model uses Mobilenet as backbone and can achieve fast object detection optimized for mobile devices.

Architecture

MobilenetSSDtakes a (3,300,300) image as input and outputs (1,3000,4) boxes and (1,3000,21) scores. Boxes contains offset values (cx,cy,w,h) from the default box. Scores contains confidence values for the presence of each of the 20 object categories, the value 0 being reserved for the background.

Source：https://arxiv.org/pdf/1512.02325.pdf

In SSD, after extracting the features using an arbitrary backbone, the bounding boxes are calculated at each resolution while reducing the resolution with Extra Feature Layers. MobilenetSSD will concatenate the output of the six levels of resolution and calculate a total of 3000 bounding boxes, and finally, filter out bounding boxes using non-maximum suppression (nms).

Source：https://arxiv.org/pdf/1512.02325.pdf

The configuration of MobilenetSSD is shown below. A default box size is defined in SSDSpec for each resolution.

image_size = 300 image_mean = np.array([127, 127, 127]) # RGB layout image_std = 128.0 iou_threshold = 0.45 center_variance = 0.1 size_variance = 0.2

specs = [ SSDSpec(19, 16, SSDBoxSizes(60, 105), [2, 3]), SSDSpec(10, 32, SSDBoxSizes(105, 150), [2, 3]), SSDSpec(5, 64, SSDBoxSizes(150, 195), [2, 3]), SSDSpec(3, 100, SSDBoxSizes(195, 240), [2, 3]), SSDSpec(2, 150, SSDBoxSizes(240, 285), [2, 3]), SSDSpec(1, 300, SSDBoxSizes(285, 330), [2, 3]) ]

SSDSpec is defined as follows.

SSDSpec = collections.namedtuple(‘SSDSpec’, [‘feature_map_size’, ‘shrinkage’, ‘box_sizes’, ‘aspect_ratios’])

In the case of SSDSpec(19, 16, SSDBoxSizes(60, 105), [2, 3]), a total of six boxes are defined with sizes 60x60, 105x105, as well as sizes 120x60, 60x120, 210x105 and 105x210 for the aspect ratio of 2.

Six levels of recognition results are concatenated, producing a total of 3000 bounding boxes.

Usage

The sample below demonstrates how to use MobilenetSSD with ailia SDK.

The following command runs the model on the web camera video stream.

$ python3 mobilenet_ssd.py -v 0
Input image (Source: https://pixabay.com/ja/photos/%E3%83%AD%E3%83%B3%E3%83%89%E3%83%B3%E5%B8%82-%E9%8A%80%E8%A1%8C-%E3%83%AD%E3%83%B3%E3%83%89%E3%83%B3-4481399/)
Inference result

Train MobilenetSSD on your own data

pytorch-ssd can be used to train MobilenetSSD on your own data.

Since pytorch-ssd uses lambda objects in DataLoader, it cannot be used on Windows, only Mac or Linux are supported.

The data format for training follows the open-image-dataset format. The following four files are required for training.

/dataset/open_images_mixed/sub-test-annotations-bbox.csv /dataset/open_images_mixed/sub-train-annotations-bbox.csv /dataset/open_images_mixed/train/images.jpg /dataset/open_images_mixed/test/images.jpg

The format of the csv is as follows.

ImageID,Source,LabelName,Confidence,XMin,XMax,YMin,YMax,IsOccluded,IsTruncated,IsGroupOf,IsDepiction,IsInside,id,ClassName

ImageId is the file name of the image (without extension), Xmin to YMax is the bounding box from 0 to 1, and ClassName is the category. Here is an example.

img_591,xclick,/m/0gxl3,1,0.40920866666666667,0.08862621809744783,0.7894286666666666,0.6620986078886312,0,0,0,0,0,/m/0gxl3,Handgun

Place the training image in the train folder, where it will be referenced as ImageId.jpg

Training is done by transfer learning, so first download the trained model.

wget -P models https://storage.googleapis.com/models-hao/mb2-ssd-lite-mp-0_686.pth

And run the training script.

python3 train_ssd.py — dataset_type open_images — datasets ./dataset — net mb2-ssd-lite — pretrained_ssd models/mb2-ssd-lite-mp-0_686.pth — scheduler cosine — lr 0.001 — t_max 100 — validation_epochs 5 — num_epochs 100 — base_net_lr 0.001 — batch_size 5

The results of the training and open-images-model-labels.txt will be output to the models folder, which will take about 38 hours to train on a MacBookPro13 CPU.

Finally, check your training results.

python3 run_ssd_example.py mb2-ssd-lite models/mb2-ssd-lite-Epoch-80-Loss-2.4882763324521524.pth models/open-images-model-labels.txt input.jpg

Since ailia SDK requires export with opset=10, add opset_version=10 to torch.onnx.export in convert_to_caffe2_models.py

torch.onnx.export(net, dummy_input, model_path, verbose=False, output_names=[‘scores’, ‘boxes’], opset_version=10)

Export to ONNX so that it can be used with ailia SDK.

python3 convert_to_caffe2_models.py mb2-ssd-lite models/mb2-ssd-lite-Epoch-80-Loss-2.4882763324521524.pth models/open-images-model-labels.txt

See below for a sample that goes from training to conversion to ONNX.

Related topics

ax Inc. has developed ailia SDK, which enables cross-platform, GPU-based rapid inference.

ax Inc. provides a wide range of services from consulting and model creation, to the development of AI-based applications and SDKs. Feel free to contact us for any inquiry.

Ailia Models
Machine Learning
Deep Learning
AI
Yolo
Recommended from ReadMedium
avatarSunny Kumar
Object tracking in YOLO using python and open cv

Yolo implementation of object tracking in python. Computer vision object tracking. open cv realtime object tracking using yolo and python3.

4 min read
avatarVipin
A Dynamic Speedometer Dashboard with OpenCV

In the world of computer vision and image processing, OpenCV (Open Source Computer Vision Library) is a powerful tool used for various…

6 min read
avatarkajal kumari
The Top 10 Python OCR Libraries for Extracting Text from Images

Introduction

12 min read
avatarFlorian BERGAMASCO
Yolo & Raspberry Pi : How to create a smart camera

A project to keep an eye on your house while you’re on holiday and receive e-mail alerts in the case of an intruder.

11 min read
avatarArpan Shaileshbhai Korat
Fine-Tuning Florence 2: Training a Vision Language Model Step-by-Step

What You’ll Learn:

9 min read
avatarDaniel García
The Next Generation of Object Detection is Here: YOLOv10 🚀

Discover How YOLOv10 Redefines Real-Time Object Detection with Unmatched Speed and Precision

5 min read