#Overview

Horizon-Torch-Samples is an algorithm tool based on the Pytorch and Pytorch plugin interfaces, which is an efficient and user-friendly algorithm toolkit for Horizon BPUs.

PyTorch, on which Horizon-Torch-Samples relies, is a tensor library optimized for deep learning by using GPUs and CPUs, which is now one of the most popular deep learning frameworks. The Pytorch plugin is a set of quantization algorithm tools developed based on Pytorch. Focusing on the implementation of quantization functions close to the computing platform, its quantization algorithms are deeply coupled with Horizon computing platforms, and the quantization models trained with this tool can be compiled and run normally on Horizon BPUs.

As the basic framework of algorithm package developed by Horizon Robotics, Horizon-Torch-Samples is open to all algorithm users, developers, and researchers. Its quantization training is closely related to the Horizon processors and contains a complete process: Floating point training --> QAT training --> Fixed-point transformation prediction --> Model check compilation (for Horizon BPU) --> On-board accuracy simulation verification. It also provides state-of-the-art (SOTA) deep-learning models for common image tasks including classification, detection, segmentation, etc.

#Features

• Based on Pytorch and horizon_plugin_pytorch.
• Include a complete process from Floating point training to On-board accuracy simulation verification.
• Include SOTA models for common image tasks such as classification, detection, and segmentation. All samples are compatible with Horizon BPUs.

#Sample Models

Horizon-Torch-Samples currently includes the following deep learning models:

Classification Model

• MobileNet (V1 and V2)
• ResNet (18 and 50)
• EfficientNet
• VargNet_V2
• SwinTransformer
• MixVarGENet
• EfficieNasNet(m and s)
• VargConvert
• ViT-small

Detection model

• FCOS
• PointPillar
• DETR
• FCOS3D
• CenterPoint
• Deformable-DETR-resnet50

Segmentation model

• UNet
• Deeplabv3plus_Efficientnetm0
• Deeplabv3plus_Efficientnetm1
• Deeplabv3plus_Efficientnetm2
• FastSCNN

Optical flow model

• PWCNet

Lane detection model

• GaNet

Multiple Object Track

• Motr

Binocular Depth Estimation

• StereoNet
• StereoNetPlus

Bev Multi-task Model

• Bev_mt_gkt
• Bev_mt_ipm_temporal
• Bev_mt_ipm
• Bev_mt_lss
• Detr3d_efficientnetb3_nuscenes
• Petr_efficientnetb3_nuscenes
• Bevformer_tiny_resnet50_nuscenes

Keypoints Detection Model

• Keypoint_efficientnetb0_carfusion

Lidar Multi-task Model

• LidarMultiTask

Trajectory Prediction Model

• DenseTNT

In the above model, ResNet18, ResNet50, VargConvert, EfficieNasNet, EfficientNet, MixVarGENet, VargNet_V2, GaNet, deeplabv3plus efficientnetm0, deeplabv3plus efficientnetm1, deeplabv3plus efficientnetm2, FastScnn, Bev_mt_gkt, Bev_mt_ipm, Bev_mt_lss, Detr3d_efficientnetb3_nuscenes and Keypoint_efficientnetb0_carfusion only need to do calibration quantization accuracy to achieve the goal, detailed accuracy reference model_zoo.