English

Feature Extraction Sample

The image conversion sample execution scripts are located in the ucp_tutorial/vp/vp_samples/script/03_feature_extraction directory. The image feature extraction is usually categorized into point features and edge features, and this sample mainly demonstrates different edge feature extraction methods, including morphological processing to obtain edges, combining XY-direction gradient to obtain edges, and canny algorithm. Detailed implementation method please combine with the sample source code comparison practice.

In this sample, executing the sample script directly will calculate the edges of the picture by the canny algorithm by default. If you want to change the method of detecting the edges of the picture, you can control the flow of execution by appending parameters when executing the sample script, the rules for appending parameters are as follows:

sh run_feature_extraction.sh [edge operate]

Among these, edge operate is an optional parameter, edge detection algorithm. A list of all available append parameters can also be obtained with the append -help command.

The result of the picture processing is saved in the vp_samples/script/03_feature_extraction directory after the algorithm is executed, and the contents of the generated object for this sample are as follows:

vp_samples
└── script                                 // Picture data catalog
    └── 03_feature_extraction
        └── feature_edge_image.jpg         // Edge output picture

In the sample implementation, in order to establish a link between the string and the edge detection function, the parameter edge_op is defined as follows :

typedef int32_t (*example_fn)(hbVPImage &src, hbVPImage &dst);

static std::map<std::string, example_fn> edge_op = {
    {"morphology_edge", morphology_edge},
    {"canny_edge", canny_edge},
    {"sobel_edge", sobel_edge},
    {"default", canny_edge}};

The method of adapting the append parameter is consistent with the other samples, and the code is as follows:

/**===============================================================
 * process image with feature edge operate
 * ===============================================================*/
if (argc > 0) {
  ret = edge_op[argv[0]](src, dst);
  LOGE_AND_RETURN_IF(0 != ret, HB_UCP_INVALID_ARGUMENT,
                     "feature edge operate failed");
} else {
  ret = edge_op["default"](src, dst);
  LOGE_AND_RETURN_IF(0 != ret, HB_UCP_INVALID_ARGUMENT,
                     "feature edge operate failed");
}

morphology_edge

The edge features of the pictures are extracted using morphological methods, mainly using hbVPDilate and hbVPErode operators, and the processing steps are as follows:

The original picture is processed using the dilate operator to obtain an inflated image, and the inflation operation will make the light-colored regions in the original picture more prominent.
The original picture is processed using the erode operator to obtain the corroded image, and the corrosion operation will make the darker regions in the original picture more prominent.
Subtracting the corroded image from the expanded image gives you the demarcation line between the dark and light areas, i.e., the edges of the image.

The core source code is as follows:

/**===============================================================
 * process gradient operate, dilate - erode
 * ===============================================================*/
ret = dilate(src, dilate_img);
LOGE_AND_RETURN_IF(0 != ret, HB_UCP_INVALID_ARGUMENT, "dilate failed");
hbSysFlushMem(&dilate_mem, HB_SYS_MEM_CACHE_INVALIDATE);
cv::Mat dst_mat_dilate(img_height, img_width, CV_8U, dilate_img.dataVirAddr);

ret = erode(src, erode_img);
LOGE_AND_RETURN_IF(0 != ret, HB_UCP_INVALID_ARGUMENT, "erode failed");
hbSysFlushMem(&erode_mem, HB_SYS_MEM_CACHE_INVALIDATE);
cv::Mat dst_mat_erode(img_height, img_width, CV_8U, erode_img.dataVirAddr);

cv::Mat dst_mat(img_height, img_width, CV_8U, dst.dataVirAddr);
dst_mat = dst_mat_dilate - dst_mat_erode;

The effect of edge extraction is shown as belows:

sobel_edge

The edge features of the pictures are extracted using the hbVPSobel operator and the processing steps are as follows:

Use the hbVPSobel operator on the original picture to obtain an X-direction gradient image.
Use the hbVPSobel operator on the original picture to obtain an Y-direction gradient image.
The edge features in the XY direction are obtained by combining the gradient image in the XY direction.

One of the core source codes for XY direction gradient image combination is as follows:

cv::Mat mat_temp(img_height, img_width, CV_8U, dst.dataVirAddr);
uint8_t *ptr = mat_temp.data;
int16_t *ptr_x = reinterpret_cast<int16_t *>(dst_x_mem.virAddr);
int16_t *ptr_y = reinterpret_cast<int16_t *>(dst_y_mem.virAddr);
for (int j = 0; j < img_height * img_width; ++j) {
  int16_t val = ptr_x[j] * 0.5 + ptr_y[j] * 0.5;
  val = val < 0 ? 0 : (val > 255 ? 255 : val);
  ptr[j] = static_cast<uint8_t>(val);
}

The effect of edge extraction is shown as belows:

canny_edge

Use hbVPCanny operator to extract the edge features of the picture, please refer to hbVPCanny for the operator principle and interface.

PTQ Conversion Tools

The hb_compile Tool

PTQ Conversion Steps

PTQ Conversion Samples

FAQ and Common Failure Resolutions

Appendix

Tutorial

Advanced Tutorial

API Reference

QAT

Model Export

Horizon Operator

Vision Process

Vision Algorithm

Sample

VP API Introduction

Data Structure

Function Interface

Model Inference Development

Model Inference API Introduction

Data Type and Data Structure

API Interface

Model Inference Tool Introduction

High Performance Library

High Performance Algorithm

HPL API Introduction

Data Structure

Functional Interface

UCP Common API Introduction

Data Structure

Functional Interface

UCP Performance Analysis Tools

Horizon Torch Samples

FRAMEWORK

TUTORIALS

EXAMPLES

API REFERENCE

Toolchain Operator Support Constraint List

Feature Extraction Sample

The hb_compile Tool

QAT

Model Export

Horizon Operator

Vision Algorithm

Sample

VP API Introduction

Data Structure

Function Interface

Model Inference API Introduction

Data Type and Data Structure

API Interface

Model Inference Tool Introduction

High Performance Algorithm

HPL API Introduction

Data Structure

Functional Interface

Data Structure

Functional Interface

FRAMEWORK

TUTORIALS

EXAMPLES

API REFERENCE

#Feature Extraction Sample

Feature Extraction Sample