PyrDown

The Pyrdowm operator is used to obtain different resolution representations of the image. There are currently two types of interpolation supported by this operator: Gaussian interpolation and bilinear interpolation. The pyramids corresponding to the two interpolation methods are called Gaussian and bilinear pyramids, respectively.

  • Gaussian pyramid downsamples the picture through processes such as Gaussian smoothing and sampling to produce an image whose length and width are half the size of the original. The current Gaussian pyramid only supports 1 level of computation.
  • The bilinear pyramid directly generates a new image of half the size of the previous image of the current layer in length and width through the processes of bilinear interpolation and downsampling.The current bilinear pyramid supports 5 levels of computation.

Operator Effect

Input ImageParameterOutput Image
imagehbVPPymParam.levels = 5
hbVPPymParam.Interpolation = HB_VP_INTER_LINEAR
image

Principle

Based on the principle of downsampling, there are two methods: Gaussian downsampling and bilinear downsampling. The implementation process of downsampling with Gaussian pyramid is to first perform Gaussian smoothing on the original image and then remove the even rows and columns to finally get the target image.

The Gaussian filter kernel used in the Gaussian smoothing stage is as follows:

kernel=1256[1464141624164624362464162416414641]\begin{aligned}kernel = \frac{1}{256}\begin{bmatrix}1 & 4 & 6 & 4 & 1 \\4 & 16 & 24 & 16 & 4 \\6 & 24 & 36 & 24 & 6 \\4 & 16 & 24 & 16 & 4 \\1 & 4 & 6 & 4 & 1 \end{bmatrix}\end{aligned}

Bilinear pyramid uses bilinear interpolation to accomplish the downsampling calculation of the image, the principle of bilinear interpolation is shown in the figure below:

image

In the interpolation process, it is assumed that the interpolated pixel point to be output is P, and the distance between the four to-be-interpolated source points P1,P2,P3,P4 adjacent to P is one. Since the interpolation method is linear, the order of interpolation in the horizontal and vertical directions does not affect the final result. The specific interpolation point formula is as follows:

P12=(1a)P1+aP2P_{12} = (1 - a) * P_1 + a * P_2

P34=(1a)P3+aP4P_{34} = (1 - a) * P_3 + a * P_4

P=(1a)(1b)P1+a(1b)P2+(1a)bP3+abP4P = (1 - a) * (1 - b) * P_1 + a * (1 - b) * P_2 + (1 - a) * b * P_3 + a * b * P_4

The coefficients of the coordinates of each point in Eq. can be considered as the weight values of the interpolated source points for the output pixel points. The pyramid interface presented in this operator uses a bilinear interpolation defaulting to equal distances from the output point to each interpolated source point. The input image data is downsampled by one-half of its aspect size. After obtaining a 2*2 image region data, the corresponding homogenization is performed to obtain a resultant pixel. The maximum number of layers for the op is up to 5 layers. The width and height of the image size of each layer is 1/2 of the previous layer. The specific calculations are as follows:

BLo(i,j)=(SRC(2i,2j)+SRC(2i+1,2j)+SRC(2i,2j+1)+SRC(2i+1,2j+1)+2)>>2BL_{o}(i,j) = \left( \text{SRC}(2*i, 2*j) + \text{SRC}(2*i + 1, 2*j) + \text{SRC}(2*i, 2*j + 1) + \text{SRC}(2*i + 1, 2*j + 1) + 2 \right) >> 2

Among them, SRC is the input image pixel, i, j are the pixel coordinates of the output bl layer, and BL is the bilinear downsampling result of the current layer output.

API Interface

int32_t hbVPPyrDown(hbUCPTaskHandle_t *taskHandle, hbVPImage *dstImgs, hbVPImage const *srcImg hbVPPymParam const *pymCfg);

For detailed interface information, please refer to hbVPPyrDown.

Usage

// Include the header #include "hobot/hb_ucp.h" #include "hobot/vp/hb_vp.h" #include "hobot/vp/hb_vp_pyr_down.h" // init Image, allocate memory for image data hbUCPSysMem src_mem; hbUCPMallocCached(&src_mem, src_stride * src_height, 0); hbVPImage src_img{HB_VP_IMAGE_FORMAT_Y, HB_VP_IMAGE_TYPE_U8C1, src_width, src_height, src_stride, src_mem.virAddr, src_mem.phyAddr, nullptr, 0, 0}; std::vector<hbUCPSysMem> dst_mems(levels); std::vector<hbVPImage> dst_imgs(levels); volatile int32_t dst_width = src_width; volatile int32_t dst_height = src_height; volatile int32_t dst_stride = src_stride; for (int32_t i{0}; i < layers; i++) { dst_width = dst_width / 2; dst_height = dst_height / 2; dst_stride = dst_stride / 2; hbUCPMallocCached(&dst_mems[i], dst_stride * dst_height, 0); dst_imgs[i] = {HB_VP_IMAGE_FORMAT_Y, HB_VP_IMAGE_TYPE_U8C1, dst_width, dst_height, dst_stride, dst_mems[i].virAddr, dst_mems[i].phyAddr, nullptr, 0, 0}; } // init pyramid config hbVPPymParam pym_cfg{}; pym_cfg.interpolation = HB_VP_INTER_GAUSSIAN; layers.levels = layers; // init task handle, schedule param and pyramid configuration hbUCPTaskHandle_t task_handle{nullptr}; hbUCPSchedParam sched_param; HB_UCP_INITIALIZE_SCHED_PARAM(&sched_param); sched_param.backend = HB_UCP_DSP_CORE_0; // create task hbVPPyrDown(&task_handle, dst_imgs.data(), &src_img, &pym_cfg); // submit task hbUCPSubmitTask(task_handle, &sched_param); // wait for task done hbUCPWaitTaskDone(task_handle, 0); // release task handle hbUCPReleaseTask(task_handle); // release memory hbUCPFree(&src_mem); for (int32_t i{0}; i < layers; i++) { hbUCPFree(&dst_mems[i]); }