root@hh-j6-3022:/data/customer/Forum# mkdir segment_f root@hh-j6-3022:/data/customer/Forum# ls batch_split segment_f root@hh-j6-3022:/data/customer/Forum# cd segment_f/ root@hh-j6-3022:/data/customer/Forum/segment_f# ls model1.onnx root@hh-j6-3022:/data/customer/Forum/segment_f# hb_compile --fast-perf -m model1.onnx --march nash-m 2024-10-21 16:26:11,000 INFO log will be stored in /data/customer/Forum/segment_f/hb_compile.log 2024-10-21 16:26:11,000 INFO Start hb_compile... 2024-10-21 16:26:11,000 INFO hbdk version: 4.1.9 2024-10-21 16:26:11,001 INFO horizon_nn version: 2.0.9 2024-10-21 16:26:11,001 INFO hb_compile version: 3.3.4 2024-10-21 16:26:11,306 WARNING The input images has dynamic input_shape, the first dim of the ['N', 3, 640, 1920] will be set to 1 2024-10-21 16:26:11,308 INFO fast_perf template yaml load success 2024-10-21 16:26:11,308 INFO Updated yaml config info: {'model_parameters': {'layer_out_dump': False, 'remove_node_type': 'Quantize;Transpose;Dequantize;Cast;Reshape;Softmax', 'working_dir': '/data/customer/Forum/segment_f/model_output', 'onnx_model': '/data/customer/Forum/segment_f/model1.onnx', 'march': 'nash-m', 'output_model_file_prefix': 'model1'}, 'input_parameters': {'input_layout_train': 'NCHW', 'input_name': 'images', 'input_shape': '1x3x640x1920', 'input_space_and_range': '', 'input_type_rt': 'nv12', 'input_type_train': 'bgr', 'norm_type': 'no_preprocess', 'input_layout_rt': 'NCHW'}, 'calibration_parameters': {'calibration_type': 'skip', 'optimization': 'run_fast', 'per_channel': False, 'run_on_bpu': '', 'run_on_cpu': ''}, 'compiler_parameters': {'compile_mode': 'latency', 'core_num': 1, 'debug': True, 'jobs': 0, 'max_time_per_fc': 0, 'optimize_level': 'O2'}} 2024-10-21 16:26:11,310 INFO Start verifying yaml 2024-10-21 16:26:11,349 INFO Using onnx model file: /data/customer/Forum/segment_f/model1.onnx 2024-10-21 16:26:11,349 INFO Model has 1 inputs according to model file 2024-10-21 16:26:11,350 INFO working_dir does not exist. Creating working_dir: /data/customer/Forum/segment_f/model_output 2024-10-21 16:26:11,350 WARNING Please note that input_layout_rt is deprecated and configuring this parameter will not have any effect 2024-10-21 16:26:11,350 WARNING norm_type parameter is deprecated and will be determined by the configuration of mean/scale/std parameter 2024-10-21 16:26:11,350 INFO Parameter calibration_type is skip. cal_data_dir check skipped 2024-10-21 16:26:11,350 WARNING Input node images's input_source not set, it will be set to None by default 2024-10-21 16:26:11,350 INFO Parameter compile_mode is set to latency, balance_factor will set to 100. 2024-10-21 16:26:11,350 INFO End verifying yaml 2024-10-21 16:26:11,350 INFO Start to Horizon NN Model Convert. 2024-10-21 16:26:11,460 INFO Loading horizon_nn debug methods:set() 2024-10-21 16:26:11,460 INFO The activation calibration parameters: calibration_type: fixed The output nodes of model are set to datatype: int8 The input of all Softmax nodes are set to : int8 2024-10-21 16:26:11,460 INFO The specified model compilation architecture: nash. 2024-10-21 16:26:11,460 INFO The specified model compilation optimization parameters: ['run_fast']. Using optimization configuration: run_fast 2024-10-21 16:26:11,460 INFO Start to prepare the onnx model. 2024-10-21 16:26:11,486 INFO Input ONNX Model Information: ONNX IR version: 6 Opset version: ['ai.onnx v11', 'horizon v1'] Producer: pytorch v1.13.0 Domain: None Version: None Graph input: images: shape=['N', 3, 640, 1920], dtype=FLOAT32 Graph output: 2035: shape=['Concat2035_dim_0', 'Concat2035_dim_1', 'Concat2035_dim_2', 'Concat2035_dim_3'], dtype=FLOAT32 2024-10-21 16:26:11,487 INFO Replace the input images's shape from ['N', 3, 640, 1920] to [1, 3, 640, 1920]. 2024-10-21 16:26:11,667 INFO The original model's opset version is 11, try converting to opset 19. 2024-10-21 16:26:11,918 INFO End to prepare the onnx model. 2024-10-21 16:26:11,980 INFO Saving model to: /data/customer/Forum/segment_f/model_output/model1_original_float_model.onnx. 2024-10-21 16:26:11,980 INFO Start to optimize the onnx model. 2024-10-21 16:26:16,536 INFO End to optimize the onnx model. 2024-10-21 16:26:16,577 INFO Saving model to: /data/customer/Forum/segment_f/model_output/model1_optimized_float_model.onnx. 2024-10-21 16:26:16,577 INFO Start to calibrate the model. 2024-10-21 16:26:16,946 INFO There are 1 samples in the calibration data set. 2024-10-21 16:26:16,946 INFO Run calibration model with fixed thresholds method. 2024-10-21 16:26:17,454 INFO End to calibrate the model. 2024-10-21 16:26:17,556 INFO Saving model to: /data/customer/Forum/segment_f/model_output/model1_calibrated_model.onnx. 2024-10-21 16:26:17,556 INFO Start to precompile the model. 2024-10-21 16:26:18,504 INFO End to precompile the model. 2024-10-21 16:26:18,608 INFO Saving model to: /data/customer/Forum/segment_f/model_output/model1_ptq_model.onnx. 2024-10-21 16:26:18,688 INFO End to Horizon NN Model Convert. ===============> HBIRHandle: 134172942266144 : 134172942266144 2024-10-21 16:26:19,025 WARNING Insert transpose op for images at input 0 2024-10-21 16:26:19,026 WARNING Insert image preprocess op for images at input 0 2024-10-21 16:26:19,075 WARNING Insert image convert op for images at input 0 2024-10-21 16:26:19,075 INFO Start to convert model. run pipeline: hbdk-set-args{mbpu=b30g2 enable-vpu-lower=false}, func.func(hbir-model-check), hbir-legalize{perf-output-dir=.}, quant-opt, func.func(op-check{expected-backends=bpu,vpu output-path=/data/customer/Forum/segment_f/model_output/ print-unit=true}), hbir-partition, func.func(hbir-source-layer), func.func(hbdk-const-fold), func.func(hbdk-insert-copy-output-op) 2024-10-21 16:26:21,698 INFO Successful convert model. ===============> HBIRHandle: 134173084615728 : 134173084615728 Remove op type hbtl.call::quant::dcast successfully, delete node type list ['Quantize', 'Transpose', 'Dequantize', 'Cast', 'Reshape', 'Softmax'] Remove op type hbir.reshape successfully, delete node type list ['Quantize', 'Transpose', 'Dequantize', 'Cast', 'Reshape', 'Softmax'] 2024-10-21 16:26:21,754 INFO Successfully remove node, the removed node info is: 2024-10-21 16:26:21,754 INFO Node Name Type 2024-10-21 16:26:21,754 INFO ---------------------------------- ---------- 2024-10-21 16:26:21,754 INFO /model/decoder/decoder/Unsqueeze_3 Dequantize 2024-10-21 16:26:21,754 INFO /model/decoder/decoder/Unsqueeze_3 Reshape ===============> HBIRHandle: 134173084615728 : 134173084615728 2024-10-21 16:26:21,846 INFO Start to compile model. [==================================================]100% [16h:42m:27s:184467us INFO hbrt4_log::logger] pid:5961 tid:5961 hbrt4_log/src/logger.rs:403: Logger of HBRT4 initialized, version = 4.1.9 [16h:42m:27s:184998us INFO hbrt4_loader::parsing] pid:5961 tid:5961 hbrt4_loader/src/parsing.rs:85: Load hbm from file; filename="/data/customer/Forum/segment_f/model_output/model1.hbm" 2024-10-21 16:42:27,242 INFO Successful compile hbm /data/customer/Forum/segment_f/model_output/model1.hbm [2024-10-21 16:42:27.302] [info] fork(): Parent process pid = 23182 [2024-10-21 16:42:27.303] [info] fork(): Child process pid = 23185 FPS=101.77, latency = 9826.2000000000007 us, DDR = 250759040 bytes (see /data/customer/Forum/segment_f/model_output/model1.html) HBDK hbm perf SUCCESS 2024-10-21 16:42:28,482 INFO Successful perf model, result generate at /data/customer/Forum/segment_f/model_output 2024-10-21 16:42:28,483 INFO hbm_path: /data/customer/Forum/segment_f/model_output/model1.hbm [16h:42m:28s:483102us INFO hbrt4_loader::parsing] pid:5961 tid:5961 hbrt4_loader/src/parsing.rs:42: Load hbm header from file; filename="/data/customer/Forum/segment_f/model_output/model1.hbm" [16h:42m:28s:486891us INFO hbrt4_log::logger] pid:5961 tid:5961 hbrt4_log/src/logger.rs:403: Logger of HBRT4 initialized, version = 4.1.9 [16h:42m:28s:486896us INFO hbrt4_loader::parsing] pid:5961 tid:5961 hbrt4_loader/src/parsing.rs:85: Load hbm from file; filename="/data/customer/Forum/segment_f/model_output/model1.hbm" HBDK hbm extract desc SUCCESS 2024-10-21 16:42:28,565 INFO ************* model1 ************* 2024-10-21 16:42:28,565 INFO ############# model deps info ############# 2024-10-21 16:42:28,565 INFO builder version : 3.3.4 2024-10-21 16:42:28,565 INFO hbdk version : 4.1.9 2024-10-21 16:42:28,565 INFO horizon nn version : 2.0.9 2024-10-21 16:42:28,565 INFO ############# model_parameters info ############# 2024-10-21 16:42:28,565 INFO onnx_model : /data/customer/Forum/segment_f/model1.onnx 2024-10-21 16:42:28,565 INFO BPU march : nash-m 2024-10-21 16:42:28,565 INFO layer_out_dump : False 2024-10-21 16:42:28,565 INFO working dir : /data/customer/Forum/segment_f/model_output 2024-10-21 16:42:28,565 INFO output_model_file_prefix: model1 2024-10-21 16:42:28,565 INFO remove_node_type : Quantize;Transpose;Dequantize;Cast;Reshape;Softmax 2024-10-21 16:42:28,565 INFO node_info : {} 2024-10-21 16:42:28,565 INFO ############# input_parameters info ############# 2024-10-21 16:42:28,565 INFO ------------------------------------------ 2024-10-21 16:42:28,565 INFO ---------input info : images --------- 2024-10-21 16:42:28,565 INFO input_name : images 2024-10-21 16:42:28,565 INFO input_type_rt : nv12 2024-10-21 16:42:28,565 INFO input_space&range : regular 2024-10-21 16:42:28,565 INFO input_type_train : bgr 2024-10-21 16:42:28,565 INFO input_layout_rt : NCHW 2024-10-21 16:42:28,566 INFO input_layout_train : NCHW 2024-10-21 16:42:28,566 INFO norm_type : no_preprocess 2024-10-21 16:42:28,566 INFO input_shape : 1x3x640x1920 2024-10-21 16:42:28,566 INFO mean_value : [] 2024-10-21 16:42:28,566 INFO scale_value : [] 2024-10-21 16:42:28,566 INFO std_value : [] 2024-10-21 16:42:28,566 INFO separate_batch : False 2024-10-21 16:42:28,566 INFO ---------input info : images end ------- 2024-10-21 16:42:28,566 INFO ------------------------------------------ 2024-10-21 16:42:28,566 INFO ############# calibration_parameters info ############# 2024-10-21 16:42:28,566 INFO calibration_type : skip 2024-10-21 16:42:28,566 INFO max_percentile : None 2024-10-21 16:42:28,566 INFO optimization : run_fast 2024-10-21 16:42:28,566 INFO per_channel : False 2024-10-21 16:42:28,566 INFO ############# compiler_parameters info ############# 2024-10-21 16:42:28,566 INFO debug : True 2024-10-21 16:42:28,566 INFO optimize_level : O2 2024-10-21 16:42:28,566 INFO compile_mode : latency 2024-10-21 16:42:28,566 INFO core_num : 1 2024-10-21 16:42:28,566 INFO balance_factor : 100 2024-10-21 16:42:28,566 INFO input_source : {'images': 'pyramid'} 2024-10-21 16:42:28,566 INFO hbdk3_compatible_mode: False 2024-10-21 16:42:28,566 INFO hbm_path: /data/customer/Forum/segment_f/model_output/model1.hbm [16h:42m:28s:566905us INFO hbrt4_loader::parsing] pid:5961 tid:5961 hbrt4_loader/src/parsing.rs:42: Load hbm header from file; filename="/data/customer/Forum/segment_f/model_output/model1.hbm" [16h:42m:28s:570760us INFO hbrt4_log::logger] pid:5961 tid:5961 hbrt4_log/src/logger.rs:403: Logger of HBRT4 initialized, version = 4.1.9 [16h:42m:28s:570765us INFO hbrt4_loader::parsing] pid:5961 tid:5961 hbrt4_loader/src/parsing.rs:85: Load hbm from file; filename="/data/customer/Forum/segment_f/model_output/model1.hbm" 2024-10-21 16:42:28,589 INFO ############# Model input/output info ############# 2024-10-21 16:42:28,589 INFO NAME TYPE SHAPE DATA_TYPE 2024-10-21 16:42:28,589 INFO --------- ------ ----------------- --------- 2024-10-21 16:42:28,589 INFO images_y input [1, 640, 1920, 1] UINT8 2024-10-21 16:42:28,589 INFO images_uv input [1, 320, 960, 2] UINT8 2024-10-21 16:42:28,589 INFO 2035 output [1, 300, 4] INT8 2024-10-21 16:42:28,592 INFO The hb_compile completes running root@hh-j6-3022:/data/customer/Forum/segment_f# ls hb_compile.log model1.onnx model_output root@hh-j6-3022:/data/customer/Forum/segment_f# cd model_output/ root@hh-j6-3022:/data/customer/Forum/segment_f/model_output# ls model1.hbm model1_advice.json model1_original_float_model.onnx model1_quantized_model.bc model1.html model1_calibrated_model.onnx model1_ptq_model.onnx model1_quantized_removed_model.bc model1.json model1_optimized_float_model.onnx model1_quant_info.json root@hh-j6-3022:/data/customer/Forum/segment_f/model_output# hb_model_info model1.hbm 2024-10-21 16:46:06,224 INFO log will be stored in /data/customer/Forum/segment_f/model_output/hb_model_info.log 2024-10-21 16:46:06,224 INFO Start hb_model_info.... 2024-10-21 16:46:06,224 INFO hb_model_info version 3.3.4 2024-10-21 16:46:06,224 INFO hbm_path: /data/customer/Forum/segment_f/model_output/model1.hbm [16h:46m:06s:234218us INFO hbrt4_log::logger] pid:23192 tid:23192 hbrt4_log/src/logger.rs:403: Logger of HBRT4 initialized, version = 4.1.9 [16h:46m:06s:234602us INFO hbrt4_loader::parsing] pid:23192 tid:23192 hbrt4_loader/src/parsing.rs:85: Load hbm from file; filename="model1.hbm" HBDK hbm extract desc SUCCESS 2024-10-21 16:46:06,315 INFO ************* model1 ************* 2024-10-21 16:46:06,315 INFO ############# model deps info ############# 2024-10-21 16:46:06,315 INFO builder version : 3.3.4 2024-10-21 16:46:06,315 INFO hbdk version : 4.1.9 2024-10-21 16:46:06,315 INFO horizon nn version : 2.0.9 2024-10-21 16:46:06,315 INFO ############# model_parameters info ############# 2024-10-21 16:46:06,315 INFO onnx_model : /data/customer/Forum/segment_f/model1.onnx 2024-10-21 16:46:06,315 INFO BPU march : nash-m 2024-10-21 16:46:06,315 INFO layer_out_dump : False 2024-10-21 16:46:06,315 INFO working dir : /data/customer/Forum/segment_f/model_output 2024-10-21 16:46:06,315 INFO output_model_file_prefix: model1 2024-10-21 16:46:06,315 INFO remove_node_type : Quantize;Transpose;Dequantize;Cast;Reshape;Softmax 2024-10-21 16:46:06,315 INFO node_info : {} 2024-10-21 16:46:06,315 INFO ############# input_parameters info ############# 2024-10-21 16:46:06,315 INFO ------------------------------------------ 2024-10-21 16:46:06,315 INFO ---------input info : images --------- 2024-10-21 16:46:06,315 INFO input_name : images 2024-10-21 16:46:06,315 INFO input_type_rt : nv12 2024-10-21 16:46:06,315 INFO input_space&range : regular 2024-10-21 16:46:06,315 INFO input_type_train : bgr 2024-10-21 16:46:06,315 INFO input_layout_rt : NCHW 2024-10-21 16:46:06,315 INFO input_layout_train : NCHW 2024-10-21 16:46:06,315 INFO norm_type : no_preprocess 2024-10-21 16:46:06,315 INFO input_shape : 1x3x640x1920 2024-10-21 16:46:06,315 INFO mean_value : [] 2024-10-21 16:46:06,315 INFO scale_value : [] 2024-10-21 16:46:06,315 INFO std_value : [] 2024-10-21 16:46:06,315 INFO separate_batch : False 2024-10-21 16:46:06,315 INFO ---------input info : images end ------- 2024-10-21 16:46:06,315 INFO ------------------------------------------ 2024-10-21 16:46:06,315 INFO ############# calibration_parameters info ############# 2024-10-21 16:46:06,315 INFO calibration_type : skip 2024-10-21 16:46:06,315 INFO max_percentile : None 2024-10-21 16:46:06,315 INFO optimization : run_fast 2024-10-21 16:46:06,315 INFO per_channel : False 2024-10-21 16:46:06,315 INFO ############# compiler_parameters info ############# 2024-10-21 16:46:06,316 INFO debug : True 2024-10-21 16:46:06,316 INFO optimize_level : O2 2024-10-21 16:46:06,316 INFO compile_mode : latency 2024-10-21 16:46:06,316 INFO core_num : 1 2024-10-21 16:46:06,316 INFO balance_factor : 100 2024-10-21 16:46:06,316 INFO input_source : {'images': 'pyramid'} 2024-10-21 16:46:06,316 INFO hbdk3_compatible_mode: False 2024-10-21 16:46:06,316 INFO hbm_path: /data/customer/Forum/segment_f/model_output/model1.hbm [16h:46m:06s:316187us INFO hbrt4_loader::parsing] pid:23192 tid:23192 hbrt4_loader/src/parsing.rs:42: Load hbm header from file; filename="model1.hbm" [16h:46m:06s:319690us INFO hbrt4_log::logger] pid:23192 tid:23192 hbrt4_log/src/logger.rs:403: Logger of HBRT4 initialized, version = 4.1.9 [16h:46m:06s:319695us INFO hbrt4_loader::parsing] pid:23192 tid:23192 hbrt4_loader/src/parsing.rs:85: Load hbm from file; filename="model1.hbm" 2024-10-21 16:46:06,338 INFO ############# Model input/output info ############# 2024-10-21 16:46:06,338 INFO NAME TYPE SHAPE DATA_TYPE 2024-10-21 16:46:06,338 INFO --------- ------ ----------------- --------- 2024-10-21 16:46:06,338 INFO images_y input [1, 640, 1920, 1] UINT8 2024-10-21 16:46:06,338 INFO images_uv input [1, 320, 960, 2] UINT8 2024-10-21 16:46:06,338 INFO 2035 output [1, 300, 4] INT8 root@hh-j6-3022:/data/customer/Forum/segment_f/model_output#