English

Model Compilation

HAT is capable of compiling fixed-point models to make the compiled Pytorch models suitable for on-board running. Users can choose to compile the model as described in the next section.

Config File as Input

This method uses the script tools/compile_perf.py to compile the model obtained from model and model_convert_pipeline in int_infer_trainer into hbm with the following command.

python3 tools/compile_perf.py -c {$CONFIG_FILE}

This command will get the compilation-related configurations from the compile_cfg field in the config file, as illustrated by the following example:

import horizon_plugin_pytorch

compile_cfg = dict(
    march=horizon_plugin_pytorch.quantization.March.NASH_E, # Can be NASH_E, NASH_M
    name="example_model",  # File name of perf
    out_dir="perf_results",  # Output path of perf
    hbm="tmp_compile/model.hbm",  # File path of the compiled model
    layer_details=True,  # Whether to output the performance of each layer
    input_source=["pyramid"],  # Input source when the model is on the board, usually pyramid is selected
    opt="O2",  # Optimization level, can be O0, O1, O2, or O3 (larger value for higher optimization level)
)

When the compilation is complete, the following files are generated.

tmp_compile/model.hbm: Compiled model file.
tmp_compile/model.hbir: Intermediate representation of the model used in the compilation process.
perf_results/example_model.json, perf_results/example_model.html: Information related to the on-board model saved in different formats, e.g., running speed, resource usage, etc.

On This Page

PTQ Conversion Tools

The hb_compile Tool

PTQ Conversion Steps

PTQ Conversion Samples

FAQ and Common Failure Resolutions

Appendix

Tutorial

Advanced Tutorial

API Reference

QAT

Model Export

Horizon Operator

Vision Process

Vision Algorithm

Sample

VP API Introduction

Data Structure

Function Interface

Model Inference Development

Model Inference API Introduction

Data Type and Data Structure

API Interface

Model Inference Tool Introduction

High Performance Library

High Performance Algorithm

HPL API Introduction

Data Structure

Functional Interface

UCP Common API Introduction

Data Structure

Functional Interface

UCP Performance Analysis Tools

Horizon Torch Samples

FRAMEWORK

TUTORIALS

EXAMPLES

API REFERENCE

Toolchain Operator Support Constraint List

Model Compilation

Config File as Input

The hb_compile Tool

QAT

Model Export

Horizon Operator

Vision Algorithm

Sample

VP API Introduction

Data Structure

Function Interface

Model Inference API Introduction

Data Type and Data Structure

API Interface

Model Inference Tool Introduction

High Performance Algorithm

HPL API Introduction

Data Structure

Functional Interface

Data Structure

Functional Interface

FRAMEWORK

TUTORIALS

EXAMPLES

API REFERENCE

#Model Compilation

#Config File as Input

Model Compilation

Config File as Input