No relevant resource is found in the selected language.
Enterprise
Worldwide
Huawei Global - English
Support Documentation
CANN V100R020C10 Development Auxiliary Tool Guide (Inference) 01
Huawei uses machine translation combined with human proofreading to translate this document to different languages in order to help you better understand the content of this document. Note: Even the most advanced machine translation cannot match the quality of professional translators. Huawei shall not bear any responsibility for translation accuracy and it is recommended that you refer to the English document (a link for which has been provided).
AIPP Configuration

AIPP Configuration

Overview

The AIPP module is introduced for image pre-processing including CSC, data normalization (by subtracting the mean value or multiplying a factor), and image cropping (by specifying the crop start and cropping the image to the size required by the neural network).

Static AIPP and dynamic AIPP modes are provided. However, the two modes cannot be both configured at a time.

  • Static AIPP: During model conversion, specify the AIPP mode to static and set the AIPP parameters. After the model is generated, the AIPP parameter values are saved in the offline model. The same AIPP parameter configurations are used in each model inference.

    If the static AIPP mode is used, batches share the same AIPP parameter configurations.

  • Dynamic AIPP: During model conversion, specify the AIPP mode to dynamic, and set different sets of dynamic AIPP configurations as required. In this way, different sets of AIPP configurations can be used for model inference. In dynamic AIPP mode, the pre-processing settings vary according to the service requirements, for example, different normalization parameters for different cameras or both YUV420 and RGB input formats to be supported. For details about APIs for dynamic AIPP parameters, see "AscendCL API Reference" > "Model Loading and Execution" > "aclmdlSetInputAIPP" in Application Software Development Guide.

    In dynamic AIPP mode, batches use different parameter configurations (such as crop) defined by the dynamic parameter structure. For details about the dynamic parameter structure, see Parameter Structure for Dynamic AIPP.

AIPP supports the following image input formats: YUV420SP_U8, RGB888_U8, XRGB8888_U8, and YUV400_U8.

  • YUV420SP_U8 is classified into YUV420SP_UV(NV12) and YUV420SP_VU(NV21 corresponding to YVU420SP_U8 in CSC Configuration) based on the UV sequence. The default format is YUV420SP_UV(NV12).
  • For RGB888_U8, the AIPP output format varies according to the value of rbuv_swap_switch.
  • In dynamic AIPP mode, parameters are computed for each inference operation, which is time consuming. Therefore, dynamic AIPP delivers poorer performance than static AIPP.
  • When AIPP is enabled, the model input is in RGB888_U8 or BGR888_U8 format. The two formats correspond to different CSC matrices.
  • Note the following requirements on input image format for model inference:
    • If AIPP is enabled, NHWC format is required for inference. In this case, the input node connected to AIPP will be changed accordingly, which may be different from that specified by the --input_format option in commands for model conversion.
    • If AIPP is not enabled and the --input_format option is not set to any input format of a model, a Caffe model and a TensorFlow model will be input in the NCHW and NHWC formats, respectively. To perform model inference, the format of input images should be consistent with that required by the model. For example, for a Caffe model, if the input image format is NHWC and the --input_format option is not set accordingly, the input model will use the default format NCHW. Therefore, the format of input images is different from that required by the model. In this case, you need to convert the input image format from NHWC to NCHW.
  • AIPP input format mapping:
    • YUV420SP_U8 corresponds to the YUV420SP format.
    • RGB888_U8 corresponds to the RGB package format.

Configuration File Template

Due to the restrictions of hardware processing logic, parameters in the configuration file must be processed in sequence: cropping -> CSC -> mean subtraction/normalization -> padding. The configuration file is described as follows:

  1. AIPP provides the following features: CSC, cropping, mean subtraction, multiplication coefficient, data exchange between channels, and single-line mode. The input images must in the RAW or uint8 data types.
  2. When this configuration file is used, uncomment the required parameters and set them to appropriate values.
  3. The parameter values in the template are default values. If some parameters in your configuration file are not configured, the default values shown in the template will be used during model conversion.
  4. The input_format parameter is required by static AIPP, while others are optional. If these parameters are not configured, the default values shown in the template will be used during model conversion.
# The AIPP configuration starts with the aipp_op flag, which indicates that it is the configuration of an AIPP operator. Multiple aipp_op flags can be configured.
aipp_op {

#========================= Global settings (start) ===========================================================================================================================================================
# aipp_mode specifies the AIPP mode. This parameter is required.
# Type: enum
# Value range: dynamic or static (dynamic indicates dynamic AIPP, and static indicates static AIPP.)
# aipp_mode: 

# related_input_rank (optional) indicates the AIPP processing start of the inputs. The default value 0 indicates that the processing starts from input 0. If the model has two inputs, to start processing from the second, set related_input_rank to 1.
# Type: int
# Value range: >= 0
# related_input_rank: 0
 
#========================= Global settings (end) =============================================================================================================================================================

#=========================Dynamic AIPP settings (start) ===================================================================================================================================
# The maximum size of the input image must be configured in dynamic AIPP. (In the dynamic batch scenario, N is set to the maximum in batch size.)
# Type: int
# max_src_image_size: 0
# If the input image format is YUV400_U8, max_src_image_size >= N * src_image_size_w * src_image_size_h * 1.
# If the input image format is YUV420SP_U8, max_src_image_size >= N * src_image_size_w * src_image_size_h * 1.5.
# If the input image format is XRGB8888_U8, max_src_image_size>=N * src_image_size_w * src_image_size_h * 4.
# If the input image format is RGB888_U8, max_src_image_size >= N * src_image_size_w * src_image_size_h * 3.

# Rotation enable. This parameter is reserved. Rotation is not supported currently.
# Type: bool
# Value: true (enabled) or false (disabled)
# support_rotation: false
#========================= Dynamic AIPP settings (end) =======================================================================================================================================

#========================= Static AIPP settings (start) ======================================================================================================================================
#Input image format, which is optional.
# Type: enum
# Value: YUV420SP_U8, XRGB8888_U8, RGB888_U8, or YUV400_U8
# input_format:
# Note: After model conversion is complete, the preceding parameters are displayed as the following enumerated values in the .om model file: 1, 2, 3, 4.

# Width and height of the source image
# Type: int32
# Value range & constraint: The ranges of width and height are [2, 4096] or 0, and [1, 4096] or 0, respectively. For YUV420SP_U8 images, the values must be even numbers.
# src_image_size_w: 0
# src_image_size_h: 0
# Note: Set src_image_size_w and src_image_size_h to the actual image width and height. If crop is disabled and padding settings are not configured, they can be set to 0 or not configured. In this case, the values of width and height are obtained from those defined in the network model, with a width and height in the range of [2, 4096] and [1, 4096] respectively.

# Padding value for the C dimension. This parameter is reserved. Padding is not supported currently.
# Type: float16
# Value range: [–65504, +65504]
# cpadding_value: 0.0

#========= Crop settings (For the configuration example, see section AIPP Configuration > Crop/Padding Configuration.) =========
# Whether to enable crop in AIPP
# Type: bool
# Value: true (enabled) or false (disabled)
# crop: false

# Start coordinates of the cropping area. W and H defined in the network input are used as the size of the cropped image.
# Type: int32
# Value range: [0, 4095]. For YUV420SP_U8 images, the value must be an even number.
# Note: load_start_pos_w < src_image_size_w, load_start_pos_h < src_image_size_h
# load_start_pos_w: 0
# load_start_pos_h: 0

# Size of the cropped image
# Type: int32
# Value range: [0, 4096], load_start_pos_w + crop_size_w <= src_image_size_w, and load_start_pos_h + crop_size_h <= src_image_size_h
# crop_size_w: 0
# crop_size_h: 0
Note: If the image cropping function is enabled and padding settings are not configured, crop_size_w and crop_size_h can be set to 0 or not configured. In this case, the cropped width and height (crop_size[W|H]) are obtained from those in the --input_shape in the model file with a value range [1,4096].

# The following conditions must be met:
# If input_format = YUV420SP_U8, the values of crop_size_w, crop_size_h, load_start_pos_w, and load_start_pos_h must be even numbers.
# If input_format is set to other values, there is no restriction on crop_size_w, crop_size_h, load_start_pos_w, and load_start_pos_h.
# If cropping is enabled, the following condition must be met: src_image_size[W|H] >= crop_size[W|H] + load_start_pos[W|H]


#========================= Resize settings ========================
# Resize enable. This parameter is reserved. Resize is not supported currently.
# Type: bool
# Value: true (enabled) or false (disabled)
resize: false
 
# Width and height of the resized image. This parameter is reserved. Resize is not supported currently.
# Type: int32
# Value range: resize_output_h ∈ [16,4096] or 0; resize_output_w ∈ [16, 1920] or 0; resize_output_w and resize_input_w ∈ [1/16,16], resize_output_h and resize_input_h ∈ [1/16,16]
resize_output_w: 0
resize_output_h: 0
# Note: If the resize function is enabled and padding settings are not configured, resize_output_w and resize_output_h can be set to 0 or not configured. In this case, the width and height of the resized image are obtained from those in the --input_shape in the model file, with a height and width in the range of [16, 4096] and [16, 1920] respectively.


#========= Padding settings (For the configuration example, see section AIPP Configuration > Crop/Padding Configuration.) =========
# Whether to enable padding in AIPP
# Type: bool
# Value: true (enabled) or false (disabled)
# padding: false
 
# Padding values of H and W, configured in static AIPP (left_padding_size, right_padding_size, top_padding_size, and bottom_padding_size ∈ [0, 32])
# Type: int32
# left_padding_size: 0
# right_padding_size: 0
# top_padding_size: 0
# bottom_padding_size: 0
# The output H and W after AIPP padding must be the same as those required by the model. W <= 1080.


#================================ Rotation settings ==================================
# Rotation angle. This parameter is reserved. Rotation is not supported currently.
# Type: uint8
# Value range: {0, 1, 2, 3}, where, 0 indicates rotation disabled, 1 indicates 90° clockwise, 2 indicates 180° clockwise, and 3 indicates 270° clockwise.
# rotation_angle :0


#========= CSC settings (For the configuration example, see section AIPP Configuration > CSC Configuration.) =============
# CSC switch
# Type: bool
# Value: true (enabled) or false (disabled)
# csc_switch: false

# R/B or U/V channel swap switch
# Type: bool
# Value: true (enabled) or false (disabled)
# rbuv_swap_switch: false

# RGBA->ARGB and YUVA->AYUV swap switch
# Type: bool
# Value: true (enabled) or false (disabled)
# ax_swap_switch: false

# Whether to enable the single-line processing mode (only the first line after image cropping). This parameter is reserved. This function is not supported currently.
# Type: bool
# Value: true (enabled) or false (disabled)
# single_line_mode: false

# If the CSC is disabled (false), this function does not take effect.
# If the input image has 4 channels, channel A or X is ignored.
# YUV2BGR conversion:
# | B |   | matrix_r0c0 matrix_r0c1 matrix_r0c2 | | Y - input_bias_0 |
# | G | = | matrix_r1c0 matrix_r1c1 matrix_r1c2 | | U - input_bias_1 | >> 8
# | R |   | matrix_r2c0 matrix_r2c1 matrix_r2c2 | | V - input_bias_2 |
# BGR2YUV conversion:
# | Y |   | matrix_r0c0 matrix_r0c1 matrix_r0c2 | | B |        | output_bias_0 |
# | U | = | matrix_r1c0 matrix_r1c1 matrix_r1c2 | | G | >> 8 + | output_bias_1 |
# | V |   | matrix_r2c0 matrix_r2c1 matrix_r2c2 | | R |        | output_bias_2 |

# Elements in the 3 x 3 CSC matrix
# Type: int16
# Value range: [–32677, +32676]
# matrix_r0c0: 298
# matrix_r0c1: 516
# matrix_r0c2: 0
# matrix_r1c0: 298
# matrix_r1c1: -100
# matrix_r1c2: -208
# matrix_r2c0: 298
# matrix_r2c1: 0
# matrix_r2c2: 409

# Output bias for RGB2YUV conversion
# Type: uint8
# Value range: [0, 255]
# output_bias_0: 16
# output_bias_1: 128
# output_bias_2: 128

Input bias for YUV2RGB conversion
# Type: uint8
# Value range: [0, 255]
# input_bias_0: 16
# input_bias_1: 128
# input_bias_2: 128


#============================== Mean subtraction and multiplication factor settings =================================
# The computation rules are as follows:
# For uint8->uint8, this function is bypassed.
# For uint8->fp16: pixel_out_chx(i) = [pixel_in_chx(i) – mean_chn_i – min_chn_i] x var_reci_chn

#Mean value of each channel
# Type: uint8
# Value range: [0, 255]
# mean_chn_0: 0
# mean_chn_1: 0
# mean_chn_2: 0
# mean_chn_3: 0

#Minimum value of each channel
# Type: float16
# Value range: [0, 255]
# min_chn_0: 0.0
# min_chn_1: 0.0
# min_chn_2: 0.0
# min_chn_3: 0.0

# Variance value of each channel
# Type: float16
# Value range: [–65504, +65504]
# var_reci_chn_0: 1.0
# var_reci_chn_1: 1.0
# var_reci_chn_2: 1.0
# var_reci_chn_3: 1.0
}



#========================= Static AIPP settings (end) =====================================================================================================================================

CSC Configuration

If the input image format is inconsistent with that required by the model, CSC needs to be performed prior to AIPP processing. The value of matrix_r*c* is fixed and does not need to be adjusted.

The following two typical CSC configurations are provided for images or videos (in various color encoding modes such as YUV420SP_U8 and RGB888_U8) input to the model:

  • For JPEG image files (such as jpg, jpeg, JPG, and JPEG), you can select from the CSC configurations in the JPEG columns in the following tables.
  • For the decoded video data, the CSC parameters can be configured according to different color video digital standards (such as BT.601 and BT.709).
    • BT.601 is the standard for standard-definition television (SDTV).
    • BT.709 is the standard for high-definition television (HDTV).

    The two standards are classified into narrow range (Video Range) and wide range (Full Range) according to their representation range.

    The value ranges of narrow and wide are listed in and respectively. For details, see How Do I Determine the Video Stream Format Standard When I Perform CSC on a Model Using AIPP?.

No change in format occurs in CSC configuration when data is processed by DVPP. If the data prior to DVPP processing is in the BT.709 format, the data input format in AIPP remains the same after DVPP processing.

YUV420SP_U8 to YUV444

aipp_op {
    aipp_mode: static
    input_format : YUV420SP_U8
    csc_switch : false
    rbuv_swap_switch : false
}

YUV420SP_U8 to YVU444

aipp_op {
    aipp_mode: static
    input_format : YUV420SP_U8
    csc_switch : false
    rbuv_swap_switch : true
}

YUV420SP_U8 to RGB

Table 2-2 YUV420SP_U8 to RGB

JPEG

BT-601NARROW

BT-601WIDE

BT-709NARROW

BT-709WIDE

 
aipp_op {
    aipp_mode: static
    input_format : YUV420SP_U8
    csc_switch : true
    rbuv_swap_switch : false
    matrix_r0c0 : 256
    matrix_r0c1 : 0
    matrix_r0c2 : 359
    matrix_r1c0 : 256
    matrix_r1c1 : -88
    matrix_r1c2 : -183
    matrix_r2c0 : 256
    matrix_r2c1 : 454
    matrix_r2c2 : 0
    input_bias_0 : 0
    input_bias_1 : 128
    input_bias_2 : 128
}
 
aipp_op {
    aipp_mode: static
    input_format : YUV420SP_U8
    csc_switch : true
    rbuv_swap_switch : false
    matrix_r0c0 : 298
    matrix_r0c1 : 0
    matrix_r0c2 : 409
    matrix_r1c0 : 298
    matrix_r1c1 : -100
    matrix_r1c2 : -208
    matrix_r2c0 : 298
    matrix_r2c1 : 516
    matrix_r2c2 : 0
    input_bias_0 : 16
    input_bias_1 : 128
    input_bias_2 : 128
}
 
aipp_op {
    aipp_mode: static
    input_format : YUV420SP_U8
    csc_switch : true
    rbuv_swap_switch : false
    matrix_r0c0 : 256
    matrix_r0c1 : 0
    matrix_r0c2 : 359
    matrix_r1c0 : 256
    matrix_r1c1 : -88
    matrix_r1c2 : -183
    matrix_r2c0 : 256
    matrix_r2c1 : 454
    matrix_r2c2 : 0
    input_bias_0 : 0
    input_bias_1 : 128
    input_bias_2 : 128
}
 
aipp_op {
    aipp_mode: static
    input_format : YUV420SP_U8
    csc_switch : true
    rbuv_swap_switch : false
    matrix_r0c0 : 298
    matrix_r0c1 : 0
    matrix_r0c2 : 459
    matrix_r1c0 : 298
    matrix_r1c1 : -55
    matrix_r1c2 : -136
    matrix_r2c0 : 298
    matrix_r2c1 : 541
    matrix_r2c2 : 0
    input_bias_0 : 16
    input_bias_1 : 128
    input_bias_2 : 128
}
 
aipp_op {
    aipp_mode: static
    input_format : YUV420SP_U8
    csc_switch : true
    rbuv_swap_switch : false
    matrix_r0c0 : 256
    matrix_r0c1 : 0
    matrix_r0c2 : 403
    matrix_r1c0 : 256
    matrix_r1c1 : -48
    matrix_r1c2 : -120
    matrix_r2c0 : 256
    matrix_r2c1 : 475
    matrix_r2c2 : 0
    input_bias_0 : 0
    input_bias_1 : 128
    input_bias_2 : 128
}

YUV420SP_U8 to BGR

Table 2-3 YUV420SP_U8 to BGR

JPEG

BT-601NARROW

BT-601WIDE

BT-709NARROW

BT-709WIDE

 
aipp_op {
    aipp_mode: static
    input_format : YUV420SP_U8
    csc_switch : true
    rbuv_swap_switch : false
    matrix_r0c0 : 256
    matrix_r0c1 : 454
    matrix_r0c2 : 0
    matrix_r1c0 : 256
    matrix_r1c1 : -88
    matrix_r1c2 : -183
    matrix_r2c0 : 256
    matrix_r2c1 : 0
    matrix_r2c2 : 359
    input_bias_0 : 0
    input_bias_1 : 128
    input_bias_2 : 128
}
 
aipp_op {
    aipp_mode: static
    input_format : YUV420SP_U8
    csc_switch : true
    rbuv_swap_switch : false
    matrix_r0c0 : 298
    matrix_r0c1 : 516
    matrix_r0c2 : 0
    matrix_r1c0 : 298
    matrix_r1c1 : -100
    matrix_r1c2 : -208
    matrix_r2c0 : 298
    matrix_r2c1 : 0
    matrix_r2c2 : 409
    input_bias_0 : 16
    input_bias_1 : 128
    input_bias_2 : 128
}
 
aipp_op {
    aipp_mode: static
    input_format : YUV420SP_U8
    csc_switch : true
    rbuv_swap_switch : false
    matrix_r0c0 : 256
    matrix_r0c1 : 454
    matrix_r0c2 : 0
    matrix_r1c0 : 256
    matrix_r1c1 : -88
    matrix_r1c2 : -183
    matrix_r2c0 : 256
    matrix_r2c1 : 0
    matrix_r2c2 : 359
    input_bias_0 : 0
    input_bias_1 : 128
    input_bias_2 : 128
}
 
aipp_op {
    aipp_mode: static
    input_format : YUV420SP_U8
    csc_switch : true
    rbuv_swap_switch : false
    matrix_r0c0 : 298
    matrix_r0c1 : 541
    matrix_r0c2 : 0
    matrix_r1c0 : 298
    matrix_r1c1 : -55
    matrix_r1c2 : -136
    matrix_r2c0 : 298
    matrix_r2c1 : 0
    matrix_r2c2 : 459
    input_bias_0 : 16
    input_bias_1 : 128
    input_bias_2 : 128
}
 
aipp_op {
    aipp_mode: static
    input_format : YUV420SP_U8
    csc_switch : true
    rbuv_swap_switch : false
    matrix_r0c0 : 256
    matrix_r0c1 : 475
    matrix_r0c2 : 0
    matrix_r1c0 : 256
    matrix_r1c1 : -48
    matrix_r1c2 : -120
    matrix_r2c0 : 256
    matrix_r2c1 : 0
    matrix_r2c2 : 403
    input_bias_0 : 0
    input_bias_1 : 128
    input_bias_2 : 128
}

YUV420SP_U8 to Gray

aipp_op {
    aipp_mode: static
    input_format : YUV420SP_U8
    csc_switch : true
    rbuv_swap_switch : false
    matrix_r0c0 : 256
    matrix_r0c1 : 0
    matrix_r0c2 : 0
    matrix_r1c0 : 0
    matrix_r1c1 : 0
    matrix_r1c2 : 0
    matrix_r2c0 : 0
    matrix_r2c1 : 0
    matrix_r2c2 : 0
    input_bias_0 : 0
    input_bias_1 : 0
    input_bias_2 : 0
}

YVU420SP_U8 to RGB

Table 2-4 YVU420SP_U8 to RGB

JPEG

BT-601NARROW

BT-601WIDE

BT-709NARROW

BT-709WIDE

 
aipp_op {
    aipp_mode: static
    input_format : YUV420SP_U8
    csc_switch : true
    rbuv_swap_switch : true
    matrix_r0c0 : 256
    matrix_r0c1 : 0
    matrix_r0c2 : 359
    matrix_r1c0 : 256
    matrix_r1c1 : -88
    matrix_r1c2 : -183
    matrix_r2c0 : 256
    matrix_r2c1 : 454
    matrix_r2c2 : 0
    input_bias_0 : 0
    input_bias_1 : 128
    input_bias_2 : 128
}
 
aipp_op {
    aipp_mode: static
    input_format : YUV420SP_U8
    csc_switch : true
    rbuv_swap_switch : true
    matrix_r0c0 : 298
    matrix_r0c1 : 0
    matrix_r0c2 : 409
    matrix_r1c0 : 298
    matrix_r1c1 : -100
    matrix_r1c2 : -208
    matrix_r2c0 : 298
    matrix_r2c1 : 516
    matrix_r2c2 : 0
    input_bias_0 : 16
    input_bias_1 : 128
    input_bias_2 : 128
}
 
aipp_op {
    aipp_mode: static
    input_format : YUV420SP_U8
    csc_switch : true
    rbuv_swap_switch : true
    matrix_r0c0 : 256
    matrix_r0c1 : 0
    matrix_r0c2 : 359
    matrix_r1c0 : 256
    matrix_r1c1 : -88
    matrix_r1c2 : -183
    matrix_r2c0 : 256
    matrix_r2c1 : 454
    matrix_r2c2 : 0
    input_bias_0 : 0
    input_bias_1 : 128
    input_bias_2 : 128
}
 
aipp_op {
    aipp_mode: static
    input_format : YUV420SP_U8
    csc_switch : true
    rbuv_swap_switch : true
    matrix_r0c0 : 298
    matrix_r0c1 : 0
    matrix_r0c2 : 459
    matrix_r1c0 : 298
    matrix_r1c1 : -55
    matrix_r1c2 : -136
    matrix_r2c0 : 298
    matrix_r2c1 : 541
    matrix_r2c2 : 0
    input_bias_0 : 16
    input_bias_1 : 128
    input_bias_2 : 128
}
 
aipp_op {
    aipp_mode: static
    input_format : YUV420SP_U8
    csc_switch : true
    rbuv_swap_switch : true
    matrix_r0c0 : 256
    matrix_r0c1 : 0
    matrix_r0c2 : 403
    matrix_r1c0 : 256
    matrix_r1c1 : -48
    matrix_r1c2 : -120
    matrix_r2c0 : 256
    matrix_r2c1 : 475
    matrix_r2c2 : 0
    input_bias_0 : 0
    input_bias_1 : 128
    input_bias_2 : 128
}

YVU420SP_U8 to BGR

Table 2-5 YVU420SP_U8 to BGR

JPEG

BT-601NARROW

BT-601WIDE

BT-709NARROW

BT-709WIDE

 
aipp_op {
    aipp_mode: static
    input_format : YUV420SP_U8
    csc_switch : true
    rbuv_swap_switch : true
    matrix_r0c0 : 256
    matrix_r0c1 : 454
    matrix_r0c2 : 0
    matrix_r1c0 : 256
    matrix_r1c1 : -88
    matrix_r1c2 : -183
    matrix_r2c0 : 256
    matrix_r2c1 : 0
    matrix_r2c2 : 359
    input_bias_0 : 0
    input_bias_1 : 128
    input_bias_2 : 128
}
 
aipp_op {
    aipp_mode: static
    input_format : YUV420SP_U8
    csc_switch : true
    rbuv_swap_switch : true
    matrix_r0c0 : 298
    matrix_r0c1 : 516
    matrix_r0c2 : 0
    matrix_r1c0 : 298
    matrix_r1c1 : -100
    matrix_r1c2 : -208
    matrix_r2c0 : 298
    matrix_r2c1 : 0
    matrix_r2c2 : 409
    input_bias_0 : 16
    input_bias_1 : 128
    input_bias_2 : 128
}
 
aipp_op {
    aipp_mode: static
    input_format : YUV420SP_U8
    csc_switch : true
    rbuv_swap_switch : true
    matrix_r0c0 : 256
    matrix_r0c1 : 454
    matrix_r0c2 : 0
    matrix_r1c0 : 256
    matrix_r1c1 : -88
    matrix_r1c2 : -183
    matrix_r2c0 : 256
    matrix_r2c1 : 0
    matrix_r2c2 : 359
    input_bias_0 : 0
    input_bias_1 : 128
    input_bias_2 : 128
}
 
aipp_op {
    aipp_mode: static
    input_format : YUV420SP_U8
    csc_switch : true
    rbuv_swap_switch : true
    matrix_r0c0 : 298
    matrix_r0c1 : 541
    matrix_r0c2 : 0
    matrix_r1c0 : 298
    matrix_r1c1 : -55
    matrix_r1c2 : -136
    matrix_r2c0 : 298
    matrix_r2c1 : 0
    matrix_r2c2 : 459
    input_bias_0 : 16
    input_bias_1 : 128
    input_bias_2 : 128
}
 
aipp_op {
    aipp_mode: static
    input_format : YUV420SP_U8
    csc_switch : true
    rbuv_swap_switch : true
    matrix_r0c0 : 256
    matrix_r0c1 : 475
    matrix_r0c2 : 0
    matrix_r1c0 : 256
    matrix_r1c1 : -48
    matrix_r1c2 : -120
    matrix_r2c0 : 256
    matrix_r2c1 : 0
    matrix_r2c2 : 403
    input_bias_0 : 0
    input_bias_1 : 128
    input_bias_2 : 128
}

RGB888_U8 to RGB

aipp_op {
    aipp_mode: static
    input_format : RGB888_U8
    csc_switch : false
    rbuv_swap_switch : false
}

RGB888_U8 to BGR

aipp_op {
    aipp_mode : static
    input_format : RGB888_U8
    csc_switch : false
    rbuv_swap_switch : true
}

RGB888_U8 to YUV444

Table 2-6 RGB888_U8 to YUV444

JPEG

BT-601NARROW

BT-601WIDE

BT-709NARROW

BT-709WIDE

 
aipp_op {
    aipp_mode: static
    input_format : RGB888_U8
    csc_switch : true
    rbuv_swap_switch : false
    matrix_r0c0 : 77
    matrix_r0c1 : 150
    matrix_r0c2 : 29
    matrix_r1c0 : -43
    matrix_r1c1 : -85
    matrix_r1c2 : 128
    matrix_r2c0 : 128
    matrix_r2c1 : -107
    matrix_r2c2 : -21
    output_bias_0 : 0
    output_bias_1 : 128
    output_bias_2 : 128
}
 
aipp_op {
    aipp_mode: static
    input_format : RGB888_U8
    csc_switch : true
    rbuv_swap_switch : false
    matrix_r0c0 : 66
    matrix_r0c1 : 129
    matrix_r0c2 : 25
    matrix_r1c0 : -38
    matrix_r1c1 : -74
    matrix_r1c2 : 112
    matrix_r2c0 : 112
    matrix_r2c1 : -94
    matrix_r2c2 : -18
    output_bias_0 : 16
    output_bias_1 : 128
    output_bias_2 : 128
}
 
aipp_op {
    aipp_mode: static
    input_format : RGB888_U8
    csc_switch : true
    rbuv_swap_switch : false
    matrix_r0c0 : 77
    matrix_r0c1 : 150
    matrix_r0c2 : 29
    matrix_r1c0 : -43
    matrix_r1c1 : -85
    matrix_r1c2 : 128
    matrix_r2c0 : 128
    matrix_r2c1 : -107
    matrix_r2c2 : -21
    output_bias_0 : 0
    output_bias_1 : 128
    output_bias_2 : 128
}
 
aipp_op {
    aipp_mode: static
    input_format : RGB888_U8
    csc_switch : true
    rbuv_swap_switch : false
    matrix_r0c0 : 47
    matrix_r0c1 : 157
    matrix_r0c2 : 16
    matrix_r1c0 : -26
    matrix_r1c1 : -87
    matrix_r1c2 : 112
    matrix_r2c0 : 112
    matrix_r2c1 : -102
    matrix_r2c2 : -10
    output_bias_0 : 16
    output_bias_1 : 128
    output_bias_2 : 128
}
 
aipp_op {
    aipp_mode: static
    input_format : RGB888_U8
    csc_switch : true
    rbuv_swap_switch : false
    matrix_r0c0 : 55
    matrix_r0c1 : 183
    matrix_r0c2 : 19
    matrix_r1c0 : -29
    matrix_r1c1 : -99
    matrix_r1c2 : 128
    matrix_r2c0 : 128
    matrix_r2c1 : -116
    matrix_r2c2 : -12
    output_bias_0 : 0
    output_bias_1 : 128
    output_bias_2 : 128
}

RGB888_U8 to YVU444

Table 2-7 RGB888_U8 to YVU444

JPEG

BT-601NARROW

BT-601WIDE

BT-709NARROW

BT-709WIDE

 
aipp_op {
    aipp_mode: static
    input_format : RGB8888_U8
    csc_switch : true
    rbuv_swap_switch : false
    matrix_r0c0 : 77
    matrix_r0c1 : 150
    matrix_r0c2 : 29
    matrix_r1c0 : 128
    matrix_r1c1 : -107
    matrix_r1c2 : -21
    matrix_r2c0 : -43
    matrix_r2c1 : -85
    matrix_r2c2 : 128
    output_bias_0 : 0
    output_bias_1 : 128
    output_bias_2 : 128
}
 
aipp_op {
    aipp_mode: static
    input_format : RGB888_U8
    csc_switch : true
    rbuv_swap_switch : false
    matrix_r0c0 : 66
    matrix_r0c1 : 129
    matrix_r0c2 : 25
    matrix_r1c0 : 112
    matrix_r1c1 : -94
    matrix_r1c2 : -18
    matrix_r2c0 : -38
    matrix_r2c1 : -74
    matrix_r2c2 : 112
    output_bias_0 : 16
    output_bias_1 : 128
    output_bias_2 : 128
}
 
aipp_op {
    aipp_mode: static
    input_format : RGB888_U8
    csc_switch : true
    rbuv_swap_switch : false
    matrix_r0c0 : 77
    matrix_r0c1 : 150
    matrix_r0c2 : 29
    matrix_r1c0 : 128
    matrix_r1c1 : -107
    matrix_r1c2 : -21
    matrix_r2c0 : -43
    matrix_r2c1 : -85
    matrix_r2c2 : 128
    output_bias_0 : 0
    output_bias_1 : 128
    output_bias_2 : 128
}
 
aipp_op {
    aipp_mode: static
    input_format : RGB888_U8
    csc_switch : true
    rbuv_swap_switch : false
    matrix_r0c0 : 47
    matrix_r0c1 : 157
    matrix_r0c2 : 16
    matrix_r1c0 : 112
    matrix_r1c1 : -102
    matrix_r1c2 : -10
    matrix_r2c0 : -26
    matrix_r2c1 : -87
    matrix_r2c2 : 112
    output_bias_0 : 16
    output_bias_1 : 128
    output_bias_2 : 128
}
 
aipp_op {
    aipp_mode: static
    input_format : RGB888_U8
    csc_switch : true
    rbuv_swap_switch : false
    matrix_r0c0 : 55
    matrix_r0c1 : 183
    matrix_r0c2 : 19
    matrix_r1c0 : 128
    matrix_r1c1 : -116
    matrix_r1c2 : -12
    matrix_r2c0 : -29
    matrix_r2c1 : -99
    matrix_r2c2 : 128
    output_bias_0 : 0
    output_bias_1 : 128
    output_bias_2 : 128
}

RGB888_U8 to Gray

aipp_op {
    aipp_mode: static
    input_format : RGB888_U8
    csc_switch : true
    rbuv_swap_switch : false
    matrix_r0c0 : 76
    matrix_r0c1 : 150
    matrix_r0c2 : 30
    matrix_r1c0 : 0
    matrix_r1c1 : 0
    matrix_r1c2 : 0
    matrix_r2c0 : 0
    matrix_r2c1 : 0
    matrix_r2c2 : 0
    output_bias_0 : 0
    output_bias_1 : 0
    output_bias_2 : 0
}

BGR888_U8 to Gray

aipp_op {
    aipp_mode: static
    input_format : RGB888_U8
    csc_switch : true
    rbuv_swap_switch : true
    matrix_r0c0 : 76
    matrix_r0c1 : 150
    matrix_r0c2 : 30
    matrix_r1c0 : 0
    matrix_r1c1 : 0
    matrix_r1c2 : 0
    matrix_r2c0 : 0
    matrix_r2c1 : 0
    matrix_r2c2 : 0
    output_bias_0 : 0
    output_bias_1 : 0
    output_bias_2 : 0
}

BGR888_U8 to RGB

aipp_op {
    aipp_mode: static
    input_format : RGB888_U8
    csc_switch : false
    rbuv_swap_switch : true
}

BGR888_U8 to BGR

aipp_op {
    aipp_mode: static
    input_format : RGB888_U8
    csc_switch : false
    rbuv_swap_switch : false
}

XRGB8888_U8 to RGB

aipp_op {
    aipp_mode: static
    input_format : XRGB8888_U8
    csc_switch : false
    rbuv_swap_switch : false
    ax_swap_switch : true
}

XRGB8888_U8 to BGR

aipp_op {
aipp_mode : static
input_format : XRGB8888_U8
csc_switch : false
rbuv_swap_switch : true
ax_swap_switch : true
}

XRGB8888_U8 to YUV444

Table 2-8 XRGB8888_U8 to YUV444

JPEG

BT-601NARROW

BT-601WIDE

BT-709NARROW

BT-709WIDE

 
aipp_op {
    aipp_mode: static
    input_format : XRGB8888_U8
    csc_switch : true
    rbuv_swap_switch : false
    ax_swap_switch : true
    matrix_r0c0 : 77
    matrix_r0c1 : 150
    matrix_r0c2 : 29
    matrix_r1c0 : -43
    matrix_r1c1 : -85
    matrix_r1c2 : 128
    matrix_r2c0 : 128
    matrix_r2c1 : -107
    matrix_r2c2 : -21
    output_bias_0 : 0
    output_bias_1 : 128
    output_bias_2 : 128
}
 
aipp_op {
    aipp_mode: static
    input_format : XRGB8888_U8
    csc_switch : true
    rbuv_swap_switch : false
    ax_swap_switch : true
    matrix_r0c0 : 66
    matrix_r0c1 : 129
    matrix_r0c2 : 25
    matrix_r1c0 : -38
    matrix_r1c1 : -74
    matrix_r1c2 : 112
    matrix_r2c0 : 112
    matrix_r2c1 : -94
    matrix_r2c2 : -18
    output_bias_0 : 16
    output_bias_1 : 128
    output_bias_2 : 128
}
 
aipp_op {
    aipp_mode: static
    input_format : XRGB8888_U8
    csc_switch : true
    rbuv_swap_switch : false
    ax_swap_switch : true
    matrix_r0c0 : 77
    matrix_r0c1 : 150
    matrix_r0c2 : 29
    matrix_r1c0 : -43
    matrix_r1c1 : -85
    matrix_r1c2 : 128
    matrix_r2c0 : 128
    matrix_r2c1 : -107
    matrix_r2c2 : -21
    output_bias_0 : 0
    output_bias_1 : 128
    output_bias_2 : 128
}
 
aipp_op {
    aipp_mode: static
    input_format : XRGB8888_U8
    csc_switch : true
    rbuv_swap_switch : false
    ax_swap_switch : true
    matrix_r0c0 : 47
    matrix_r0c1 : 157
    matrix_r0c2 : 16
    matrix_r1c0 : -26
    matrix_r1c1 : -87
    matrix_r1c2 : 112
    matrix_r2c0 : 112
    matrix_r2c1 : -102
    matrix_r2c2 : -10
    output_bias_0 : 16
    output_bias_1 : 128
    output_bias_2 : 128
}
 
aipp_op {
    aipp_mode: static
    input_format : XRGB8888_U8
    csc_switch : true
    rbuv_swap_switch : false
    ax_swap_switch : true
    matrix_r0c0 : 55
    matrix_r0c1 : 183
    matrix_r0c2 : 19
    matrix_r1c0 : -29
    matrix_r1c1 : -99
    matrix_r1c2 : 128
    matrix_r2c0 : 128
    matrix_r2c1 : -116
    matrix_r2c2 : -12
    output_bias_0 : 0
    output_bias_1 : 128
    output_bias_2 : 128
}

XRGB8888_U8 to YVU444

Table 2-9 XRGB8888_U8 to YVU444

JPEG

BT-601NARROW

BT-601WIDE

BT-709NARROW

BT-709WIDE

 
aipp_op {
    aipp_mode: static
    input_format : XRGB8888_U8
    csc_switch : true
    rbuv_swap_switch : false
    ax_swap_switch : true
    matrix_r0c0 : 77
    matrix_r0c1 : 150
    matrix_r0c2 : 29
    matrix_r1c0 : 128
    matrix_r1c1 : -107
    matrix_r1c2 : -21
    matrix_r2c0 : -43
    matrix_r2c1 : -85
    matrix_r2c2 : 128
    output_bias_0 : 0
    output_bias_1 : 128
    output_bias_2 : 128
}
 
aipp_op {
    aipp_mode: static
    input_format : XRGB8888_U8
    csc_switch : true
    rbuv_swap_switch : false
    ax_swap_switch : true
    matrix_r0c0 : 66
    matrix_r0c1 : 129
    matrix_r0c2 : 25
    matrix_r1c0 : 112
    matrix_r1c1 : -94
    matrix_r1c2 : -18
    matrix_r2c0 : -38
    matrix_r2c1 : -74
    matrix_r2c2 : 112
    output_bias_0 : 16
    output_bias_1 : 128
    output_bias_2 : 128
}
 
aipp_op {
    aipp_mode: static
    input_format : XRGB8888_U8
    csc_switch : true
    rbuv_swap_switch : false
    ax_swap_switch : true
    matrix_r0c0 : 77
    matrix_r0c1 : 150
    matrix_r0c2 : 29
    matrix_r1c0 : 128
    matrix_r1c1 : -107
    matrix_r1c2 : -21
    matrix_r2c0 : -43
    matrix_r2c1 : -85
    matrix_r2c2 : 128
    output_bias_0 : 0
    output_bias_1 : 128
    output_bias_2 : 128
}
 
aipp_op {
    aipp_mode: static
    input_format : XRGB8888_U8
    csc_switch : true
    rbuv_swap_switch : false
    ax_swap_switch : true
    matrix_r0c0 : 47
    matrix_r0c1 : 157
    matrix_r0c2 : 16
    matrix_r1c0 : 112
    matrix_r1c1 : -102
    matrix_r1c2 : -10
    matrix_r2c0 : -26
    matrix_r2c1 : -87
    matrix_r2c2 : 112
    output_bias_0 : 16
    output_bias_1 : 128
    output_bias_2 : 128
}
 
aipp_op {
    aipp_mode: static
    input_format : XRGB8888_U8
    csc_switch : true
    rbuv_swap_switch : false
    ax_swap_switch : true
    matrix_r0c0 : 55
    matrix_r0c1 : 183
    matrix_r0c2 : 19
    matrix_r1c0 : 128
    matrix_r1c1 : -116
    matrix_r1c2 : -12
    matrix_r2c0 : -29
    matrix_r2c1 : -99
    matrix_r2c2 : 128
    output_bias_0 : 0
    output_bias_1 : 128
    output_bias_2 : 128
}

XRGB8888_U8 to Gray

aipp_op {
    aipp_mode: static
    input_format : XRGB8888_U8
    csc_switch : true
    rbuv_swap_switch : false
    ax_swap_switch : true
    matrix_r0c0 : 76
    matrix_r0c1 : 150
    matrix_r0c2 : 30
    matrix_r1c0 : 0
    matrix_r1c1 : 0
    matrix_r1c2 : 0
    matrix_r2c0 : 0
    matrix_r2c1 : 0
    matrix_r2c2 : 0
    output_bias_0 : 0
    output_bias_1 : 0
    output_bias_2 : 0
}

XBGR8888_U8 to Gray

aipp_op {
    aipp_mode: static
    input_format : XRGB8888_U8
    csc_switch : true
    rbuv_swap_switch : true
    ax_swap_switch : true
    matrix_r0c0 : 76
    matrix_r0c1 : 150
    matrix_r0c2 : 30
    matrix_r1c0 : 0
    matrix_r1c1 : 0
    matrix_r1c2 : 0
    matrix_r2c0 : 0
    matrix_r2c1 : 0
    matrix_r2c2 : 0
    output_bias_0 : 0
    output_bias_1 : 0
    output_bias_2 : 0
}

RGBX8888_U8 to Gray

aipp_op {
    aipp_mode: static
    input_format : XRGB8888_U8
    csc_switch : true
    rbuv_swap_switch : false
    ax_swap_switch : false
    matrix_r0c0 : 76
    matrix_r0c1 : 150
    matrix_r0c2 : 30
    matrix_r1c0 : 0
    matrix_r1c1 : 0
    matrix_r1c2 : 0
    matrix_r2c0 : 0
    matrix_r2c1 : 0
    matrix_r2c2 : 0
    output_bias_0 : 0
    output_bias_1 : 0
    output_bias_2 : 0
}

BGRX8888_U8 to Gray

aipp_op {
    aipp_mode: static
    input_format : XRGB8888_U8
    csc_switch : true
    rbuv_swap_switch : true
    ax_swap_switch : false
    matrix_r0c0 : 76
    matrix_r0c1 : 150
    matrix_r0c2 : 30
    matrix_r1c0 : 0
    matrix_r1c1 : 0
    matrix_r1c2 : 0
    matrix_r2c0 : 0
    matrix_r2c1 : 0
    matrix_r2c2 : 0
    output_bias_0 : 0
    output_bias_1 : 0
    output_bias_2 : 0
}

YUV400_U8 to Gray

aipp_op {
    aipp_mode: static
    input_format : YUV400_U8
    csc_switch : false  
}

Cropping and Padding Configuration

Currently, AIPP supports cropping and padding to change image sizes. Cropping is the first operation.

Figure 2-5 Image resize

The solid box indicates the current image size, and the dotted box indicates the image size processed by AIPP.

The width and height of the image processed by AIPP will be the same as those in --input_shape in the model file.

For YUV420SP_U8 images, the values of load_start_pos_w, load_start_pos_h, crop_size_w, and crop_size_h must be even numbers.

A configuration example is as follows.

aipp_op {
    aipp_mode: static
    input_format : YUV420SP_U8

    src_image_size_w :320  
    src_image_size_h :240

    crop :true
    load_start_pos_w :10
    load_start_pos_h :20
    crop_size_w :50
    crop_size_h :60

    padding : true
    left_padding_size :20
    right_padding_size :40
    top_padding_size :20
    bottom_padding_size :40
}

AIPP Configuration for a Multi-Input Model

The AIPP configuration file can define multiple sets of AIPP parameters. Separate AIPP processing is performed for different model inputs. Enclose each set of AIPP parameters within aipp_op { }. The following provides two sample sets of AIPP parameters.

Static AIPP Configuration Sample

Two sets of AIPP parameters are defined to perform AIPP processing on the first and second model inputs.

aipp_op {
       related_input_rank: 0 # Sequence number of the input to be processed by AIPP
       src_image_size_w : 608
       src_image_size_h : 608
       crop : false
       input_format : YUV420SP_U8
       aipp_mode: static
       csc_switch : true
       rbuv_swap_switch : false
       matrix_r0c0 : 298
       matrix_r0c1 : 0
       matrix_r0c2 : 409
       matrix_r1c0 : 298
       matrix_r1c1 : -100
       matrix_r1c2 : -208
       matrix_r2c0 : 298
       matrix_r2c1 : 516
       matrix_r2c2 : 0
       input_bias_0 : 16
       input_bias_1 : 128
       input_bias_2 : 128
       mean_chn_0 : 104
       mean_chn_1 : 117
       mean_chn_2 : 123
}
aipp_op {
       related_input_rank: 1 # Sequence number of the input to be processed by AIPP
       src_image_size_w : 608
       src_image_size_h : 608
       crop : false
       input_format : YUV420SP_U8
       aipp_mode: static
       csc_switch : true
       rbuv_swap_switch : false
       matrix_r0c0 : 298
       matrix_r0c1 : 0
       matrix_r0c2 : 409
       matrix_r1c0 : 298
       matrix_r1c1 : -100
       matrix_r1c2 : -208
       matrix_r2c0 : 298
       matrix_r2c1 : 516
       matrix_r2c2 : 0
       input_bias_0 : 16
       input_bias_1 : 128
       input_bias_2 : 128
       mean_chn_0 : 104
       mean_chn_1 : 117
       mean_chn_2 : 123
}

.

Dynamic AIPP Configuration Sample

aipp_op
{
    aipp_mode: dynamic
    related_input_rank: 0 # Sequence number of the input to be processed by AIPP
    max_src_image_size: 752640 #: Maximum size of the input image. This parameter is required.
}
aipp_op
{
    aipp_mode: dynamic
    related_input_rank: 1 # Sequence number of the input to be processed by AIPP
    max_src_image_size: 752640 #: Maximum size of the input image. This parameter is required.
}

AIPP Verification of the Model Input Size

If AIPP is configured, whether static AIPP or dynamic AIPP, the input size (input_size) of the eventually generated offline model is subject to operations such as cropping and padding. Assume that the batch size is N (the maximum batch size in the dynamic batch scenario), the input image width is src_image_size_w, and the input image height is src_image_size_h, the input image size can be calculated in the formulas in Table 2-10.

Table 2-10 input_size verification formulas

input_format

input_size

YUV400_U8

 
N * src_image_size_w * src_image_size_h * 1

YUV420SP_U8

 
N * src_image_size_w * src_image_size_h * 1.5

XRGB8888_U8

 
N * src_image_size_w * src_image_size_h * 4

RGB888_U8

 
N * src_image_size_w * src_image_size_h * 3

For dynamic AIPP, the ATC adds a new model input for passing the AIPP parameters. input_size of the new input is calculated as follows.

sizeof(kAippDynamicPara) – sizeof(kAippDynamicBatchPara) + batch_count * sizeof(kAippDynamicBatchPara)

For details about the kAippDynamicPara and kAippDynamicBatchPara parameters, see Parameter Structure for Dynamic AIPP.

Parameter Structure for Dynamic AIPP

After the dynamic AIPP file is configured based on the Configuration File Template, the following struct will be constructed automatically with the file. No manual operation is required.

typedef struct tagAippDynamicBatchPara
{
    int8_t cropSwitch;              //crop switch
    int8_t scfSwitch;               //resize switch
    int8_t paddingSwitch;   // 0: unable padding, 
                           // 1: padding config value,sfr_filling_hblank_ch0 ~    sfr_filling_hblank_ch2
                          // 2: padding source picture data, single row/collumn copy
                          // 3: padding source picture data, block copy
                          // 4: padding source picture data, mirror copy
    int8_t rotateSwitch;  //rotate switch, 0: rotation disabled, 1: 90° clockwise, 2: 180° clockwise, 3: 270° clockwise
    int8_t reserve[4];
    int32_t cropStartPosW;          //the start horizontal position of cropping
    int32_t cropStartPosH;          //the start vertical position of cropping
    int32_t cropSizeW;              //crop width
    int32_t cropSizeH;              //crop height
    int32_t scfInputSizeW;          //input width of scf
    int32_t scfInputSizeH;          //input height of scf
    int32_t scfOutputSizeW;         //output width of scf
    int32_t scfOutputSizeH;         //output height of scf
    int32_t paddingSizeTop;         //top padding size
    int32_t paddingSizeBottom;      //bottom padding size
    int32_t paddingSizeLeft;        //left padding size
    int32_t paddingSizeRight;       //right padding size
    int16_t dtcPixelMeanChn0;       //mean value of channel 0
    int16_t dtcPixelMeanChn1;       //mean value of channel 1
    int16_t dtcPixelMeanChn2;       //mean value of channel 2
    int16_t dtcPixelMeanChn3;       //mean value of channel 3
#ifndef DAVINCI_TINY
    uint16_t dtcPixelMinChn0;       //min value of channel 0
    uint16_t dtcPixelMinChn1;       //min value of channel 1
    uint16_t dtcPixelMinChn2;       //min value of channel 2
    uint16_t dtcPixelMinChn3;       //min value of channel 3
    uint16_t dtcPixelVarReciChn0;   //sfr_dtc_pixel_variance_reci_ch0
    uint16_t dtcPixelVarReciChn1;   //sfr_dtc_pixel_variance_reci_ch1
    uint16_t dtcPixelVarReciChn2;   //sfr_dtc_pixel_variance_reci_ch2
    uint16_t dtcPixelVarReciChn3;   //sfr_dtc_pixel_variance_reci_ch3
    int8_t reserve1[16];            //32B assign, for ub copy
#endif
}kAippDynamicBatchPara;
typedef struct tagAippDynamicPara
{
    uint8_t inputFormat;       //Input format: YUV420SP_U8, XRGB8888_U8, or RGB888_U8
    //uint8_t outDataType; //output data type: CC_DATA_HALF,CC_DATA_INT8, CC_DATA_UINT8
    int8_t cscSwitch;               //csc switch
    int8_t rbuvSwapSwitch;          //rb/ub swap switch
    int8_t axSwapSwitch;            //RGBA->ARGB, YUVA->AYUV swap switch
    int8_t batchNum;                //batch parameter number
    int8_t reserve1[3];
    int32_t srcImageSizeW;          //source image width
    int32_t srcImageSizeH;          //source image height
    int16_t cscMatrixR0C0;          //csc_matrix_r0_c0
    int16_t cscMatrixR0C1;          //csc_matrix_r0_c1
    int16_t cscMatrixR0C2;          //csc_matrix_r0_c2
    int16_t cscMatrixR1C0;          //csc_matrix_r1_c0
    int16_t cscMatrixR1C1;          //csc_matrix_r1_c1
    int16_t cscMatrixR1C2;          //csc_matrix_r1_c2
    int16_t cscMatrixR2C0;          //csc_matrix_r2_c0
    int16_t cscMatrixR2C1;          //csc_matrix_r2_c1
    int16_t cscMatrixR2C2;          //csc_matrix_r2_c2
    int16_t reserve2[3];
    uint8_t cscOutputBiasR0;   //output bias for RGB to YUV, element of row 0, unsigned number
    uint8_t cscOutputBiasR1;   //output bias for RGB to YUV, element of row 1, unsigned number
    uint8_t cscOutputBiasR2;   //output bias for RGB to YUV, element of row 2, unsigned number
    uint8_t cscInputBiasR0;    //input bias for YUV to RGB, element of row 0, unsigned number
    uint8_t cscInputBiasR1;    //input bias for YUV to RGB, element of row 1, unsigned number
    uint8_t cscInputBiasR2;    //input bias for YUV to RGB, element of row 2, unsigned number
    uint8_t reserve3[2];
    int8_t reserve4[16];            //32B assign, for ub copy
    kAippDynamicBatchPara aippBatchPara;  //allow transfer several batch para.
} kAippDynamicPara;

Samples

YUV400_U8 to Gray

In this sample, a YUV400_U8 image (224 x 224) input to AIPP is output in Gray format. The valid data starts with (0,0) coordinating in the upper left corner. For the original Caffe/TensorFlow network model (C=1, H < 224, W < 224, for example, 220), the crop parameter is required, and the coordinate parameters load_start_pos_h and load_start_pos_w need to be set to 0. Then the inference will start from coordinates (0, 0) and end at coordinates (220, 220).

In this case, the csc_switch parameter does not need to be configured. For example, a YUV420SP_U8 image is input to AIPP, the AIPP parameter configuration can be used in the same original network model (Caffe or TensorFlow). In other words, only the data of Y channel is selected.

The AIPP configuration file is as follows.

aipp_op{
    aipp_mode:static
    csc_switch : false
    crop:true
    input_format : YUV400_U8
    load_start_pos_h : 0
    load_start_pos_w : 0
    src_image_size_w : 224
    src_image_size_h : 224
    # The normalization parameters are set as required. The following values are examples only.
    mean_chn_0 :128
    min_chn_0 :0.0
    var_reci_chn_0 :0.00390625
}

YUV420SP_U8 to BGR

In this sample, a YUV420SP_U8 (NV12) image (256 x 256) input to AIPP is output in BGR format. For the original Caffe or TensorFlow network model (C=3, H and W equal to those input sizes), no configuration is required for the crop parameter, but the csc_switch and corresponding matrix parameters are necessary.

The AIPP configuration file is as follows.

aipp_op {
    related_input_rank : 0
    src_image_size_w : 256
    src_image_size_h : 256
    crop : false
    input_format : YUV420SP_U8
    aipp_mode: static
    csc_switch : true
    # If the input image is a YVU420SP_U8 (NV21) image, set rbuv_swap_switch to true.
    rbuv_swap_switch : false
    matrix_r0c0 : 298
    matrix_r0c1 : 516
    matrix_r0c2 : 0
    matrix_r1c0 : 298
    matrix_r1c1 : -100
    matrix_r1c2 : -208
    matrix_r2c0 : 298
    matrix_r2c1 : 0
    matrix_r2c2 : 409
    input_bias_0 : 16
    input_bias_1 : 128
    input_bias_2 : 128
    # The normalization parameters are set as required. The following values are examples only.
    # The normalization parameters are applied to channels after CSC and channel swap.
    mean_chn_0 : 104
    mean_chn_1 : 117
    mean_chn_2 : 123
    min_chn_0 : 0.0
    min_chn_1 : 0.0
    min_chn_2 : 0.0
    var_reci_chn_0 : 1.0
    var_reci_chn_1 : 1.0
    var_reci_chn_2 : 1.0
}

RGB888_U8 to RGB (or BGR)

In this sample, a RGB888_U8 image (250 x 250) input to AIPP is output in RGB format. The valid data starts with (0,0) coordinating in the upper left corner. For the Caffe or TensorFlow original network model (C=3, H < 250; W < 250, for example, 240), the crop parameter is required, and the coordinate parameters load_start_pos_h and load_start_pos_w need to be set to 0. Then the inference will start from coordinates (0, 0) and end at coordinates (240, 240).

In this case, no configuration is required for the rbuv_swap_switch, csc_switch, and corresponding matrix parameters.

The AIPP configuration file is as follows.

aipp_op {
    related_input_rank : 0
    src_image_size_w : 250
    src_image_size_h : 250
    crop : true
    load_start_pos_w : 0
    load_start_pos_h : 0
    aipp_mode: static
    input_format : RGB888_U8
    csc_switch : false
    # If the original Caffe/TensorFlow model requires the BGR format, set rbuv_swap_switch to true.
    rbuv_swap_switch : false
    # The normalization parameters are set as required. In this sample, these parameters are set to default values, so the pixel values are not changed.
    # The configured normalization parameters are applied to channels after channel swap.
}

Translation
简体中文
Download
Updated: 2020-12-11

Document ID: EDOC1100164848

Views: 102731

Downloads: 27

Average rating:
This Document Applies to these Products

Related Version

Related Documents

Share
Previous Next
Huawei e+ App Huawei e+

Copyright © 2022 Huawei Technologies Co., Ltd. All rights reserved.

You are going to visit :