Operator Information Definition

Information

Required or Optional

Description

[OpType]

Required

Operator type, which defines the start of operator information and is the same as the value of OpType in REG_OP(OpType) in Operator Prototype Definition.

input0.name

Required

Name of input0, which must be the same as that in the Operator Prototype Definition.

If input0.paramType is set to dynamic, set input0.name to x. The name of x must be the same as that in Operator Prototype Definition. When the graph is running, x0, x1, x2, and so on are automatically generated based on the number of inputs. The inputs are numbered starting at 0 in ascending order.

input0.paramType

Required

Type of the input0.

• dynamic: indicates the number of dynamic inputs, which can be one or more. This parameter is used together with input0.name.

  If set to dynamic, input0.name must also be specified.

• optional: indicates that the input is optional.
• required: indicates that only one value can be passed.

Defaults to required.

input0.dtype

Optional

Data types supported by input0. If multiple data types are supported, separate them with commas (,).

The options are as follows:

float16, float, int8, int16, int32, uint8, uint16, uint32, bool, and more

Notes:

• If the input tensor supports multiple dtype and format combinations, dtype and format must be configured in pairs in sequence. List all the combinations supported by the operator and separate them by commas (,).
  In the following example, 12 items are configured for input0.dtype and input0.format respectively.

  input0.dtype=float16,float16,float16,float16,float,float,float,float,int32,int32,int32,int32
  input0.format=NCHW,NC1HWC0,NHWC,ND,NCHW,NC1HWC0,NHWC,ND,NCHW,NC1HWC0,NHWC,ND

• If dynamicFormat.flag is set to true, this parameter does not need to be specified. However, the op_select_format Function function must be implemented in the operator implementation file.

input0.format

Optional

Format of the input0.

• If all data formats in the source graph are supported, set this parameter to ND.
• Separate multiple formats by commas (,). The options are as follows: NCHW, NHWC, NC1HWC0, FRACTAL, and ND.

Note:

• If the input tensor supports multiple dtype and format combinations, dtype and format must be configured in pairs in sequence. List all the combinations supported by the operator and separate them by commas (,).
• If dynamicFormat.flag is set to true, this parameter does not need to be specified. However, the op_select_format Function function must be implemented in the operator implementation file.
• If op.pattern is set, this parameter does not need to be specified. FE automatically infers the format.

input0.reshapeType

Optional

Dimension padding method for input0.

If the input data format of the operator is NC1HWC0, but the operator input on the original network is not 4D input, you need to set this parameter to pad the non-4D shape on the original network to 4D shape according to the following rules:

• If the operator input on the original network is 1D, set reshapeType to N, C, H, or W. The input in the original network can be one of the four dimensions, and other three dimensions are padded as 1.

  Example: The shape on the original network is [20], and the original format is NHWC. In this case, set reshapeType to N and pad the shape to [20, 1, 1, 1].

• If the operator input on the original network is 2D and the original format is NCHW, reshapeType can be set to any two dimensions of the original shape, that is, NC, CH, HW, CW, NH, or NW and the other two dimensions are padded as 1. Note that the dimension order of the original format must be retained. Example: reshapeType is set to NC and the original shape is [20,30]. The H and W dimensions are padded as 1, that is, the shape changes to [20,30,1,1]. Note that the dimension order cannot be changed in dimension padding. Therefore, this parameter cannot be set to CH in this example.
• If the operator input on the original network is 3D, the processing principle is similar to 2D input. In addition, the setting of reshapeType also takes effect when the dimension count of reshapeType is greater than or equal to that of the original input.

  Example: The original format is NCHW and reshapeType is set to NCH. If the original input is 1D, the shape after dimension padding is [N,1,1,1]. If the original input is 2D, the shape after dimension padding is [N,C,1,1]. If the original dimension is 3D, the shape after dimension padding is [N,C,H,1].

• If the input shape of the operator on the original network is 4D, the reshapeType settings do not take effect.

If this parameter is not set and the operator input format must be NC1HWC0:

• When the original format is NCHW: if the original shape is 1D, pad the shape to [1,C,1,1]; if the original shape is 2D, [1,C,H,1]; if the original shape has three dimensions, pad the shape to [1,C,H,W].
• When the original format is NHWC: if the original shape is 1D, pad the shape to [1,H,1,1]; if the original shape is 2D, pad the shape to [1,H,W,1]; if the original shape has three dimensions, pad the shape to [1,H,W,C].

input0.shape

Optional

Shapes supported by input0.

• Fixed shape. For example, [56,56,56,56]. If multiple shapes are supported, separate them by semicolons (;), for example, [3,5,6];[4,5,8,9].
• If this parameter is not specified, the default value all is used, indicating that the operator has no shape restrictions.

input1.name

Optional

If the operator has multiple input tensors, add the configurations of input1.xx and input2.xx by referring to the parameter configuration of input0.xx. The inputs are numbered starting at 0 in ascending order.

Name of the input tensor, which must be the same as that in the operator prototype definition.

......

Optional

Configuration for other input parameters of input1. Configure other inputs (input1, input2, input3, ...) of the operator by referring to input0, if any.

attr.list

Optional

List of operator attributes required for operator implementation. Separate multiple attributes by commas (,). The attribute sequence must be consistent with the operator declaration.

The list is used by FE to obtain the corresponding attributes from OpDesc.

Example: stride,padding

Note: This parameter needs to be configured if related attributes are required in the operator implementation. Otherwise, build fails.

attr_key.type

Optional

Type of an attribute. attr is a fixed prefix, and key corresponds to a specific parameter. For example:

attr_stride.type=int indicates that the type attribute of the stride parameter is of the int type and the method of obtaining the operator attribute from OpDesc is GetInt.

This field must correspond to the attributes in attr.list. The number of attributes in attr.list and the number of attr_key.type records to be configured must be the same.

• int: int
• float: float
• bool: bool
• str: string
• listInt: list of integers
• listFloat: list of floats
• listBool: list of bools
• listStr: list of strings

attr_key.value

Optional

Attribute value.

all indicates that all attribute values are supported.

Separate multiple values by commas (,).

Example: 1,3,5

This field is optional and does not affect compilation. It is used for verification during TBE precompilation.

attr_key.paramType

Optional

Whether a parameter is required in OpDesc.

For example, in two different scenarios of quantization and non-quantization, some parameters of the convolution operator may exist or may not exist.

The value can be:

• optional: The parameter is optional. If the value cannot be obtained from OpDesc, no error is reported.
• required: The parameter is required. If the value cannot be obtained from OpDesc, an error is reported.

Defaults to required.

attr_key.defaultValue

Optional

If attr_key.paramType is set to optional and the parameter value fails to be obtained from OpDesc, the default value of this parameter is used.

Note:

This parameter is invalid when attr_key.paramType is set to required. When attr_key.paramType is optional, if the value fails to be obtained from OpDesc and the defaultValue is not defined, a failure message is returned.

output0.name

Required

Name of output0.

If output0.paramType is set to dynamic, set output0.name to y. The name of y must be the same as that in Operator Prototype Definition. When the graph is running, y0, y1, y2, and so on are automatically generated based on the number of inputs. The outputs are numbered starting at 0 in ascending order.

output0.paramType

Optional

Type of output0.

• dynamic: indicates the number of dynamic outputs, which can be one or more. If there are multiple outputs, this parameter must be used together with output0.name.
• optional: indicates that the output is optional.
• required: indicates that only one value can be output.

Defaults to required.

output0.dtype

Optional

Data type of output0.

Note:

• If the output tensor supports multiple dtype and format combinations, dtype and format must be configured in pairs in sequence. List all the combinations supported by the operator and separate them by commas (,).
  In the following example, 12 items are configured for output0.dtype and out0.format respectively.

  output0.dtype=float16,float16,float16,float16,float,float,float,float,int32,int32,int32,int32
  output0.format=NCHW,NC1HWC0,NHWC,ND,NCHW,NC1HWC0,NHWC,ND,NCHW,NC1HWC0,NHWC,ND

• If dynamicFormat.flag is set to true, this parameter does not need to be specified. However, the op_select_format Function function must be implemented in the operator implementation file.

output0.format

Optional

Format of output0.

• If all formats in the source graph are supported, set format to ND.
• Separate multiple formats by commas (,). The options are as follows: NCHW, NHWC, NC1HWC0, FRACTAL, and ND.

Note:

• If the output tensor supports multiple dtype and format combinations, dtype and format must be configured in pairs in sequence. List all the combinations supported by the operator and separate them by commas (,).
• If dynamicFormat.flag is set to true, this parameter does not need to be specified. However, the op_select_format Function function must be implemented in the operator implementation file.
• If op.pattern is set, this parameter does not need to be specified. FE automatically infers the format.

output0.shape

Optional

Shape supports by the first output (output0)

• Fixed shape. For example, [56,56,56,56]. If multiple shapes are supported, separate them by semicolons (;), for example, [3,5,6];[4,5,8,9].
• If this parameter is not specified, the default value all is used, indicating that the operator has no shape restrictions.

output1.name

Optional

If the operator has multiple output tensors, add the configurations of output1.xx by referring to the parameter configuration of output0.xx. The sequence numbers increase from output1 and output2, respectively.

opFile.value

Optional

Name of the operator implementation file. FE searches for the operator implementation file based on the file name.

If this parameter is not specified, uppercase letters in the name are converted into underscores (_) based on the OpType field to match the operator implementation file name. For details about the mapping rule, see 1.

opInterface.value

Optional

Operator implementation API name. FE calls the operator based on the API name.

If this parameter is not specified, uppercase letters in the name are converted into underscores (_) based on the OpType field to match the operator API name. For details about the mapping rule, see 1.

op.pattern

Optional

Defines whether an operator belongs to the broadcast, reduce, or formatAgnostic class.

• If dynamicFormat.flag is set to true, this parameter does not need to be specified. The op_select_format Function API in the operator implementation file is preferentially called to obtain the input and output dtype and format supported by the operator.
• If dynamicFormat.flag is set to false and op.pattern is set to any of the following values, there is no need to specify the input and output formats supported by the operator.
  • broadcast: For an operator supporting broadcast, FE matches the operator shape from a group of formats based on the shape of the operator.
  • reduce: For a reduce operator, FE matches the operator shape from a group of formats based on the shape, keep_dims, and axis settings of the operator.
  • formatAgnostic: For a format-independent operator, that is, the operator is a single-input, single-output operator and supports any format (for example, the Cast operator), you only need to set op.pattern to formatAgnostic. There is no need to enumerate all the input and output formats supported by the operator.

dynamicFormat.flag

Optional

Defaults to false.

If this parameter is set to true, the input and output dtype and format configured for the operator are ignored. The op_select_format Function API in the operator implementation file is called to obtain the input and output dtype and format supported by the operator.

precision_reduce.flag

Optional

Controls the operator precision mode during ATC model conversion or network commissioning. This parameter is valid only when precision_mode is set to allow_mix_precision.

• If set to false, the operator is in the blocklist. In this case, the original data type of the operator must be retained.
• If set to true, the operator is in the allowlist. If the operator supports both float32 and float16 and the original image format of the operator is float32 or float16, float16 is preferentially selected.
• If not set, the operator is in the greylist. In this case, if this operator has a previous operator, the data type of the previous operator is selected. Otherwise, the original data type of the current operator is selected.

heavyOp

Optional

If set to true, heavy format inheritance happens to the operator to reduce the insertion of the format conversion operator and improve the network running efficiency. This parameter is mainly used for Cube operators.

According to the inheritance logic, if the input or output format of an operator is one of the following heavy formats:

NC1HWC0, C1HWNCoC0, FRACTAL_Z, FRACTAL_NZ, NDC1HWC0, FRACTAL_Z_3D, FRACTAL_Z_3D_TRANSPOSE

recursively search for the connected operator of the input or output, and infer the format of the found operator to the selected heavy format. Stop format inference in the following cases:

• The connected operator does not support the heavy format of the current operator.
• The connected operator has the same heavy format as the current operator.
• The heavyOp field of the connected operator is set to true.

Defaults to false.