Operator Information Library Definition
Principles
The operator information library, one of the deliverables for operator development, mainly describes the implementation specifications of operators in the Ascend AI Processor, including the input and output dtype, format, and input shape supported by operators. During network execution, FE performs basic verification based on the operator information in the operator information library (including information such as dtype and format), and locates the corresponding operator implementation file according to the preceding information, and builds the implementation file to generate the binary file of the operator.
Configuration Description
You need to configure the operator information library file to register the operator implementation information in the Ascend AI Processor to the operator information library file.
Path of the operator information library file: /tbe/op_info_cfg/ai_core/<soc_version>/xx.ini under the custom operator project directory
For details about the configuration rules, see Table 9-1.
Configuration Item |
Required/Optional |
Description |
[OpType] |
Required |
Operator type, which defines the start of operator information and is the same as the value of OpType in REG_OP(OpType) in Operator Prototype Definition. For details about the OpType naming rules, see Naming Rules for Operator Definitions. |
input0.name |
Required |
Name of input0, same as that in Operator Prototype Definition. If input0.paramType is set to dynamic, set input0.name to x. The name of x must be the same as that in Operator Prototype Definition. During graph execution, x0, x1, x2, and more are automatically generated based on the number of inputs. The inputs are numbered starting at 0 in ascending order. |
input0.paramType |
Required |
Type of input0.
Defaults to required. |
input0.dtype |
Optional |
Data type (or data types separated by commas) supported by input0. Value range: float16, float, int8, int16, int32, uint8, uint16, uint32, bool, and more Notes:
|
input0.format |
Optional |
Format (or formats separated by commas) of input0.
Notes:
|
input0.reshapeType |
Optional |
Reshape method for input0. If the original operator input has dimensions lower than 4, but there is an input format requirement of NC1HWC0, set this parameter to reshape the input to 4D according to the following rules:
If this parameter is not set but there is an input format requirement of NC1HWC0:
|
input1.name |
Optional |
If the operator has multiple input tensors, add the configurations of input1.xx and input2.xx by referring to the parameter configuration of input0.xx. The inputs are numbered starting at 0 in ascending order. Name of the input tensor, same as that in the operator prototype definition. |
...... |
Optional |
Configuration for other input parameters of input1. Configure other inputs (input1, input2, input3, ...) of the operator by referring to input0, if any. |
attr.list |
Optional |
List of operator attributes required for operator implementation. Separate multiple attributes by commas (,). The attribute sequence must be consistent with the operator declaration. The list is used by FE to obtain the corresponding attributes from OpDesc. Example: stride,padding Note: This parameter needs to be configured if related attributes are required in the operator implementation. Otherwise, build fails. |
attr_key.type |
- |
The parameter is required if attr.list is set. Otherwise, the build fails. Type of an attribute. attr is a fixed prefix, and key corresponds to a specific parameter. For example: attr_stride.type=int indicates that the type attribute of the stride parameter is of the int type and the method of obtaining the operator attribute from OpDesc is GetInt. This field must correspond to the attributes in attr.list. The number of attributes in attr.list and the number of attr_key.type records to be configured must be the same.
|
attr_key.value |
- |
The parameter is required if attr.list is set. Otherwise, the build fails. Value range of the attribute.
|
attr_key.paramType |
Optional |
Determines whether a parameter is required in OpDesc. For example, in the quantization and non-quantization scenarios, some parameters of the convolution operator may exist or may not exist. The value can be:
Defaults to required. |
attr_key.defaultValue |
Optional |
If attr_key.paramType is set to optional and the parameter value fails to be obtained from OpDesc, the default value of this parameter is used. Notes:
|
output0.name |
Required |
Name of output0. If output0.paramType is set to dynamic, set output0.name to y. The name of y must be the same as that in Operator Prototype Definition. During graph execution, y0, y1, y2, and more are automatically generated based on the number of inputs. The outputs are numbered starting at 0 in ascending order. |
output0.paramType |
Optional |
Type of output0.
Defaults to required. |
output0.dtype |
Optional |
Data type (or data types separated by commas) of output0. Notes:
|
output0.format |
Optional |
Format (or formats separated by commas) of output0.
Notes:
|
output1.name |
Optional |
If the operator has multiple output tensors, add the configurations of output1.xx by referring to the parameter configuration of output0.xx. The sequence numbers increase from output1 and output2, respectively. |
opFile.value |
Optional |
Name of the operator implementation file. FE locates the operator implementation file based on the file name. If this parameter is not specified, uppercase letters in the name are converted into underscores (_) based on the OpType field to match the operator implementation file name. For details about the mapping rule, see Naming Rules for Operator Definitions. |
opInterface.value |
Optional |
Operator implementation API name. FE calls an operator based on the API name. If this parameter is not specified, uppercase letters in the name are converted into underscores (_) based on the OpType field to match the operator API name. For details about the mapping rule, see Naming Rules for Operator Definitions. |
op.pattern |
Optional |
Defines whether an operator belongs to the broadcast, reduce, or formatAgnostic type.
|
dynamicFormat.flag |
Optional |
Defaults to false. If this parameter is set to true, the input and output dtype and format configured for the operator are ignored. The Function op_select_format API in the operator implementation file is called to obtain the input and output dtype and format supported by the operator. |
precision_reduce.flag |
Optional |
Operator precision mode during ATC model conversion or network debugging. Valid only when precision_mode is set to allow_mix_precision.
|
heavyOp |
Optional |
If set to true, heavy format inheritance happens to the operator to reduce the insertion of the format conversion operator and improve the network execution efficiency. This parameter is mainly used for Cube operators. According to the inheritance logic, if the input or output format of an operator is one of the following heavy formats: NC1HWC0, C1HWNCoC0, FRACTAL_Z, FRACTAL_NZ, NDC1HWC0, FRACTAL_Z_3D, FRACTAL_Z_3D_TRANSPOSE Recursively search for the connected operator of the input or output, and infer the format of the found operator to the selected heavy format. Stop format inference in the following cases:
Defaults to false. |
needCheckSupport.flag |
Optional |
Whether to call the check_supported API in the operator implementation file to verify the data type and shape in the operator fusion phase.
Defaults to false. |
After the custom operator information library file is built and deployed, the operator definition information is stored in the operator information library of the corresponding version of the Ascend AI Processor. The default storage directory is opp/op_impl/custom/ai_core/tbe/config/<SoC_version>/aic-<SoC_version>-ops-info.json in the OPP installation path.
Naming Rules for Operator Definitions
- The operator type must be named in upper camel case to distinguish different semantics.
- You can name the operator file and operator function in either of the following naming rules:
- To create user-defined names, configure the opFile.value and opInterface.value in Operator Information Definition.
- If opFile.value and opInterface.value in the Operator Information Definition are not configured, the FE converts OpType and matches the operator file name with the operator function name as follows:The conversion rules are as follows:
- Convert the first uppercase letter to a lowercase letter.
Example: Abc -> abc
- Convert the uppercase letter after a lowercase letter to a lowercase letter with an underscore (_) prefix.
Example: AbcDef -> abc_def
- Uppercase letters following a digit or an uppercase letter are regarded as a semantic string. If there is a lowercase letter after this string, convert the last uppercase letter in this string into an underscore (_) and a lowercase letter, and convert the other uppercase letters into lowercase letters. If there is no lowercase letter after the string, directly convert the string into lowercase letters.
Examples: ABCDef -> abc_def; Abc2DEf -> abc2d_ef; Abc2DEF -> abc2def; ABC2dEF -> abc2d_ef
- Convert the first uppercase letter to a lowercase letter.
Function op_select_format
If dynamicFormat.flag is set to true, you do not need to configure the dtype and format supported by the operator input and output in the operator information library file. Instead, you need to implement the op_select_format function in the operator implementation file (.py) to infer the dtype and format supported by the operator input and output.
The op_select_format function is defined as follows:
def op_select_format(input, output, attr, kernel_name="xx"):
The arguments (including the operator input, output, attributes, and kernel name) to the op_select_format function call must be consistent with those of the operator API call. The op_select_format function returns a string containing the supported input and output formats and data types of the operator.
{
"input0": {
"name": "x",
"dtype": "float16,float16,int8,int8",
"format": "NC1HWC0_C04,NC1HWC0,NC1HWC0_C04,NC1HWC0"
},
"input1": {
"name": "y",
"dtype": "float16,float16,int8,int8",
"format": "FRACTAL_Z_C04,FRACTAL_Z,FRACTAL_Z_C04,FRACTAL_Z"
},
"output0": {
"name": "z",
"dtype": "float16,float16,int32,int32",
"format": "NC1HWC0,NC1HWC0,NC1HWC0,NC1HWC0"
}
}
An implementation sample is provided in Function op_select_format.
Function check_supported
If needCheckSupport.flag is set to true, the check_supported API in the operator implementation file is called to verify related information in the operator fusion phase.
The check_supported function is declared as follows:
def check_supported(input_x1, input_x2, output_y, attribute1=None, attribute2=None,..., kernel_name="xx"):
Keep the arguments passed to the check_supported call consistent with those passed to the operator API call (in terms of the operator input, output, attributes, and kernel name).
Returns True if the verification is passed; otherwise, False.
An implementation sample is provided in Function check_supported.
