Principles

The operator prototype definition specifies the constraints on operators that can run on the Ascend AI Processor, mainly reflecting the mathematical meanings of operators. The constraints include defining the input, output, attributes, and value ranges of operators, verifying basic parameters, and shape deduction. The information defined by the prototype is registered with the operator prototype library of GE. During network running, GE calls the verification API of the operator prototype library to verify operator arguments. If the verification passes, GE infers the output shape and dtype of each node by calling the inference function of the operator prototype library and allocates static memory for the result tensor.

Figure 7-1 shows the implementation process of the operator prototype library in the entire network running scenario.

Figure 7-1 IR registration process on GE

Operator registration includes OP registration, InferShape registration, and Verify registration. During registration, the operator type (OpType) is used as the key.

1. GE receives and initializes the source network model of the third-party framework. The topology of a network model is called graph.
2. The operator prototype library management module loads the .so file of the operator prototype library from the opp/op_proto/ directory in the OPP installation path on the host.
3. The operator prototype library management module registers the operator information in OperatorFactory based on the information in the .so file, including operator registration, InferShape registration, and Verify registration. The three parts are registered with the three map files by using the operator type (OpType) as the key.
4. In the graph preparation phase, GE sends the graph a request for calling InferShape.
5. The graph traverses the Infer methods of all nodes and sends a request for calling the Infer method to the OpDesc.
6. The OpDesc obtains the InferShape and Verify functions from the OperatorFactory based on OpType.
7. The Verify function is executed for verification. If the verification is successful, go to the next step. If the verification fails, return to the previous step.
8. The InferShape function is executed to derive the shape of the output tensor.
9. The InferShape result to GE, which allocates the static memory for the output tensor based on the InferShape result.
10. GE performs other operations.