Principles

The operator prototype definition specifies the constraints on an operator that runs on the Ascend AI Processor, mainly reflecting the mathematical meanings of the operator by defining inputs, outputs, attributes, and their value ranges, verifying arguments, and inferring the shape. The information defined by the prototype is registered with the operator prototype library of GE. To generate a model, GE calls the verification API of the operator prototype library to verify operator arguments. If the verification passes, GE infers the output shape and dtype of each node by calling the inference function of the operator prototype library and allocates static memory for the result tensor.

Figure 8-1 shows the role of the operator prototype library in the model generation workflow.

Figure 8-1 IR registration workflow with GE

Operator registration includes the registration of the operator, InferShape function, and Verify function. During registration, the operator type (OpType) is used as the key.

1. GE receives and initializes the source network trained under a third-party framework. The topology of the network is called a graph.
2. The operator prototype library management module loads the operator prototype library (an .so file) from the opp/op_proto/ directory in the OPP installation path on the host.
3. The operator prototype library management module registers the operator information in OperatorFactory based on the information in the .so file, including the registration of the operator, InferShape function, and Verify function. The three parts are registered with three map files by using the operator type (OpType) as the key.
4. In the graph preparation phase, GE sends the graph a request for calling InferShape.
5. The graph traverses the Infer methods of all nodes and sends a request for calling the Infer method to the OpDesc.
6. The OpDesc obtains the InferShape and Verify functions from the OperatorFactory based on OpType.
7. The Verify function is executed for verification. If the verification is successful, go to the next step. If the verification fails, return to the previous step.
8. The InferShape function is executed to derive the shape of the output tensor.
9. The InferShape result to GE, which allocates the static memory for the output tensor based on the InferShape result.
10. GE performs other operations.