Operator Development in Command Line

Operator analysis

Before developing an operator, you need to analyze the operator, specify the operator function, inputs, and outputs, select an operator development mode, and name the operator type and operator implementation function.

Operator Analysis

Project creation

In the CLI, you can create a custom operator project by using one of the following methods:

• Develop based on existing sample project.
  • Develop a TensorFlow custom operator.

    Modify the custom operator based on the sample project and add your own custom operator.

    The TensorFlow custom operator sample project is stored in the atc/sample/op/tensorflow directory in the ATC installation path.

  • Develop a Caffe custom operator.

    Modify the custom operator based on the sample project and add your own custom operator.

    The Caffe custom operator sample project is stored in atc/sample/op/caffe in the ATC installation path.

  • Develop a Caffe custom operator and a TensorFlow custom operator at a time.

    Modify the custom operator based on the sample project and add your own custom operator.

    The Caffe and TensorFlow custom operator sample projects are stored in the atc/sample/op/all directory under the ATC installation path.

• Use the Operator Deliverable Generator to generate custom operator project, or use this tool to add custom operator to an existing TensorFlow custom operator project.

Develop and store operator deliverables according to the following rules:

• tbe/impl: stores the operator implementation file (.py).
• tbe/op_info_cfg/ai_core/<SoC_version>/: stores the operator information definition file (.ini) in the directory of corresponding Ascend AI Processor version.
• framework/xx_plugin: stores the operator adaptation plug-in file (.cpp).
• op_proto: stores the prototype definition files (.h and .cpp).

Operator Project Structure

Operator implementation

Operator code implementation: implements operator compute logic and scheduling

Operator Code Implementation

Operator plug-in implementation: required in custom operator development on a third-party framework (such as TensorFlow). With the custom operator implementation code delivered, you also need to develop a plug-in to map the TensorFlow operator to an operator that adapts to the Ascend AI Processor.

Operator Plug-in Implementation

Operator prototype definition: defines the constraints on the operator to run on the Ascend AI Processor, mainly the mathematical meanings of the operator by defining operator inputs, outputs, attributes, and their value ranges, verifying arguments, and inferring shape. The information defined by the prototype is registered with the operator prototype library of GE. During offline model conversion, GE calls the verification API of the operator prototype library to verify operator arguments. If the verification passes, GE infers the output shape and dtype of each node by calling the inference function of the operator prototype library and allocates static memory for the result tensor.

Operator Prototype Definition

Operator information definition: registers the operator information to the operator information library, including the input and output dtype and format, and input shape of the operator. During offline model conversion, based on the operator information in the operator information library, FE performs basic verification, inserts conversion nodes for the operator as required, and finds the operator implementation code to build the operator binary file.

Operator Information Library Definition

Operator build and deployment

• Operator build: builds the operator plug-in implementation file, prototype definition file, and information definition file into the operator plug-in library, operator prototype library, and operator information library, respectively.
• Operator deployment: deploys the operator implementation file, plug-in library, prototype library, and information library to the operator library, that is, a corresponding directory in the opp directory.

In the CLI, you can use the build file of the sample project for one-click compilation, generating a custom operator installation package. Specify the opp path and execute the installation package to deploy the custom operator.

Operator Project Building

Verification by network execution

Constructs a model file that contains only a single operator, and uses the AscendCL to load the model file to verify the functions of the single operator.

Operator Verification on Network