Implementation

Configuring the Custom Operator .json File

The .json file of a custom operator describes the operator in the aspects of it inputs, outputs, and attributes. Configure the operator .json file by referring to Operator Prototype Definition. The following is an example.

[
  {
    "op": "Conv2D",
    "input_desc": [
      {
        "format": "NCHW",
        "shape": [8, 512, 7, 7],
        "type": "float16"
      },
      {
        "format": "NCHW",
        "shape": [512, 512, 3, 3],
        "type": "float16"
      }
    ],
    "output_desc": [
      {
        "format": "NCHW",
        "shape": [8, 512, 7, 7],
        "type": "float16"
      }
    ],
    "attr": [
      {
        "name": "strides",
        "type": "list_int",
        "value": [1, 1, 1, 1]
      },
      {
        "name": "pads",
        "type": "list_int",
        "value": [1, 1, 1, 1]
      },
      {
        "name": "dilations",
        "type": "list_int",
        "value": [1, 1, 1, 1]
      }
    ]
  }
]

The .json file is an OpDesc array. Its parameters are described as follows.

Parameters of the TensorDesc array are described as follows.

Parameters of the Attr array are described as follows:

Generating a Single-Operator Model File Using ATC

1. Set the environment variables.

   The following is an example of setting environment variables. For details about the environment variables, see "Conversion Example" in ATC Tool Instructions.

   export install_path=/home/HwHiAiUser/Ascend/ascend-toolkit/latest
   export PATH=${install_path}/atc/ccec_compiler/bin:${install_path}/atc/bin:$PATH
   export PYTHONPATH=${install_path}/atc/python/site-packages:${install_path}/atc/python/site-packages/auto_tune.egg/auto_tune:${install_path}/atc/python/site-packages/schedule_search.egg:$PYTHONPATH
   export LD_LIBRARY_PATH=${install_path}/atc/lib64:$LD_LIBRARY_PATH
   export ASCEND_OPP_PATH=${install_path}/opp

   install_path: installation path of the ATC and OPP components.

2. Use the ATC tool to load the single-operator description file (.json file) to generate a single-operator offline model.

   atc --singleop=test_data/config/xxx.json --soc_version={soc_version} --output=op_models

   • singleop: specifies the operator description file (.json), which is the relative path where the atc command is executed.
   • soc_version: version of the Ascend AI Processor. Replace it with the actual version.

     You can view the supported Ascend AI Processor version in the atc/data/platform_config in the ATC installation directory. The name of the .ini file is value of soc_version.

   • output: specifies the path for storing the generated model file, which is the relative path where the atc command is executed.

   For details about other parameters of the ATC tool, see "Conversion Example" in ATC Tool Instructions.

Constructing the Input Data

Before implementing the AscendCL code for single-operator verification, you need to create a binary file of the operator test data and name it input_x.bin, where x is an integer starting at 0, for example, input_0.bin and input_1.bin.

Note: You can generate test data by using the random function of NumPy and export the data as a binary file.

Implementing AscendCL Single-Operator Verification on Network

1. Construct an operator description object.

   ACLlib provides a single-operator verification sample, which requires only few manual tweaks.

   Modify the CreateOpDesc() function in the src/main.cpp file to construct an operator description object.

   • The following is an example of constructing opDesc of the Add operator without attributes:

     OperatorDesc CreateOpDesc()
     {
         std::string opType = "Add";          // Operator type
         // Construct the input and output description of the operator.
         std::vector<int64_t> shape{8, 16};
         aclDataType dataType = ACL_INT32;
         aclFormat format = ACL_FORMAT_ND;
         OperatorDesc opDesc(opType);
         opDesc.AddInputTensorDesc(dataType, shape.size(), shape.data(), format);
         opDesc.AddInputTensorDesc(dataType, shape.size(), shape.data(), format);
         opDesc.AddOutputTensorDesc(dataType, shape.size(), shape.data(), format);
         return opDesc;
     }

   • The following is an example of constructing opDesc of the ConcatD operator with attributes:

     OperatorDesc CreateOpDesc()
     {
         std::string opType = "ConcatD";
         std::vector<int64_t> shape1{1, 1, 4, 4};
         std::vector<int64_t> shape2{1, 2, 4, 4};
         aclDataType dataType = ACL_INT32;
         aclFormat format = ACL_FORMAT_ND;
         OperatorDesc opDesc(opType);
         opDesc.AddInputTensorDesc(dataType, shape1.size(), shape1.data(), format);
         opDesc.AddInputTensorDesc(dataType, shape1.size(), shape1.data(), format);
         opDesc.AddOutputTensorDesc(dataType, shape2.size(), shape2.data(), format);
         // Construct operator attributes.
         auto opAttr = opDesc.opAttr;
         aclopSetAttrInt(opAttr, "N", 2);
         aclopSetAttrInt(opAttr, "concat_dim", 1);
         return opDesc;
     }

   • The following is an example of constructing opDesc of the Conv2D operator:

     OperatorDesc CreateOpDesc()
     {
         std::vector<int64_t> shape{8, 512, 7, 7};
         std::string opType = "Conv2D";
         aclDataType dataType = ACL_FLOAT16;
         aclFormat format = ACL_FORMAT_NCHW;
         OperatorDesc opDesc(opType);
         opDesc.AddInputTensorDesc(dataType, shape.size(), shape.data(), format); // The first input of the Conv2D operator
         std::vector<int64_t> shape1{512,512,3,3};  
         opDesc.AddInputTensorDesc(dataType, shape1.size(), shape1.data(), format);   // The second input of the Conv2D operator
         std::vector<int64_t> shape2{8, 512, 7, 7};
         opDesc.AddOutputTensorDesc(dataType, shape2.size(), shape2.data(), format);  // Output of the Conv2D operator
         int64_t intList[4]{1, 1, 1, 1};
         auto opAttr = opDesc.opAttr;
         aclopSetAttrListInt(opAttr, "strides", 4, intList);
         aclopSetAttrListInt(opAttr, "pads", 4, intList);
         aclopSetAttrListInt(opAttr, "dilations", 4, intList);
         return opDesc;
     }

   OperatorDesc is the operator description object constructed in acl_execute_add/inc/operator_desc.h.

   • std::string opType: operator type
   • std::vector<const aclTensorDesc *> inputDesc: input description of the operator
   • std::vector<const aclTensorDesc *> outputDesc: output description of the operator
   • aclopAttr *opAttr: operator attribute

     Call the aclopSetAttr**() API to construct attributes. Different types of attributes call different aclopSetAttr* APIs. For details, see APIs of the aclopAttr class in AscendCL API Reference > Data Types and Operations of the Application Software Development Guide.

   • OperatorDesc &AddInputTensorDesc(aclDataType dataType, int numDims, const int64_t *dims, aclFormat format)

     Constructs the operator input. If the operator has multiple inputs, state each input using a separate opDesc.AddInputTensorDesc() statement.

   • OperatorDesc &AddOutputTensorDesc(aclDataType dataType, int numDims, const int64_t *dims, aclFormat format)

     Constructs the operator output. If the operator has multiple outputs, state each input using a separate opDesc.AddOutputTensorDesc() statement.

     For details about the value ranges of aclDataType and aclFormat, see AscendCL API Reference > Data Types and Operations of the Application Software Development Guide.

2. Load and run the single-operator model.

   The following key APIs are involved:

   Call aclInit to initialize the AscendCL and call aclrtSetDevice to specify the compute device.

   Call aclopSetModelDir to set the directory for loading the model file. The single-operator model file (.om file) is stored in the directory.

   Call aclrtMalloc to allocate memory on the device to store the input and output data of the operator.

   Allocate memory for storing the input and output data of the operator based on the constructed operator description information (including the input and output tensor description and operator attributes), and call aclopExecute to load and execute the operator.

   Call aclrtSynchronizeStream to block application execution until all tasks in the specified stream are complete.

   Call aclrtDestroyStream to release the memory.

   Call aclrtResetDevice to release the resources on the device.

   The acl_execute_add/src/main.cpp file provides the common process sample code for loading and executing a single-operator model. You can use the sample code to load your single-operator model file and operator description object to verify the single-operator function.