Single-Operator JSON File Configuration

CANN V100R020C10 Development Auxiliary Tool Guide (Inference) 01

Huawei uses machine translation combined with human proofreading to translate this document to different languages in order to help you better understand the content of this document. Note: Even the most advanced machine translation cannot match the quality of professional translators. Huawei shall not bear any responsibility for translation accuracy and it is recommended that you refer to the English document (a link for which has been provided).

Configuration File Templates
Parameters

Configuration File Templates

A configuration file can define multiple sets of single-operator JSON configurations, each including the operator type, operator input and output information, and optional attributes. The following provides some configuration code examples. You can modify the examples as required.

Single-Operator JSON File Template

If format = ND:

[
{
  "op": "GEMM",
  "input_desc": [
    {
      "format": "ND",
      "shape": [16, 16],
      "type": "float16"
    },
    {
      "format": "ND",
      "shape": [16, 16],
      "type": "float16"
    },
    {
      "format": "ND",
      "shape": [16, 16],
      "type": "float16"
    },
    {
      "format": "ND",
      "shape": [],
      "type": "float16"
    },
    {
      "format": "ND",
      "shape": [],
      "type": "float16"
    }
  ],
  "output_desc": [
    {
      "format": "ND",
      "shape": [16, 16],
      "type": "float16"
    }
  ],
  "attr": [
  {
    "name": "transpose_a",
    "type": "bool",
    "value": false
  },
  {
    "name": "transpose_b",
    "type": "bool",
    "value": false
    }
  ]
}
]

If format = NCHW:

[
 {
   "op": "Conv2D",  
   "input_desc": [     
     {        
       "format": "NCHW",      
       "shape": [1, 3, 16, 16],     
       "type": "float16"     
     },      
     {       
       "format": "NCHW",       
       "shape": [3, 3, 3, 3],        
       "type": "float16"       
     }     
   ],    
   "output_desc": [     
     {     
       "format": "NCHW",        
       "shape": [1, 3, 16, 16],     
       "type": "float16"      
     }    
   ],   
   "attr": [      
       {       
         "name": "strides",       
         "type": "list_int",       
         "value": [1, 1, 1, 1]    
       },     
       {       
         "name": "pads",      
         "type": "list_int",      
         "value": [1, 1, 1, 1]     
       },      
       {        
         "name": "dilations",      
         "type": "list_int",     
         "value": [1, 1, 1, 1]    
       }  
   ]  
 } 
 ]

Multi-Operator JSON File Template

If multiple groups of operators are configured in the JSON file, multiple .om files are generated after model conversion and are displayed in sequence: 0_xx, 1_xx, and so on. The following configuration file is only an example. Modify it as required.

[
  {
    "op": "MatMul",
    "input_desc": [
      {
        "format": "ND",
        "shape": [
          16,
          16
        ],
        "type": "float16"
      },
      ...
    ],
    "output_desc": [
      {
        "format": "ND",
        "shape": [
          16,
          16
        ],
        "type": "float16"
      }
    ],
    "attr": [
      {
        "name": "alpha",
        "type": "float",
        "value": 1.0
      },
     ...
    ]
  },
  {
    "op": "MatMul",
    "input_desc": [
      {
        "format": "ND",
        "shape": [
          256,
          256
        ],
        "type": "float16"
      },
     ...
    ],
    "output_desc": [
      {
        "format": "ND",
        "shape": [
          256,
          256
        ],
        "type": "float16"
      }
    ],
    "attr": [
      {
        "name": "alpha",
        "type": "float",
        "value": 1.0
      },
      ...
    ]
  }
]

Dynamic-Shape Single-Operator JSON File Template

In this scenario, single-operators come in the following types:

The shape is not specified during model build. During model execution, the output shape can be inferred based on the input shape.

[
  {
    "op": "Add",
    "input_desc": [
      {
        "format": "ND",
        "shape": [-1,16],
        "shape_range": [[0, 32]],
        "type": "int64"
      },
      {
        "format": "ND",
        "shape": [-1,16],
        "shape_range": [[0, 32]],
        "type": "int64"
      }
    ],
    "output_desc": [
      {
        "format": "ND",
        "shape": [-1,16],
        "shape_range": [[0, 32]],
        "type": "int64"
      }
    ]
  }
]

The shape is not specified during model build. During model execution, the output shape can be inferred based on the input shape and constant.

[
  {
    "op": "TopK",
    "input_desc": [
      {
        "format": "ND",
        "shape": [-1],
        "shape_range": [[1,-1]],
        "type": "int32"
      },
      {
        "format": "ND",
        "shape": [],
        "type": "int32"
      }
    ],
    "output_desc": [
      {
        "format": "ND",
        "shape": [-1],
        "shape_range": [[1,-1]],
        "type": "int32"
      },
	  {
        "format": "ND",
        "shape": [-1],
        "shape_range": [[1,-1]],
        "type": "int32"
      }],
      "attr": [
      {
        "name": "sorted",
        "type": "bool",
        "value": true
      }
    ]
  }
]

The shape is not specified during model build. During model execution, the output shape cannot be inferred based on the input shape, but the output shape range can be obtained.

[
  {
    "op": "Where",
    "input_desc": [
      {
        "format": "ND",
        "shape": [-1],
        "shape_range": [[1,-1]],
        "type": "int32"
      }
    ],
    "output_desc": [
      {
        "format": "ND",
        "shape": [-1, 1],
        "shape_range": [[1,-1]],
        "type": "int64"
      }
    ]
  }
]

Parameters

The .json file consists of OpDesc arrays. The parameters are described as follows:

Table 2-11 OpDesc parameters

Attribute Name	Type	Description	Required or Not
op	string	Operator type	Yes
input_desc	TensorDesc array	Operator input description	Yes
output_desc	TensorDesc array	Operator output description	Yes
attr	Attr array	Operator attributes	Not

Table 2-12 TensorDesc array parameters

Attribute Name	Type	Description
format	string	(Required) Tensor format, a format supported by the original operator framework. Currently, the supported formats are as follows: NCHW NHWC ND: any format NC1HWC0: the 5D format defined by Huawei C0 is closely related to the micro-architecture, and the value is equal to the Cube Unit size, for example, 16. C1 is obtained by dividing the C dimension by C0, that is, C1 = (C + C0 – 1)/C0. When the division is not exact, the result is rounded up. FRACTAL_Z: format of the convolution weight FRACTAL_NZ: fractal format defined by Huawei. The data format of the output matrix is NW1H1H0W0 during Cube unit computation. The matrix is divided into (H1 x W1) fractals in column-major order, which looks like an N-shape layout. Each fractal consists of (H0 x W0) elements in row-major order, resembling a Z-shaped layout. Thus the NW1H1H0W0 data format is referred to as an Nz format. (H0 x W0) indicates the size of a fractal, as shown in the following figure.
type	string	(Required) Tensor data type. The supported types are as follows: bool int8 uint8 int16 uint16 int32 int64 uint32 uint64 float16 float
shape	Array of ints	(Required) Tensor shape, for example, [1, 224, 224, 3] The actual shape size (product of dimension sizes) cannot exceed the maximum int32 value, 214748364.
name	string	(Optional) Tensor name If the operator input is dynamic, this field is required. For example, if the dynamic input definition of the ConcatD operator in the operator prototype is DYNAMIC_INPUT(input_values, ...), set this attribute to "input_values0", "input_values1", ...
shape_range	Array of int[2]s	(Optional) Value range of an unknown dimension when the shape is dynamic. For example, for shape [16, –1, 20, –1], –1 indicates an unknown dimension. Assume that shape_range is [1,128], [1,-1], where, [1,128] indicates the value range from 1 to 128, corresponding to the value range of the first –1 dimension, [1,-1] indicates the value range from 1 to infinity, corresponding to the value range of the second –1 dimension.

Table 2-13 Attr array parameters

Attribute Name	Type	Description
name	string	(Required) Attribute name
type	string	(Required) Attribute data type: bool int float string list_bool list_int list_float list_string list_list_int
value	Determined by the value of type	(Required) Attribute value. Varies according to type. bool: true/false int: 10 float: 1.0 string: "NCHW" list_bool: [false, true] list_int: [1, 224, 224, 3] list_float: [1.0, 0.0] list_string: ["str1", "str2"] list_list_int: [[1, 3, 5, 7], [2, 4, 6, 8]]

Attribute Name

Type

Description

name

string

(Required) Attribute name

type

string

(Required) Attribute data type:

bool
int
float
string
list_bool
list_int
list_float
list_string
list_list_int

value

Determined by the value of type

(Required) Attribute value. Varies according to type.

bool: true/false
int: 10
float: 1.0
string: "NCHW"
list_bool: [false, true]
list_int: [1, 224, 224, 3]
list_float: [1.0, 0.0]
list_string: ["str1", "str2"]
list_list_int: [[1, 3, 5, 7], [2, 4, 6, 8]]

Configuration File Templates
Parameters

Translation

简体中文

Download

Updated: 2020-12-11

Document ID: EDOC1100164848

Downloads: 27

Average rating:

This Document Applies to these Products

Configuration File Templates

Single-Operator JSON File Template

Multi-Operator JSON File Template

Dynamic-Shape Single-Operator JSON File Template

Parameters

Related Version

Related Documents