Verifying the Execution of the MatmulTik Operator

Verify the execution of a single-operator in the standard form as follows:

1. Log in to the development environment as the HwHiAiUser user (running user) and go to the acllib/sample/acl_execute_op/acl_execute_matmul directory.
2. Generate the single-operator offline model file of the MatmulTik operator.
   1. Set the environment variables.
      The following is an example of setting environment variables. For details about the environment variables, see "Conversion Example" in ATC Tool Instructions.

      export install_path=/home/HwHiAiUser/Ascend/ascend-toolkit/latest
      export PATH=${install_path}/atc/ccec_compiler/bin:${install_path}/atc/bin:$PATH
      export PYTHONPATH=${install_path}/atc/python/site-packages:${install_path}/atc/python/site-packages/auto_tune.egg/auto_tune:${install_path}/atc/python/site-packages/schedule_search.egg:$PYTHONPATH
      export LD_LIBRARY_PATH=${install_path}/atc/lib64:$LD_LIBRARY_PATH
      export ASCEND_OPP_PATH=${install_path}/opp

      install_path: installation path of the ATC and OPP components.

   2. Go to the acl_execute_matmul/run/out directory of the sample project and run the following command in the out directory to generate a single-operator model file:

      atc --singleop=test_data/config/matmul_tik_op.json --soc_version={soc_version} --output=op_models

      Specifically,

      • singleop: indicates the operator description file (.json).
      • soc_version: version of the Ascend AI Processor. Replace it with the actual version.

        You can view the supported Ascend AI Processor version in the atc/data/platform_config in the ATC installation directory. The name of the .ini file is value of soc_version.

      • --output=op_models: indicates that the generated model file is stored in the op_models folder in the current directory.
      After the model conversion is successful, the following files are generated:

      • The single-operator model file 0_MatmulTik_1_2_16_64_1_2_64_1024_1_2_16_1024.om is generated in the op_models directory of the current directory. The file is named in the format of No.+opType+input description (dateType_format_shape)+output description.

        For details about the mapping between data types, formats, and shapes, see aclDataType and aclFormat in "AscendCL API Reference" in Application Software Development Guide.

      Note: During model conversion, operators in the custom operator library are preferentially searched to match the operators in the model file.

3. Generate test data.

   Go to the run/out/test_data/data directory of the sample project and run the following command:

   python3.7.5 generate_datatik.py

   The data files input_0.bin and input_1.bin (that has the float16 data type and the shapes of (16,64) and (64,1024), respectively) are generated in the current directory. The two files are used for verifying the MatmulTik operator and generating the expected result file np_result.bin.

4. Build the sample project to generate an executable file for single-operator verification.
   1. Configure the paths of the header files and library files on which build depends.

      In the src/CMakeLists.txt build script, set the path of the header files to /usr/local/Ascend/acllib/include/ and set the path of the library files to /usr/local/Ascend/acllib/lib64/stub/.

      • If the environment variables ${DDK_PATH} and ${NPU_HOST_LIB} are not set, the paths predefined in the CMakeLists.txt build script are used.
      • If the environment variables ${DDK_PATH} and ${NPU_HOST_LIB} are set, the CMakeLists.txt build script looks up for the header files and library files in the specified paths, respectively.

        The following is an example of setting environment variables. Replace /home/HwHiAiUser/Ascend/ascend-toolkit/latest with the actual installation path of the standard-form ACLlib.

        export DDK_PATH=/home/HwHiAiUser/Ascend/ascend-toolkit/latest
        export NPU_HOST_LIB=/home/HwHiAiUser/Ascend/ascend-toolkit/latest/acllib/lib64/stub

   2. Go to the acl_execute_matmul directory of the sample project and run the following command in this directory to create a directory for storing the generated executable file, for example, build/intermediates/host.

      mkdir -p build/intermediates/host

   3. Go to the build/intermediates/host directory and run the cmake command.

      cd build/intermediates/host

      cmake ../../../src -DCMAKE_CXX_COMPILER=g++ -DCMAKE_SKIP_RPATH=TRUE

      • Replace ../../../src with the actual directory of CMakeLists.txt.
      • DCMAKE_CXX_COMPILER: indicates the compiler used to build the app.
      • DCMAKE_SKIP_RPATH: If set to TRUE, this parameter indicates that rpath (path specified by NPU_HOST_LIB, that is, xxx/acllib/lib64/stub) is not added to the executable file generated after build.

        The executable automatically looks up for dynamic libraries in the path (xxx/acllib/lib64) included in LD_LIBRARY_PATH.

        The .so library files in the /home/HwHiAiUser/Ascend/ascend-toolkit/latest/acllib/lib64/stub directory are required to build the code logic based on the AscendCL APIs, without depending on any .so library files of other components (such as Driver).

        After successful build, when you run an app on the host, the app can be linked to the .so library files in the /home/HwHiAiUser/Ascend/ascend-toolkit/latest/acllib/lib64 directory on the host by configuring corresponding environmental variables. The app is automatically linked to the dependent .so library files of other components during runtime.

   4. Run the following command to generate an executable file:

      make

      The executable file execute_matmul_tik_op is generated in the run/out directory of the project.

5. Execute the single-operator verification file on the host side of the hardware device.
   1. As the HwHiAiUser user (running user), copy the out folder in the acl_execute_matmul/run directory of the sample project in the development environment to any directory in the operating environment (host), for example, /home/HwHiAiUser/HIAI_PROJECTS/run_matmul/.
   2. Set the environment variables.
      Run the export command to declare environment variables on the current terminal. The environment variables become invalid when the shell terminal is closed.

      export LD_LIBRARY_PATH=/home/HwHiAiUser/Ascend/ascend-toolkit/latest/acllib/lib64

      /home/HwHiAiUser/Ascend/ascend-toolkit/latest indicates the standard-form ACLlib installation path.

   3. Execute the execute_matmul_tik_op file in the operating environment to verify the single-operator model file.

      Run the following command in /home/HwHiAiUser/HIAI_PROJECTS/run_matmul/out:

      chmod +x execute_matmul_tik_op

      ./execute_matmul_tik_op

      After the execution is complete, the result binary file result_files/output_0.bin is generated.

   4. Compare the actual result with the expected result.

      Go to the test_data/data directory and run the following command:

      python3.7.5 check_out.py

      This script is used to compare the actual operator running result with the expected result.

      If the following information is displayed, the actual running result is consistent with the expected result.

      Compared with the numpy calculation method, the result is correct.

      If the following information is displayed, the actual running result is not consistent with the expected result.

      Compared with the numpy calculation method, the result is wrong.

Verify the execution of a single-operator as follows:

1. Log in to the development environment as the HwHiAiUser user (running user) and go to the acllib/sample/acl_execute_op/acl_execute_matmul directory.
2. Generate the single-operator offline model file of the MatmulTik operator.
   1. Set the environment variables.
      The following is an example of setting environment variables. For details about the environment variables, see "Conversion Example" in ATC Tool Instructions.

      export install_path=/home/HwHiAiUser/Ascend/ascend-toolkit/latest
      export PATH=${install_path}/atc/ccec_compiler/bin:${install_path}/atc/bin:$PATH
      export PYTHONPATH=${install_path}/atc/python/site-packages:${install_path}/atc/python/site-packages/auto_tune.egg/auto_tune:${install_path}/atc/python/site-packages/schedule_search.egg:$PYTHONPATH
      export LD_LIBRARY_PATH=${install_path}/atc/lib64:$LD_LIBRARY_PATH
      export ASCEND_OPP_PATH=${install_path}/opp

      install_path: installation path of the ATC and OPP components.

   2. Go to the acl_execute_matmul/run/out directory of the sample project and run the following command in the out directory to generate a single-operator model file:

      atc --singleop=test_data/config/matmul_tik_op.json --soc_version={soc_version} --output=op_models

      Specifically,

      • singleop: indicates the operator description file (.json).
      • soc_version: version of the Ascend AI Processor. Replace it with the actual version.

        You can view the supported Ascend AI Processor version in the atc/data/platform_config in the ATC installation directory. The name of the .ini file is value of soc_version.

      • --output=op_models: indicates that the generated model file is stored in the op_models folder in the current directory.
      After the model conversion is successful, the following files are generated:

      • The single-operator model file 0_MatmulTik_1_2_16_64_1_2_64_1024_1_2_16_1024.om is generated in the op_models directory of the current directory. The file is named in the format of No.+opType+input description (dateType_format_shape)+output description.

        For details about the mapping between data types, formats, and shapes, see aclDataType and aclFormat in "AscendCL API Reference" in Application Software Development Guide.

      Note: During model conversion, operators in the custom operator library are preferentially searched to match the operators in the model file.

3. Generate test data.

   Go to the run/out/test_data/data directory of the sample project and run the following command:

   python3.7.5 generate_datatik.py

   The data files input_0.bin and input_1.bin (that has the float16 data type and the shapes of (16,64) and (64,1024), respectively) are generated in the current directory. The two files are used for verifying the MatmulTik operator and generating the expected result file np_result.bin.

4. Build the sample project to generate an executable file for single-operator verification.
   1. Configure the paths of the header files and library files on which build depends.

      In the CMakeLists.txt build script, set the path of the header files to /usr/local/Ascend/acllib/include/ and set the path of the library files to /usr/local/Ascend/acllib/lib64/stub/.

      • If the environment variables ${DDK_PATH} and ${NPU_HOST_LIB} are not set, the paths predefined in the CMakeLists.txt build script are used.
      • If the environment variables ${DDK_PATH} and ${NPU_HOST_LIB} are set, the CMakeLists.txt build script looks up for the header files and library files in the specified paths, respectively.

        The following is an example of setting environment variables. Replace /home/HwHiAiUser/Ascend/ascend-toolkit/latest with the actual installation path of the Ascend RC ACLlib.

        export DDK_PATH=/home/HwHiAiUser/Ascend/ascend-toolkit/latest
        export NPU_HOST_LIB=/home/HwHiAiUser/Ascend/ascend-toolkit/latest/acllib/lib64/stub

   2. Go to the acl_execute_matmul directory of the sample project and run the following command in this directory to create a directory for storing the generated executable file, for example, build/intermediates/host.

      mkdir -p build/intermediates/host

   3. Go to the build/intermediates/host directory and run the cmake command.

      cd build/intermediates/host

      cmake ../../../src -DCMAKE_CXX_COMPILER=aarch64-linux-gnu-g++ -DCMAKE_SKIP_RPATH=TRUE

      • Replace ../../../src with the actual directory of CMakeLists.txt.
      • DCMAKE_CXX_COMPILER: indicates the compiler used to build the app.
      • DCMAKE_SKIP_RPATH: If set to TRUE, this parameter indicates that rpath (path specified by NPU_HOST_LIB, that is, xxx/acllib/lib64/stub) is not added to the executable file generated after build.

        The executable automatically looks up for dynamic libraries in the path (xxx/acllib/lib64) included in LD_LIBRARY_PATH.

        The .so library files in the /home/HwHiAiUser/Ascend/ascend-toolkit/latest/acllib/lib64/stub directory are required to build the code logic based on the AscendCL APIs, without depending on any .so library files of other components (such as Driver).

        After successful build, when you run an app on the Atlas 200 DK, the app can be linked to the .so library files in the /home/HwHiAiUser/Ascend/ascend-toolkit/latest/acllib/lib64 directory on the Atlas 200 DK. The app is automatically linked to the dependent .so library files of other components during runtime.

   4. Run the following command to generate an executable file:

      make

      The executable file execute_matmul_tik_op is generated in the run/out directory of the project.

5. Execute the single-operator verification file on the operating environment.
   1. As the HwHiAiUser user (running user), copy the out folder in the acl_execute_matmul/run/ directory of the sample project in the development environment to any directory on the board environment, for example, /home/HwHiAiUser/HIAI_PROJECTS/run_matmul/.
   2. Execute the execute_matmul_tik_op file in the board environment to verify the single-operator model file.

      Run the following command in /home/HwHiAiUser/HIAI_PROJECTS/run_matmul/out:

      chmod +x execute_matmul_tik_op

      ./execute_matmul_tik_op

      After the execution is complete, the input and output data is displayed on the screen, and the result binary file result_files/output_0.bin is generated.

   3. The check_out.py script in the test_data/data directory is used to compare the actual operator running result with the expected result generated by calling NumPy.

      You can copy the out folder generated after the result file is generated to the development environment, go to the test_data/data directory in the development environment, and perform the following operations to verify the result:

      python3.7.5 check_out.py

      This script is used to compare the actual operator running result with the expected result.

      If the following information is displayed, the actual running result is consistent with the expected result.

      Compared with the numpy calculation method, the result is correct.

      If the following information is displayed, the actual running result is not consistent with the expected result.

      Compared with the numpy calculation method, the result is wrong.