Image Classification Based on Caffe ResNet-50 (Including Image Decoding and Resizing)
Sample Overview
Sample path: ${pyACL_install_path}/sample/acl_execute_model/acl_dvpp_resnet50/
${pyACL_install_path} indicates the installation path of the pyACL package. (The pyACL package is installed independently. If the pyACL package is a combined package, for example, Toolkit and NNRT, ${pyACL_install_path} indicates the directory where pyACL is located after the combination package is successfully installed.)
If the cann-toolkit or cann-nnrt software package is not installed in the environment, upload the code sample file acl_dvpp_resnet50 from the environment where the cann-toolkit or cann-nnrt software package is installed.
Function Description
This sample shows how to classify images based on the Caffe ResNet-50 network (single input and single batch).
Convert the model file of the Caffe ResNet-50 network into an offline model (.om file) that adapts to Ascend AI Processors. In the sample, load the .om file, decode, resize and infer two .jpg images, obtain the inference results, process the inference results, and outputs the class indexes with the top confidence values.
During model conversion, you need to set the CSC parameters to convert YUV420SP images into RGB images to meet the input requirements of the model.
Main APIs
Table 10-3 shows the main APIs.
Function |
ACL Module |
ACL Interface Function |
Description |
|---|---|---|---|
Resource initialization |
Initialization |
acl.init |
Initializes the ACL configuration. |
Device management |
acl.rt.set_device |
Specifies the device for computation. |
|
Context management |
acl.rt.create_context |
Creates a context. |
|
Stream management |
acl.rt.create_stream |
Creates a stream. |
|
Operator loading and execution |
acl.op.set_model_dir |
Loads the directory of a model file. |
|
Model initialization |
Model loading and execution |
acl.mdl.load_from_file |
Loads the model from the .om file to the device. |
Data types and operation APIs |
acl.mdl.create_desc |
Creates data for describing the model. |
|
Data types and operation APIs |
acl.mdl.get_desc |
Obtains data for describing the model. |
|
Data preprocessing |
Media data module |
acl.media.dvpp_jpeg_decode_async |
Functions as a graphic decoding API. |
Data types and operation APIs |
acl.media.dvpp_vpc_resize_async |
Resizes the input image to the size of the output image. |
|
Data types and operation APIs |
acl.media.dvpp_set_pic_desc series |
Sets image description parameters. |
|
Model inference |
Model loading and execution |
acl.mdl.execute |
Performs synchronous model inference. |
Data postprocessing |
Data types and operation APIs |
acl.op.create_attr |
Creates data of the aclopAttr type. |
Data types and operation APIs |
acl.create_tensor_desc |
Creates data of the aclTensorDesc type. |
|
Data types and operation APIs |
acl.get_tensor_desc_size |
Obtains the size of a tensor description. |
|
Data types and operation APIs |
acl.create_data_buffer |
Creates data of the aclDataBuffer type. |
|
Data exchange |
Memory management |
acl.rt.memcpy |
Sends data from the host to the device or from the device to the host. |
Memory management |
acl.media.dvpp_malloc |
Allocates memory for media data processing on the device. |
|
Memory management |
acl.rt.maclloc |
Allocates memory on the device. |
|
Memory management |
acl.rt.malloc_host |
Allocates memory on the host. |
|
Single-operator inference |
Operator loading and execution |
acl.op.execute |
Loads and executes an operator. It is an asynchronous interface. |
Common module |
-- |
acl.util.ptr_to_numpy |
Converts a pointer to data of the NumPy type. |
-- |
acl.util.numpy_to_ptr |
Converts data of the NumPy type to a pointer. |
|
Resource release |
Memory management |
acl.rt.free |
Frees the memory on the device. |
Memory management |
acl.media.dvpp_free |
Allocates memory by using acldvppMalloc. |
|
Memory management |
acl.rt.free_host |
Frees the memory on the host. |
|
Model loading and execution |
acl.mdl.unload |
Unloads a model. |
|
Stream management |
acl.rt.destroy_stream |
Destroys a stream. |
|
Context management |
acl.rt.destroy_context |
Destroys a context. |
|
Device management |
acl.rt.reset_device |
Resets the current device and reclaims the resources on the device. |
|
Initialization |
acl.finalize |
Deinitializes the ACL. |
Image Decoding and Resizing Process
Figure 10-2 shows the image decoding and resizing process.
Directory Structure
The directory structure is as follows:
acl_dvpp_resnet50
├── acl_dvpp.py //Implementing file of image decoding and resizing
├── acl_model.py //Implementation file of model inference
├── acl_op.py //Implementation file of single-operator precision conversion
├── acl_sample.py //Running file for process logic control
├── acl_util.py //Implementation file of tool functions
├── constant.py //Constant definition
├── caffe_model
│ ├── resnet50.caffemodel //ResNet-50 model
│ └── resnet50.prototxt //ResNet-50 model file
├── data //Test data
│ ├── dog1_1024_683.jpg
│ └── dog2_1024_683.jpg
├── model
│ └── resnet50_aipp.om //Inference model
├── op_models
├── 0_Cast_0_2_1000_1_2_1000.om //Precision conversion custom operator
├── 1_ArgMaxD_1_2_1000_3_2_1.om //precision conversion custom operator
└── op_list.json //Configuration file of the precision conversion operator
Application Running
Model Conversion
- Log in to the development environment as the HwHiAiUser user (running user).
- For details, see "Getting Started with ATC" in ATC Tool Instructions.
- Prepare data.
Obtain the ResNet-50 weight file (*.caffemodel) and model file (resnet50.prototxt) from the following URL and upload the files to the acl_dvpp_resnet50/caffe_model directory in the development environment as the HwHiAiUser user (running user):
https://github.com/Ascend-Huawei/models/tree/master/computer_vision/classification/resnet50: find the download link in the README_en.md file.
- Convert the ResNet-50 network into an offline model (.om file) that adapts to Ascend AI Processors. During model conversion, you need to set CSC parameters to convert YUV420SP images to RGB images.
Go to the acl_dvpp_resnet50 directory and run the following command:
atc --model=caffe_model/resnet50.prototxt --weight=caffe_model/resnet50.caffemodel --framework=0 --output=model/resnet50_aipp --soc_version=Ascend310 --insert_op_conf=caffe_model/aipp.cfg
--output: The generated resnet50_aipp.om file is stored in the ./model directory.
- Compile the operator description information (.json file) of the Cast and ArgMaxD operators into an offline model (.om file) that adapts to the Ascend AI Processors to verify the running of a single operator.
Go to the acl_dvpp_resnet50 directory and run the following command:
atc --singleop=op_models/op_list.json --soc_version=Ascend310 --output=op_models/
--output: specifies the directory for storing the generated om file, that is, the ./op_models directory.
For details about the parameters, see the parameter description in ATC Tool Instructions.
Application Running
- Upload the data directory, model directory, op_models directory and all Python script files in the acl_dvpp_resnet50 directory of the development environment to the same directory (for example, /home/HwHiAiUser/acl_dvpp_resnet50) of the operating environment (host) as the running user.
If the development environment and operating environment are on the same host, skip this step.
- Configure the operating environment by referring to Environment Variable Configuration.
- Run acl_sample.py. After the command is successfully run, the class index with the top confidence is displayed.
