Image Classification Based on Caffe ResNet-50 (Including Video Decoding)
Sample Overview
Sample path: ${pyACL_install_path}/sample/acl_execute_model/acl_vdec_resnet50/
${pyACL_install_path} indicates the installation path of the pyACL package. (The pyACL package is installed independently. If the pyACL package is a combined package, for example, Toolkit and NNRT, ${pyACL_install_path} indicates the directory where pyACL is located after the combination package is successfully installed.)
If the cann-toolkit or cann-nnrt software package is not installed in the environment, upload the code sample file acl_vdec_resnet50 from the environment where the cann-toolkit or cann-nnrt software package is installed.
Function Description
This sample shows how to classify images based on the Caffe ResNet-50 network (single input and single batch).
Convert the model file of the Caffe ResNet-50 network into an offline model (.om file) that adapts to Ascend AI Processors. In the sample, load the .om file and decode a .H265 video stream (containing only one frame) for 10 times to obtain 10 YUV420SP NV12 images, resize the 10 images, infer the resized images and obtain the inference results. Then, process the inference results and output the class indexes with the top confidence values and the sum of the top 5 confidence values.
Main APIs
Table 10-4 shows the main APIs.
Function |
ACL Module |
ACL Interface Function |
Description |
|---|---|---|---|
Resource initialization |
Initialization |
acl.init |
Initializes the ACL configuration. |
Device management |
acl.rt.set_device |
Specifies the device for computation. |
|
Context management |
acl.rt.create_context |
Creates a context. |
|
Stream management |
acl.rt.create_stream |
Creates a stream. |
|
Model initialization |
Model loading and execution |
acl.mdl.load_from_file |
Loads the model from the .om file to the device. |
Data types and operation APIs |
acl.mdl.create_desc |
Creates data for describing the model. |
|
Data types and operation APIs |
acl.mdl.get_desc |
Obtains data for describing the model. |
|
Data preprocessing |
Media data module |
acl.media.vdec_send_frame |
Functions as a video decoding API. |
Data types and operation APIs |
acl.media.vdec_set_channel_desc series |
Sets the description of a video processing channel. |
|
Data types and operation APIs |
acl.media.dvpp_vpc_resize_async |
Resizes the input image to the size of the output image. |
|
Data types and operation APIs |
acl.media.dvpp_set_pic_desc series |
Sets image description parameters. |
|
Model inference |
Model loading and execution |
acl.mdl.execute |
Performs synchronous model inference. |
Data postprocessing |
Data types and operation APIs |
acl.op.create_attr |
Creates data of the aclopAttr type. |
Data types and operation APIs |
acl.create_tensor_desc |
Creates data of the aclTensorDesc type. |
|
Data types and operation APIs |
acl.get_tensor_desc_size |
Obtains the size of a tensor description. |
|
Data types and operation APIs |
acl.create_data_buffer |
Creates data of the aclDataBuffer type. |
|
Data exchange |
Memory management |
acl.rt.memcpy |
Sends data from the host to the device or from the device to the host. |
Memory management |
acl.media.dvpp_malloc |
Allocates memory for media data processing on the device. |
|
Memory management |
acl.rt.maclloc |
Allocates memory on the device. |
|
Memory management |
acl.rt.malloc_host |
Allocates memory on the host. |
|
Single-operator inference |
Operator loading and execution |
acl.op.execute |
Loads and executes an operator. It is an asynchronous interface. |
Common module |
-- |
acl.util.ptr_to_numpy |
Converts a pointer to data of the NumPy type. |
-- |
acl.util.numpy_to_ptr |
Converts data of the NumPy type to a pointer. |
|
Resource release |
Memory management |
acl.rt.free |
Frees the memory on the device. |
Memory management |
acl.media.dvpp_free |
Allocates memory by using acldvppMalloc. |
|
Memory management |
acl.rt.free_host |
Frees the memory on the host. |
|
Model loading and execution |
acl.mdl.unload |
Unloads a model. |
|
Stream management |
acl.rt.destroy_stream |
Destroys a stream. |
|
Context management |
acl.rt.destroy_context |
Destroys a context. |
|
Device management |
acl.rt.reset_device |
Resets the current device and reclaims the resources on the device. |
|
Initialization |
acl.finalize |
Deinitializes the ACL. |
Video Decoding and Model Inference Process
Figure 10-3 shows the video decoding and model inference process.
Directory Structure
The directory structure is as follows:
acl_vdec_resnet50 ├── acl_dvpp.py //Implementation file of image resizing ├── acl_model.py //Implementation file of model inference ├── acl_sample.py //Running file ├── acl_util.py //Implementation file of tool functions ├── acl_vdec.py //Implementation file of video decoding ├── constant.py //Constant definition ├── caffe_model │ ├── resnet50.caffemodel //ResNet-50 model │ └── resnet50.prototxt //ResNet-50 model file ├── data //The directory is created by the user. │ └── vdec_h265_1frame_rabbit_1280x720.h265 //Video file to be processed by the user, which is obtained by the user. ├── model │ └── resnet50_aipp.om //Inference model └── vdec_out //Video decoding output path
Application Running
Model Conversion
- Log in to the development environment as the HwHiAiUser user (running user).
- For details, see "Getting Started with ATC" in ATC Tool Instructions.
- Prepare data.
Obtain the ResNet-50 weight file (*.caffemodel) and model file (resnet50.prototxt) from the following URL and upload the files to the acl_vdec_resnet50/caffe_model directory in the development environment as the HwHiAiUser user (running user):
https://github.com/Ascend-Huawei/models/tree/master/computer_vision/classification/resnet50: find the download link in the README_en.md file.
Prepare the input video stream.
Click here to obtain the input video stream file vdec_h265_1frame_rabbit_1280x720.h265 and upload it to the acl_vdec_resnet50/data directory in the development environment as the HwHiAiUser user.
- Convert the ResNet-50 network into an offline model (.om file) that adapts to Ascend AI Processors. During model conversion, you need to set CSC parameters to convert YUV420SP images to RGB images.
Go to the acl_vdec_resnet50 directory and run the following command:
atc --model=caffe_model/resnet50.prototxt --weight=caffe_model/resnet50.caffemodel --framework=0 --output=model/resnet50_aipp --soc_version=Ascend310 --insert_op_conf=caffe_model/aipp.cfg
--output: The generated resnet50_aipp.om file is stored in the ./model directory.
Application Running
- Upload the data directory, model directory and all Python script files in the acl_vdec_resnet50 directory of the development environment to the same directory (for example, /home/HwHiAiUser/acl_vdec_resnet50) of the operating environment (host) as the running user.
If the development environment and operating environment are on the same host, skip this step.
- Configure the operating environment by referring to Environment Variable Configuration.
- Run the acl_sample.py command. After the command is successfully run, the following information is displayed:
