VDEC

Functions and Restrictions

The video decoder (VDEC) decodes videos. After VPC processing, the VDEC outputs YUV420SP images (including NV12 and NV21).

• About input
  • Input resolution range:

    128 x 128 to 4096 x 4096

  • Input formats:

    H.264 BP/MP/HP Level 5.1 YUV420 encoded streams

    H.265 8-bit/10-bit Level 5.1 YUV420 encoded streams

  • Input memory:

    Allocate memory by calling acl.rt.malloc and free memory by calling acl.rt.free. Alternatively, allocate memory by calling acl.media.dvpp_malloc and free memory by calling acl.media.dvpp_free.

• About output
  • Output image formats of VDEC:
    • YUV420SP NV12
    • YUV420SP NV21
  • Output memory:
    • Calculate the output memory size according to the image format.

      YUV420SP: widthStride x heightStride x 3/2

    • The start address of the output memory must be 16-byte aligned. Allocate memory by calling acl.media.dvpp_malloc and free the memory by calling acl.media.dvpp_free.
  • widthStride and heightStride alignment:
    • widthStride: Round up the input width to the nearest multiple of 16.
    • heightStride: Round up the output height to the nearest multiple of 2.
• The VDEC decodes input streams frame by frame.
• Bad frames or frame loss in the streams may cause VDEC frame loss.
• The VDEC cannot decode the streams encoded in interlaced scanning mode.

Performance Specifications

• The recommended number of channels (n) for VDEC decoding is as follows:
  720p indicates a resolution of 1280 x 720 pixels. 1080p indicates a resolution of 1920 x 1080 pixels. 4K indicates a resolution of 3840 x 2160 pixels.

  Resolution	Total Frame Rate	Single-channel performance (n)	Memory Consumption Per VDEC Channel (for Reference Only)
  3840*2160	120fps	120 fps/n (n = 4 is recommended, 30 fps per channel.)	About 170 MB
  1920*1080	480fps	480 fps/n (n = 16 is recommended, 30 fps per channel.)	About 86 MB
  ≤1280*720	960 fps	960 fps/n (n = 32 is recommended, 30 fps per channel.)	About 70 MB

• Suggestions on the number of decoding channels and frame rate:

  The specifications in the following table are for reference only. If the number of channels started in a single process exceeds the specifications in the following table, memory insufficiency or performance drop may occur, leading to decoding channel creation failures and slow decoding.

  720p indicates a resolution of 1280 x 720 pixels. 1080p indicates a resolution of 1920 x 1080 pixels. 4K indicates a resolution of 3840 x 2160 pixels.

  Typical Resolution	Suggested Channel Count Per Process (Based on the Input Frame Rate)
  -	Input frame rate ≥ 25 fps	20 fps < input frame rate < 25 fps	15 fps < input frame rate ≤ 20 fps	10 fps < input frame rate ≤ 15 fps	Input frame rate ≤ 10 fps
  ≤720p	32 channels	32 channels	32 channels	32 channels	32 channels
  1080p	16 channels	19 channels	24 channels	32 channels	32 channels
  4K	4 channels	4 channels	6 channels	8 channels	12 channels

Basic Principles

1. Call acl.media.vdec_create_channel to create a channel for video decoding.
   • Perform the following steps before creating a channel for video decoding.
     1. Call acl.media.vdec_create_channel_desc to create channel description.
     2. Call acl.media.vdec_set_channel_desc series to set attributes of the channel description, including the decoding channel ID, thread, callback function, and video encoding protocol.
        1. The callback function needs to be created by the user in advance. It is used to obtain the decoding data after video decoding and free resources in a timely manner. The callback function prototype is as follows:

           vdec_callback(input_stream_desc, output_pic_desc, user_data)
               """
               VDEC callback function: The function and parameter names can be customized. The number and type of parameters must match.
               :param input_stream_desc:
               :param output_pic_desc:  
               :param user_data: 
               :return: 
               """

           Call acl.media.dvpp_get_pic_desc_ret_code in the callback function to obtain the return code retCode. The value 0 indicates the success of decoding, while 1 indicates failure. If decoding fails, locate the fault based on the return code in the log. For details, see Return Code.

           After the decoding is complete, you are advised to free the memory for storing the input streams and output images of the VDEC, and the corresponding video stream description and image description in the callback function in a timely manner.

        2. The user needs to create a thread in advance and customize a thread function. Calling acl.rt.process_report in the thread function triggers the callback function in 1.b.i after a specified period of time.

        If acl.media.vdec_set_channel_desc_out_pic_format is not called to set the output format, YUV420SP NV12 is used by default.

   • The following APIs are encapsulated in acl.media.vdec_create_channel and do not need to be called separately:
     1. acl.rt.create_stream: explicitly creates a stream. It is internally used for VDEC.
     2. acl.rt.subscribe_report: specifies a thread for processing the callback function in a stream. The callback function and thread are specified by calling acl.media.venc_set_channel_desc series.
2. Call acl.media.vdec_send_frame to decode a video stream into a YUV420SP image.
   • Perform the following steps before decoding a video:
     • Call acl.media.dvpp_create_stream_desc to create the description of the input video stream, and call acl.media.dvpp_set_stream_desc series to configure the input video, such as the memory address, memory size, and stream format.
     • Call acl.media.dvpp_create_pic_desc to create the description of the output image, and call acl.dvpp.set_pic_desc series to configure the output image, such as the memory address, memory size, and image format.
   • acl.rt.launch_callback is encapsulated in the acl.media.vdec_send_frame interface to add a callback function to be executed on the host to the stream task queue. acl.rt.launch_callback does not need to be called separately.
3. Call acl.media.vdec_destroy_channel to destroy a video processing channel.
   • The channel is destroyed only after the transmitted frames are decoded and the callback function is processed.
   • The following APIs are encapsulated in acl.media.vdec_destroy_channel and do not need to be called separately:
     • acl.rt.unsubscribe_report: unsubscribes a thread. (The callback function in the stream is no longer processed by the specified thread.)
     • acl.rt.destroy_stream: destroys a stream.
   • Call acl.media.vdec_destroy_channel_desc to destroy the channel description after the channel is destroyed.

Sample Code

import acl
# ......

# 1. Initialize the ACL.
ret = acl.init()

# 2. Allocate runtime resources, including devices, contexts, and streams.
ret = acl.rt.set_device(self.device_id)
self.context, ret = acl.rt.create_context(self.device_id)
self.stream, ret = acl.rt.create_stream()

# 3. Create a callback function.
def _callback(self, input_stream_desc, output_pic_desc, user_data):
    # Input stream description
    if input_stream_desc:
        ret = acl.media.dvpp_destroy_stream_desc(input_stream_desc)
        if ret != 0:
            print("acl.media.dvpp_destroy_stream_desc failed")
    # Output image description.
    if output_pic_desc:
        vdec_out_buffer = acl.media.dvpp_get_pic_desc_data(output_pic_desc)
        ret_code = acl.media.dvpp_get_pic_desc_ret_code(output_pic_desc)
        data_size = acl.media.dvpp_get_pic_desc_size(output_pic_desc)
        self.images_buffer.append(dict({"buffer": vdec_out_buffer,
                                        "size": data_size}))
        ret = acl.media.dvpp_destroy_pic_desc(output_pic_desc)
        if ret != 0:
            print("acl.media.dvpp_destroy_pic_desc failed")
    self.output_count += 1
    print("[Vdec] [_callback] _callback exit success")
    # In this example, release vdec_out_buffer after the VDEC process.

# 4. Create a thread for processing a callback function, submit and register the thread to execute the callback function.
timeout = 100
cb_thread_id, ret = acl.util.start_thread(self._thread_func, [timeout])
acl.rt.subscribe_report(cb_thread_id, self.stream)

# 5. Create the description of the video stream processing channel, and set the attributes of the video processing channel description. The thread and callback function need to be created in advance.
def init_resource(self, cb_thread_id):
    print("[Vdec] class Vdec init resource stage:")
    self.vdec_channel_desc = acl.media.vdec_create_channel_desc()
    acl.media.vdec_set_channel_desc_channel_id(self.vdec_channel_desc, self._channel_id)
    acl.media.vdec_set_channel_desc_thread_id(self.vdec_channel_desc, cb_thread_id) 
    acl.media.vdec_set_channel_desc_callback(self.vdec_channel_desc, self._callback)
    acl.media.vdec_set_channel_desc_entype(self.vdec_channel_desc, self._en_type)
    acl.media.vdec_set_channel_desc_out_pic_format(self.vdec_channel_desc, self._format)
    acl.media.vdec_create_channel(self.vdec_channel_desc)
    print("[Vdec] class Vdec init resource stage success")

# 6. Create a channel for processing video streams.
ret = acl.media.vdec_create_channel(self.vdec_channel_desc)
# ......
# 7. Allocate device buffer dataDev to store the input video data for decoding.
# Copy image data from the host to the device by calling acl.rt.memcpy. After data copy is complete, call acl.rt.free_host to free the host memory in a timely manner.
# Set the alignment mode of the output image size.
output_pic_size = (self.input_width * self.input_height * 3) // 2
img = np.fromfile(img_path, dtype=self.dtype)
input_stream_size = img.size
img_ptr = acl.util.numpy_to_ptr(img)
self.input_stream_mem, ret = acl.media.dvpp_malloc(input_stream_size)
ret = acl.rt.memcpy(self.input_stream_mem,
                    input_stream_size,
                    img_ptr,
                    input_stream_size,
                    ACL_MEMCPY_HOST_TO_DEVICE)
       
# 8. Perform video decoding for 10 times and output 10 YUV420SP NV12 images.
def forward(self, output_pic_size, input_stream_size):
    self.frame_config = acl.media.vdec_create_frame_config()
    for i in range(self.rest_len):
        print("[Vdec] forward index:{}".format(i))
         # 8.1 Create the description of the input video stream and set the stream attributes.
        self._set_input(input_stream_size)
        # 8.2 Create the description of the output image and set the attributes of the image description.
        self._set_pic_output(output_pic_size)
        # 8.3 Decode video streams. The callback function is called after each decoded frame to write data to the file, and free resources in a timely manner.
        ret = acl.media.vdec_send_frame(self.vdec_channel_desc,
                                        self.dvpp_stream_desc,
                                        self.dvpp_pic_desc,
                                        self.frame_config,
                                        None)
        check_ret("acl.media.vdec_send_frame", ret)
        print('[Vdec] vdec_send_frame stage success')

# 9. Release the image processing channel and image description.
ret = acl.media.vdec_destroy_channel(self.vdec_channel_desc)
acl.media.vdec_destroy_channel_desc(self.vdec_channel_desc)

# 10. Release runtime resources.
ret = acl.rt.destroy_dtream(self.stream)
ret = acl.rt.destroy_context(self.context)
ret = acl.rt.reset_device(self.device_id)
ret = acl.finalize()

# ......

Return Code