VENC

Functions and Restrictions

The video encoder (VENC) encodes YUV420SP NV12/NV21 8-bit image data into H.264/H.265 video streams. Multi-thread processes are not supported.

• About input
  • Input resolution range:

    128 x 128 to 1920 x 1920

  • Input formats:

    YUV420SP NV12/NV21-8bit

  • Input memory:

    Allocate the device memory by calling acl.media.dvpp_malloc and free the memory by calling acl.media.dvpp_free.

• About output
  • Output formats:

    H264 BP/MP/HP

    H.265 MP (slice streams only)

  • Output memory:

    The output memory is managed by the system.

Performance Specifications

1080p indicates a resolution of 1920 x 1080 pixels.

Basic Principles

1. Call acl.media.vdec_create_channel to create a channel for video decoding.
   • Perform the following steps before creating a channel for video encoding.
     1. Call acl.media.venc_create_channel_desc to create channel description.
     2. Call acl.media.venc_set_channel_desc series to set attributes of the channel description, including the thread, callback function, video encoding protocol, and input image format.
        1. The callback function needs to be created by the user in advance. It is used to obtain the encoding data after video encoding and free resources in a timely manner. For details about the callback function prototype, see aclvencCallback.

           venc_callback(input_pic_desc, output_tream_desc, user_data)
               """
               VENC callback function: The function and parameter names can be customized. The number and type of parameters must match.
               :param input_pic_desc: 
               :param output_tream_desc: 
               :param user_data: 
               :return: 
               """

           After the encoding is complete, you are advised to free the memory for storing the input images and the corresponding image description in the callback function in a timely manner. The output memory is managed by the system. Therefore, the output memory does not need to be freed by the user.

        2. The user needs to create a thread in advance and customize a thread function. Calling acl.rt.process_report in the thread function triggers the callback function in 1.b.i after a specified period of time.
   • The following APIs are encapsulated in acl.media.venc_create_channel and do not need to be called separately:
     1. acl.rt.create_stream: explicitly creates a stream. It is internally used for VENC.
     2. acl.rt.subscribe_report: specifies a thread for processing the callback function in a stream. The callback function and thread are specified by calling the acl.media.venc_set_channel_desc series.
2. Call acl.media.venc_send_frame to encode YUV420SP images into H.264/H.265 video streams.
   • Perform the following steps before encoding a video:
     • Call acl.media.dvpp_create_pic_desc to create the description of the input image, and call acl.media.dvpp_set_pic_desc to configure the input image, such as the memory address, memory size, and image format.
     • Call acl.media.venc_create_frame_config to create the single-frame configuration data, and call acl.media.venc_set_frame_config series to configure whether to forcibly restart the I-frame interval or end the frame.
   • acl.rt.launch_callback is encapsulated in the acl.media.venc_send_frame interface to add a callback function to be executed on the host to the stream task queue. acl.rt.launch_callback does not need to be called separately.
3. Call acl.media.venc_destroy_channel to destroy a video processing channel.
   • The channel is destroyed only after the transmitted frames are decoded and the callback function is processed.
   • The following APIs are encapsulated in acl.media.venc_destroy_channel and do not need to be called separately:
     • acl.rt.unsubscribe_report: unsubscribes a thread. (The callback function in the stream is no longer processed by the specified thread.)
     • acl.rt.destroy_stream: destroys a stream.
   • Call acl.media.venc_destroy_channel_desc to destroy the channel description after the channel is destroyed.

Sample Code

You can view the complete sample code in the acl_venc.py file of the acl_venc sample.

import acl
# ......

# 1. Resource initialization: Create an AclVenc object for initialization.
# 1.1 Initialize the ACL.
ret = acl.init()
# 1.2 Allocate runtime resources, including devices, and contexts.
device_id = 0
ret = acl.rt.set_device(device_id )
self.context, ret = acl.rt.create_context( device_id )
# Call acl.rt.get_run_mode to obtain the run mode of the software stack and determine the logic for calling the memory application interface based on the run mode. For details, see "Typical Feature Walkthrough > Data Transfer".
runMode, ret = acl.rt.get_run_mode()

# 2. Create a thread for executing the callback function and thread function.
def cb_thread_func(self, args_list):
    ctx = args_list[0]
    timeout = args_list[1]
    print("[info] thread args_list = ", self.ctx, timeout, self.g_callbackRunFlag, "\n")
    ret = acl.rt.set_context(ctx)
    assert ret == 0

    while self.g_callbackRunFlag is True:
        print("[info] thread g_callbackRunFlag = ", self.g_callbackRunFlag, "\n")
        ret = acl.rt.process_report(timeout)
        print("[info] acl.rt.process_report =", ret)

timeout = 1000
self.cb_thread_id, ret = acl.util.start_thread(self.cb_thread_func, [self.ctx, timeout])

# 3. Create a callback function.
# 3.1 Obtain stream data output.
def get_stream_data(self, stream_desc):
    stream_data = acl.media.dvpp_get_stream_desc_data(stream_desc)
    assert stream_data is not None
    stream_data_size = acl.media.dvpp_get_stream_desc_size(stream_desc)
    print("[info] stream_data size", stream_data_size)
    # stream memcpy d2h
    np_data = np.zeros(stream_data_size, dtype=np.byte)
    np_data_ptr = acl.util.numpy_to_ptr(np_data)
    ret = acl.rt.memcpy(np_data_ptr, stream_data_size,
                        stream_data, stream_data_size,
                        memcpy_kind.get("ACL_MEMCPY_DEVICE_TO_HOST"))
    assert ret == 0
    return np_data
# 3.2 Save the video stream to a file using the callback function.
def callback_func(self, input_pic_desc, output_tream_desc, user_data):
    # Obtain the video encoding result and convert it into a NumPy object.
    output_numpy = self.get_stream_data(output_tream_desc)
    with open('./data/output.h265', 'ab') as f:
        f.write(output_numpy)
    print("[INFO] [callback_func] stream size =", acl.media.dvpp_get_stream_desc_size(output_stream_desc))
  
# 4. Create the description of a video encoding channel.
self.venc_channel_desc = acl.media.venc_create_channel_desc()

# 5. Set the description of a video encoding channel. The thread and callback function need to be created in advance.
# vencChannelDesc_ is of the aclvdecChannelDesc type.
venc_format = 1
venc_entype = 0
input_width = 1280
input_height = 720
ret = acl.media.venc_set_channel_desc_thread_id(self.venc_channel_desc, self.cb_thread_id)
ret = acl.media.venc_set_channel_desc_callback(self.venc_channel_desc, self.callback_func)
ret = acl.media.venc_set_channel_desc_entype(self.venc_channel_desc, venc_entype)
ret = acl.media.venc_set_channel_desc_pic_format(self.venc_channel_desc, venc_format)
ret = acl.media.venc_set_channel_desc_key_frame_interval(self.venc_channel_desc, 16)
ret = acl.media.venc_set_channel_desc_pic_height(self.venc_channel_desc, input_height)
ret = acl.media.venc_set_channel_desc_pic_width(self.venc_channel_desc, input_width )

# 6. Create a channel for processing video streams and the single-frame encoding configuration.
ret = acl.media.venc_create_channel(self.venc_channel_desc)
# vencFrameConfig_ is of the aclvencFrameConfig type.
frame_config = acl.media.venc_create_frame_config()

# 7. Allocate device buffer dataDev to store the input video data for encoding.
# 7.1 Read image data
output_pic_size = (input_width * input_height * 3) // 2
print("[INFO] output_pic_size:", output_pic_size, " load vdec file:", venc_file_path)
file_context = np.fromfile(venc_file_path, dtype=np.byte)
file_size = file_context.size
file_mem = acl.util.numpy_to_ptr(file_context)
input_size = file_size
# If the run mode of the software stack obtained by calling acl.rt.get_run_mode is ACL_HOST, transmit image data of the host to the device by calling acl.rt.memcpy. After the data transfer is complete, call acl.rt.free_host to release the host memory in a timely manner.
# If the run mode of the software stack obtained by calling acl.rt.get_run_mode is ACL_DEVICE, allocate the device memory directly to store the input image data.
# The run mode is ACL_HOST.
input_mem, ret = acl.media.dvpp_malloc(input_size)
assert ret == 0
ret = acl.rt.memcpy(input_mem, input_size, file_mem,
                    file_size, memcpy_kind.get("ACL_MEMCPY_HOST_TO_DEVICE"))
assert ret == 0

# 8. Perform video encoding.
def venc_set_frame_config(self, frame_confg, eos, iframe):
    ret = acl.media.venc_set_frame_config_eos(frame_confg, eos)
    assert ret == 0
    ret = acl.media.venc_set_frame_config_force_i_frame(frame_confg, iframe)
    assert ret == 0

def venc_process(self, venc_channel_desc, input_mem, input_size, frame_config):
   # Set the frame to a non-end frame.
    self.venc_set_frame_config(frame_config, 0, 0)
    print("[INFO] set frame config")
    self.test_get_frame_config(frame_config)

    # set picture description
    dvpp_pic_desc = acl.media.dvpp_create_pic_desc()
    assert dvpp_pic_desc is not None
    ret = acl.media.dvpp_set_pic_desc_data(dvpp_pic_desc, input_mem)
    assert ret == 0
    ret = acl.media.dvpp_set_pic_desc_size(dvpp_pic_desc, input_size)
    assert ret == 0
    print("[INFO] set pic desc")

    # Send one image to the encoder for encoding.
    venc_cnt = 16
    while venc_cnt:
        ret = acl.media.venc_send_frame(venc_channel_desc, dvpp_pic_desc, 0, frame_config, 0)
        assert ret == 0
        print("[INFO] venc send frame")
        venc_cnt -= 1
    # When an empty image with EOS = 1 is sent, the current encoding ends.
    self.venc_set_frame_config(frame_config, 1, 0)
    ret = acl.media.venc_send_frame(venc_channel_desc, 0, 0, frame_config, 0)
    assert ret == 0

# 9. Release resources.
ret = acl.media.dvpp_free(input_mem)
ret = acl.media.venc_destroy_frame_config(frame_config)
assert ret == 0
print("[INFO] free resources")
self.thread_join()
print("[INFO] thread join")
# ......