"RuntimeError: Cannot find scale_d of layer '**' in record file" Is Displayed During Quantization

Symptom

When the save_model API is called to save the quantization model during quantization, the data quantization parameters scale_d and offset_d computed in the calibration phase need to be read. If the corresponding parameters cannot be found in the corresponding record file, the quantization model cannot be saved. Therefore, the preceding AMCT error is reported and the process is terminated.

Possible Cause

The scale_d and offset_d parameters are saved during calibration (when the Caffe framework is called to perform forward calculation on the calibration model). The AMCT inserts the IFMR layer into the calibration model, and the IFMR layer needs to accumulate enough user-specified batch_num data and then perform quantization to obtain scale_d and offset_d. The causes of the "RuntimeError: Cannot find scale_d of layer * in record file" error are as follows:

1. An error occurs when the Caffe inference is executed. The possible causes are as follows: The compiled Caffe is faulty, the calibration model is faulty, or the corresponding dataset cannot be found. You can view the exception information thrown by the Caffe framework.
2. The data amount of the calibration dataset provided by the user does not meet the data amount required by batch_num. For example, if the user provides only one batch of data as the calibration dataset and batch_num is set to 2, the IFMR layer does not store sufficient data during calibration, the quantization operation cannot be performed. As a result, scale_d and offset_d cannot be computed, and the preceding error is also triggered. You can view the process information printed during the quantization of the IFMR to locate the fault. The amount of saved data is displayed in the IFMR layer.

   When the specified amount of data is accumulated, the quantization operation is triggered.

   The quantization of the current layer is complete only when the "Do layer * activation calibration success! information" is displayed.

Solution

1. Rectify the fault based on the error reported by the Caffe framework.
2. Increase the data volume of the calibration dataset or decrease the value of batch_num for the quantization algorithm until the data volume of the calibration dataset is greater than or equal to the value of batch_num. Note that decreasing the value of batch_num may reduce the model accuracy after quantization.