Quantization Configuration

If inference based on the config.json quantization configuration file generated by the create_quant_retrain_config call has significant accuracy drop, tune the config.json file until the accuracy is as expected. The following is an example of the file content. Keep the layer names unique in the JSON file.

{
    "version":1,
    "conv1":{
        "retrain_enable":true,
        "retrain_data_config":{
            "algo":"ulq_quantize"
        },
        "retrain_weight_config":{
            "algo":"arq_retrain",
            "channel_wise":true
        }
    },
    "conv2_1/expand":{
        "retrain_enable":true,
        "retrain_data_config":{
            "algo":"ulq_quantize"
        },
        "retrain_weight_config":{
            "algo":"arq_retrain",
            "channel_wise":true
        }
    },
    "conv2_1/dwise":{
        "retrain_enable":true,
        "retrain_data_config":{
            "algo":"ulq_quantize"
        },
        "retrain_weight_config":{
            "algo":"arq_retrain",
            "channel_wise":true
        }
    },
}

The following describes the configuration options available in the configuration file.

If the inference accuracy of the model quantized based on the default configuration in the config.json file drops significantly, perform the following steps to tune the quantization configuration file:

1. Execute the quantization script in the amct_tensorflow_sample.tar.gz package to perform quantization based on the default configuration generated by the create_quant_retrain_config API.
2. If the inference accuracy with the model quantized in 1 is as expected, configuration tuning ends. Otherwise, go to 3.
3. Tune the quantization configuration file and include the fixed_min, clip_max, and clip_min options in the file.
   • If the fixed_min option is included: when the upstream layer of the quantization layer is ReLU, you need to manually set this option to true; when the upstream layer of the quantization layer is not ReLU, you need to manually set this option to false
   • clip_max and clip_min control the upper max and lower limit min based on the data distribution of the feature map at each layer. The recommended value range of clip_max is [0.3 * max, 1.7 * max], and the recommended value range of clip_min is [0.3 * min, 1.7 * min]. The mean square error (MSE) after quantization and dequantization is computed to find the optimal clip_max and clip_min values.
   The following is an example of the tuned quantization configuration file:

   {
       "version":1,
       "inference/Conv2D":{
           "retrain_enable":true,
           "retrain_data_config":{
               "algo":"ulq_quantize",
               "clip_max":3.0,
               "clip_min":-3.0,
               "fixed_min":true
           },
           "retrain_weight_config":{
               "algo":"arq_retrain",
               "channel_wise":true
           }
       },
       "inference/Conv2D_1":{
           "retrain_enable":true,
           "retrain_data_config":{
               "algo":"ulq_quantize",
               "clip_max":3.0,
               "clip_min":-3.0,
               "fixed_min":true
           },
           "retrain_weight_config":{
               "algo":"arq_retrain",
               "channel_wise":true
           }
       }
   }

4. Configuration tuning ends if the inference accuracy meets the requirement. Otherwise, it indicates that retrain has severe adverse impact on the inference accuracy. In this case, remove the retrain configuration.