npu_bridge.estimator.npu.npu_loss_scale_manager
FixedLossScaleManager Constructor
Prototype
def __init__(self, loss_scale)
Description
Constructor of the FixedLossScaleManager class, which is used to define static loss scale parameters.
Parameters
Parameter |
Input/Output |
Description |
|---|---|---|
loss_scale |
Input |
Loss scale value. The value is of the float type and cannot be less than 1. If the value of loss scale is too small, model convergence may be affected. If the value of loss scale is too large, overflow may occur during training. The value can be the same as that of GPU. |
Returns
An object of the FixedLossScaleManager class
ExponentialUpdateLossScaleManager Constructor
Prototype
def __init__(self,
init_loss_scale,
incr_every_n_steps,
decr_every_n_nan_or_inf=2,
incr_ratio=2,
decr_ratio=0.8)
Description
Constructor of the ExponentialUpdateLossScaleManager class, which is used to define dynamic loss scale parameters.
Parameters
Parameter |
Input/Output |
Description |
|---|---|---|
init_loss_scale |
Input |
Initial loss scale value. A float. |
incr_every_n_steps |
Input |
If no overflow occurs for N iterations, increase the value of loss scale. |
decr_every_n_nan_or_inf |
Input |
If an overflow occurs for N iterations, decrease the value of loss scale. Defaults to 2. |
incr_ratio |
Input |
Percentage increase of loss scale. Defaults to 2. |
decr_ratio |
Input |
Percentage decrease of loss scale. Defaults to 0.8. |
Returns
An object of the ExponentialUpdateLossScaleManager class