Huawei uses machine translation combined with human proofreading to translate this document to different languages in order to help you better understand the content of this document.
Note: Even the most advanced machine translation cannot match the quality of professional translators.
Huawei shall not bear any responsibility for translation accuracy and it is recommended that you refer to the English document (a link for which has been provided).
Overview
In a large-scale AI training cluster, training is usually completed in a data-parallel manner. Data parallelism means that all devices use the same model and different training samples, and gradient data calculated by each device needs to be aggregated for parameter update.
Figure 4-1 Schematic diagram of data-parallel training
![]()
If classification is performed in gradient aggregation manner, the mainstream implementation of data parallelism includes the Parameter Server-Workers (PS-Worker) architecture and AllReduce collective communication. The Ascend platform supports both implementation types. For details, see Distributed Training Based on the AllReduce Architecture and Distributed Training Based on the PS-Worker Architecture.