Deploying the NPU Driver and Firmware and CANN

1. Log in to the server as the root user.
2. Configure the IP address and user name of the server.

   Go to the /home/ascend-deployer-5.0.0/ascend_deployer directory and edit the inventory_file file.

   Add the server IP address and user name under [worker], and comment out or delete localhost ansible_connection='local'. Run the :wq command to save the settings and exit. The content is as follows:

   [worker]
   # localhost ansible_connection='local'
   192.168.1.2 ansible_ssh_user='root' ansible_ssh_pass=' xxx '# root user

   The preceding IP address is only an example. Replace it with the actual service IP address of the server. xxx indicates the password of the root user.

3. Go to the /home/ascend-deployer-5.0.0/ascend_deployer directory and run the following command to install the NPU driver and firmware, GCC, Python, Toolkit, and kernels:

   bash install.sh --install=sys_pkg,npu,gcc,python,toolkit,kernels

   During the installation, you need to sign the Huawei Enterprise End User License Agreement (EULA) before starting the installation process. Enter y or Y to confirm the agreement based on the command output. Enter any other character to reject the agreement. After you accept the agreement, the installation automatically starts.

   If the current language environment does not meet the requirements, run the following commands to configure the default language environment:

   # Set the language to Chinese (simplified).
   export LANG=zh_CN.UTF-8
   # Set the language to English.
   export LANG=en_US.UTF-8

   If the following information is displayed, the installation is successful:

   --install=sys_pkg,npu,gcc,python,toolkit,kernels successfully

4. Run the reboot command to reboot the server.
5. Configure environment variables.
   1. Log in to the server as the root user.
   2. Run the following commands to set environment variables:

      # Configure the Toolkit environment variables.
      source /usr/local/Ascend/ascend-toolkit/set_env.sh
      
      # Configure the environment variables of Python 3.9.2 installed using Ascend Deployer.
      export LD_LIBRARY_PATH=/usr/local/python3.9.2/lib:$LD_LIBRARY_PATH
      export PATH=/usr/local/python3.9.2/bin:$PATH
      
      # If the built-in GCC version of the OS (such as CentOS 7.6) is earlier than 7.3.0, Ascend Deployer automatically installs GCC 7.3.0. You can run the following commands to configure the GCC environment variables of GCC 7.3.0 installed by Ascend Deployer:
      export LD_LIBRARY_PATH=/usr/local/gcc7.3.0/lib64:${LD_LIBRARY_PATH}
      export CC=/usr/local/gcc7.3.0/bin/gcc
      export CXX=/usr/local/gcc7.3.0/bin/g++
      export PATH=/usr/local/gcc7.3.0/bin:${PATH}

      You can also configure permanent environment variables by modifying the ~/.bashrc file. The procedure is as follows:

      1. Run the vi ~/.bashrc command to open the .bashrc file and append the preceding line to the end of the last line in the file.
      2. Run the :wq! command to save the file and exit.
      3. Run the source ~/.bashrc command for the modification to take effect immediately.
   3. Run the following commands to check whether the environment variables are successfully configured:

      python3 -V
      gcc --version

      If the following versions are displayed, the configuration is successful:

      Python 3.9.2
      gcc (GCC) 7.3.0

   To compile a model using GCC 7.3.0 but the GCC version cannot be controlled using environment variables due to compilation script problems, perform the following operations to modify the soft link:

   1. Before modifying the soft link, check whether the gcc7.3.0 folder exists in the /usr/local directory on the server. If the folder does not exist, GCC 7.3.0 is not installed. In this case, skip the following steps:
   2. Run the following command in any directory on the server to create a soft link:

      ln -sf /usr/local/gcc7.3.0/bin/gcc /usr/bin/gcc

   3. Run the gcc --version command to check the GCC version. If the following information is displayed, the configuration is successful:

      gcc (GCC) 7.3.0

Installing PyTorch 1.11.0

1. Click here to obtain the torch package and click here to obtain the torch_npu package.
2. Upload the torch and torch_npu packages to any directory (for example, /home) on the server.
3. Install the Python dependencies.

   pip3 install pyyaml wheel typing_extensions

4. Run the following command to install torch and torch_npu:

   pip3 install xxx.whl

   xxx indicates the name of the torch or torch_npu software package.

5. Run the following command. If True is returned, PyTorch installation is successful.

   python3 -c "import torch;import torch_npu;print(torch_npu.npu.is_available())"

Installing the Acceleration Library

The Transformer inference acceleration library depends on the CANN Toolkit. Before installing the Transformer acceleration library, ensure that the Toolkit of the matching version has been installed in the current environment and related environment variables have been configured.

1. Log in to the server as the root user.
2. Run the following command to check whether to install the abi0 or abi1 software package:

   python3 -c "import torch;print(torch.compiled_with_cxx11_abi())"

   • If True is returned, install the abi1 software package.
   • If False is returned, install the abi0 software package.

3. Upload the software package of the Transformer inference acceleration library to any path (for example, /home) in the installation environment.
4. Grant the execute permission on the software package.

   # abi0
   chmod +x Ascend-cann-atb_7.0.0_linux-aarch64_abi0.run
   # abi1
   chmod +x Ascend-cann-atb_7.0.0_linux-aarch64_abi1.run

5. Install the software.

   # abi0
   ./Ascend-cann-atb_7.0.0_linux-aarch64_abi0.run --install  
   # abi1
   ./Ascend-cann-atb_7.0.0_linux-aarch64_abi1.run --install 

   If the following information is displayed, the software is successfully installed:

   xxx install success

   xxx indicates the name of the software package to be installed.

6. Set environment variables. You can modify the ~/.bashrc file to set permanent environment variables. The procedure is as follows:
   1. Run the vi ~/.bashrc command in any directory as the running user to open the .bashrc file and append the following content to the file:

      source /usr/local/Ascend/atb/set_env.sh

   2. Run the :wq! command to save the file and exit.
   3. Run the source ~/.bashrc command for the modification to take effect immediately.

Configuring transformer-llm Environment Variables

1. Upload the transformer-llm model package to any path (for example, /home/transformer-llm) on the server.
2. Run the following command in the path of the transformer-llm model package to decompress the package:

   # abi0
   tar xzf Ascend-cann-llm_7.0.0_linux-aarch64_torch1.11.0-abi0.tar.gz
   # abi1
   tar xzf Ascend-cann-llm_7.0.0_linux-aarch64_torch1.11.0-abi1.tar.gz

3. Configure environment variables.

   source /home/transformer-llm/set_env.sh