Slow Data Copy Speed Between VMs

Publication Date:  2016-03-28 Views:  592 Downloads:  0
Issue Description
The service system server was deployed on VM A. When a user accessed the web page of VM A using a browser from VM B and uploaded data to VM A, the data upload speed was slow.

When a user copied data directly from VM A to VM B, the data copy speed was 7 MB/s. However, in normal conditions, data copy speed between VMs in the same local area network (LAN) can be dozens of MB/s.

Figure 1-1


Handling Process
1. After running the command to check data read and write speed between the host where the VMs are running and the shared storage device used by the host, we found that the data read and write speed is normal.

Figure 1-2 



Figure 1-3



2. After checking the network communication between VM A and VM B using JPerf, we found that network bandwidth values fluctuated a lot and the packet loss rate was high.

Figure 1-4



Figure 1-5



3. Then we concluded that the problem was caused by network exceptions. Then we attempted to locate the problem by checking the data packet transfer path from VM A to VM B. The data packet transfer path is VM A > Domain 0 of the host where VM A is running > physical NIC of the host where VM A is running > switch module > physical NIC of the host where VM B is running > Domain 0 of the host where VM B is running > VM B.

4. We first checked the connection between host where VM A is running and that where VM B is running. We pinged the IP address of the host where VM B is running from the host where VM A is running. It is found that network communication is stable and no network jitter occurred.

CNA13:~ # ping 192.168.8.19
PING 192.168.183.13 (192.168.183.13) 56(84) bytes of data.
64 bytes from 192.168.183.13: icmp_seq=1 ttl=64 time=0.238 ms
64 bytes from 192.168.183.13: icmp_seq=2 ttl=64 time=0.233 ms
64 bytes from 192.168.183.13: icmp_seq=3 ttl=64 time=0.236 ms
64 bytes from 192.168.183.13: icmp_seq=4 ttl=64 time=0.239 ms

5. Then we checked the connection between VM A and the host where VM A is running. We found that the version of the paravirtualized (PV) driver installed on VM A is 1.1.5.28, but the PV driver version of Domain 0 of the host where VM A is running is 1.2.0.42.

6. After we upgraded the PV driver of VM A, data copy speed from VM A to VM B was significantly improved. After we upgraded the PV driver of VM B, the data copy speed between the two VMs fell back to the normal range.
Solution
Upgrade PV driver version of the VMs to that of the host Domain 0 using either of the following methods:
     When a small number of VMs are running on the host, manually install the PV drivers on the VMs. For details, see Operation and Maintenance > Virtual Machine Management > Tools Management > Uninstalling the Tools in the FusionCompute Product Documentation.

     When a large number of VMs are running on the host, use the upgrade tool to upgrade the VM PV drivers. For details, see the FusionCompute V100R003C00SPC300 Upgrade Guide at
http://support.huawei.com/enterprise/docinforeader.action?contentId=DOC1000028007&idPath=7919749|7919788|9856606|8576912

Note:
The VM PV driver upgrade takes effect after the VMs are restarted. Therefore, inform users of the service interruption before perform the upgrade.

END