No relevant resource is found in the selected language.

This site uses cookies. By continuing to browse the site you are agreeing to our use of cookies. Read our privacy policy>Search


To have a better experience, please upgrade your IE browser.


Mirror volume is low performance on VIS6600T

Publication Date:  2016-11-27 Views:  104 Downloads:  0
Issue Description

Fault symptom: The customer defined a 5 TB volume which is based on 2 Logical Disk of 5 TB each in a mirrored configuration. The status of the mirroring operation is just at 13% 24 hours after its definition. The average mirror speed is about 7.8MB/s.

Alarm Information
No alarm found on VIS6600T and backend storage.
Handling Process

There're two possibilities may cause this issue, low performance on backend storage and VIS limited performance on mirror task.

1. Execute “vxdctl -c mode” in VIS cli command line and check the last line of the command output. there’s a segment called “MASTER-Node name” and the last bit of Node name is the node ID of Master node. Login Master node and execute “vxtask list -l” to get all the running task status.

vxtasklist       k list -l
Task:      167772202 RUNNING
Type:      ATCOPY
Started:   Tue Oct 25 16:37:10 2016
Throttle:  100
Progress:  38.99% 4186828800 of 10737418240 Blocks
Work time: 2 days, 16 hours, 49 minutes, 24 seconds (101:25:14 remaining)

As the command output, the mirror task is in speed limitation mode, and the throttle is 100%. So, this is the root cause.

2. Execute command in VIS CLI “vxdmpadm listenclosure” to check backend storage model.

vxdmpadm listenclosure
other_disks       OTHER_DISKS    OTHER_DISKS          CONNECTED    OTHER_DISKS   2
huawei-vis6000-1  HUAWEI-VIS6000-  xxxxxxxxxxxxxxxxx     CONNECTED    A/A        128
huawei-s5500t0    HUAWEI-S5500T  xxxxxxxxxxxxxxxxx    CONNECTED    A/A-A-HUAWEI  44
huawei-s5500t1    HUAWEI-S5500T  xxxxxxxxxxxxxxxxx    CONNECTED    A/A-A-HUAWEI  41
huawei-s5500t2    HUAWEI-S5500T  xxxxxxxxxxxxxxxxx    CONNECTED    A/A-A-HUAWEI  57
huawei-s5500t3    HUAWEI-S5500T  xxxxxxxxxxxxxxxxx    CONNECTED    A/A-A-HUAWEI  54
huawei-s5500t4    HUAWEI-S5500T  xxxxxxxxxxxxxxxxx    CONNECTED    ALUA       114
huawei-s5500t5    HUAWEI-S5500T  xxxxxxxxxxxxxxxxx    CONNECTED    ALUA       112

3. Login Master node of VIS, execute “developer” to enter developer mode. Then input “minisystem” to login minisystem mode.
Execute “iostat -dx 3 40” to monitoring the disk performance on VIS.

iostat -dx 3 40
Linux (VIS_7039_0) 10/28/16 _x86_64_(12 CPU)

Device:         rrqm/s   wrqm/s     r/s     w/s    rkB/s    wkB/s avgrq-sz avgqu-sz   await r_await w_await  svctm  %util
sdea              0.00     0.00  169.90   17.83  3097.67   655.40    39.98     0.42    2.23    2.27    1.83   1.57  29.52

Root Cause
VIS will start performance limitation on mirror speed when backend storage has bottleneck or customer setting. We can execute "vxtask list -l" and check the throttle value to confirm it.

1. Login master node and execute command “vx_tune_mir_mode off ”

2. Execute “vxtask set slow=0 167772202” to disable the speed limitation. Please note, 167772202 represent to the mirror task ID, need to be replaced as the real task ID.