The customer has a problem with freezing of U2000 app during obtain infromation from OLT (ONT Info, synchronization etc.)
There were no changes made by our customer. He also tried revert to previous snapshot of virtual server, but there was still the same problem. Virtual server has good recources and CPU, memory is not under heavy load. U2000 is slow. Informations about ONT are loaded with 3-5 minutes delay.
OLT and U2000 SNs:
U2000 - U2000 V100R006C00SPC300
OLT -VERSION : MA5600V800R017C10
PATCH : SPH202
PRODUCT : MA5600T
We asked for SNMP logs as below:
//step1: start capture on the device where the U2000 is running
//step2: start capture on OLT
huawei(config-acl-user-3000)# rule 1 permit udp source-port eq 161
huawei(config-acl-user-3000)# rule 2 permit udp destination-port eq 161
(diagnose)%%file-server auto-backup debug primary [18.104.22.168] tftp //[22.214.171.124]: replace it with the tftp server IP
(diagnose)%%capture item 1 bidirection ip-group 3000 rule 1 port [F/S/P] //[F/S/P]: the port that the snmp packet will be transfered/received
(diagnose)%%capture item 2 bidirection ip-group 3000 rule 2 port [F/S/P] //[F/S/P]: the port that the snmp packet will be transfered/received
(diagnose)%%capture start -c 1000000 //start capture
//step3: open debugging on OLT
(su)%%dbwin print 0x46 1
(su)%%dbwin send 0x46 1 1 1
(su)%%dbwin send 0x46 1 2 1
//step4: reproduce the issue
//step5: close debugging on OLT
(su)%%dbwin send 0x46 1 2 0
(su)%%dbwin send 0x46 1 1 0
(su)%%dbwin print 0x46 0
//step6: stop capture on the device where U2000 is running
//step7: stop capture on OLT
(diagnose)%%undo file-server auto-backup debug primary
After checking the output from attached logs, the solution is as below.
1. From the captured file, it is found that the get-request message is received on OLT(it is supposed olt_mirror_20180626.pcapng is the file which is captured on the uplink port of OLT.), but no get response is sent.
A. Related messages on OLT:
B. Related messages on NMS
2. From the capture file, it is found that there are getBulkRequest messages from 10.255.245.1 every 10 seconds before the above get-request message:
And get-response for the getBulkRequest after the above get-request message, and there are 465 ONT information items returned:
1) As there are lots of ONTs, it will take much time, and 10 seconds should be not enough.
2) When getBulkRequest message is processed on OLT, the get-request message will not be processed, and no response.
3) Advice measures:
a) Extend the response timeout for getBulkRequest message;
b) Stagger the execution of getBulkRequest and get-request message.