No relevant resource is found in the selected language.

This site uses cookies. By continuing to browse the site you are agreeing to our use of cookies. Read our privacy policy>

Reminder

To have a better experience, please upgrade your IE browser.

upgrade

Enterprise Datacomm---CloudEngine 5800--- CE5855 drops

Publication Date:  2017-12-29 Views:  518 Downloads:  0
Issue Description

Drops of packets from CE5855EI to Server. Average output traffic to server less than 1% of max BW.

CE5855EI connected to Linux server by eth-trunk.

  

Alarm Information

Oct 30 2017 12:14:56
CE5855-E-5103%%01QOS/4/hwXQoSPacketsDropInterfaceAlarm_active(l): CID=0x807f042b-alarmID=0x09622019;The
interface has packets droped.
(Interface=GE1/0/35, QueueId=0, SlotId=1)
Oct 30 2017 12:14:56
CE5855-E-5103%%01QOS/4/hwXQoSPacketsDropInterfaceAlarm_active(l): CID=0x807f042b-alarmID=0x09622019;The
interface has packets droped
.(Interface=GE1/0/36, QueueId=0, SlotId=1)



Handling Process

1. Output traffic to server less than 1% of BW, but it is drops
Eth-Trunk27
    Input:
           0 errors,0 drops
    Output:
           0 errors,8126082 drops
    Last 300 seconds output utility rate: 0.68%
GE1/0/35 current state : UP
Output Flow-control:          DISABLE
  Output peak rate 30247150 bits/sec
    Output:
      Discard:                4068235,   Buffers Purged: 0
        Last 300 seconds output utility rate: 0.71%
GE1/0/36 current state : UP
Output Flow-control:          DISABLE
    Output peak rate 23990062 bits/sec
    Output:
      Discard:                4057847,   Buffers Purged: 0
    Last 300 seconds output utility rate: 0.64%
2. Buffers in not full
 <CE5855-E-5103>dis qos buffer-usage interfaceGE1/0/36
 Total        : 11150 cells (2264 KBytes)
 Current used : 0 cells (0 KBytes)
 Remained     : 11150 cells (2264 KBytes)
 Peak used    : 4245 cells (862 KBytes)
 Average used : 0 cells (0 KBytes)
  Buffer Usage on each Queue: (cells/KBytes)
-----------------------------------------------------------------------------
 QueueIndex          Current Peak               Average
-----------------------------------------------------------------------------
 0                      0/0 4245/862                0/0
3. NMS statistics (pool interval – 1 minute). Here we also don’t see any traffic bursts.

4. Need to collect packet dump at server or using port mirroring at switch, and analyze this dump by using IO Graphs tool in Wireshark.

Interval - 1 Seconds: Peak Rate:  429 Mbit/s :

 Interval - 1ms. Peak Rate:  > 1x10^6 bit/ms  =>  1G bit/s :

 

 At last graph we found ms traffic bursts with BW more than max interface BW.


Root Cause

In customer network happened traffic burst in ms interval.
Such small bursts we can not observe in real-time by using NMS or switch statistics.
Interface don’t reach and exceed peak rate interface bandwidth, because interface peak rate is statistics of 300s traffic average value, so we cannot use that value for reference.

Solution
Adjust application layer service settings;
Add more members to Eth-trunk;
Enable output flow-control if Server support it.

END