No relevant resource is found in the selected language.

This site uses cookies. By continuing to browse the site you are agreeing to our use of cookies. Read our privacy policy>

Reminder

To have a better experience, please upgrade your IE browser.

upgrade

Huge Latency Performance issue

Publication Date:  2018-02-28 Views:  69 Downloads:  7
Issue Description

A huge latency occur this morning (10:00 am) on one of our storage that cause all servers connected to the storage to have more than 1seconde latency.
The Storage seemed to freeze with a latency of 1000ms on all LUN (Average of 8000 MS)

Handling Process

First of all when we talk about latency issues we need to collect and analyze the performance logs from that storage

 

 

Root Cause

Operations consumed to much cache Structure and the Hosts needed extremply long time to allocate the cache structures

Solution

After we analyzed the event.txt log we found a lot of activating/deactivating the snapshot activity before the issue occurred, print screen attached

Moving forward we found some operations that consumed a lot of cache Structure and the host IO need extremely long to allocate the cache structures, print screen attached

At the end finally many IOs were timeout and hosts sent abort commands to cancel the IOs

END