No relevant resource is found in the selected language.

This site uses cookies. By continuing to browse the site you are agreeing to our use of cookies. Read our privacy policy>

Reminder

To have a better experience, please upgrade your IE browser.

upgrade

Short circuit of one pic leaded to all the pics went down.

Publication Date:  2014-12-12 Views:  60 Downloads:  0
Issue Description

 

Product Name

Version

CX600-X2

CX600 V600R007C00SPC300

 

The PIC4 on the CX600-X2 device could no register ,so engineer decide to change it with a spare part, but after the new pic insert on slot 4 ,there was a Crackling noise and suddenly , the interfaces on all the pics were down.


 

Handling Process

After the above new pic plug in , all of the pics on chassis became fault ,the record was as following.

Nov 27 2014 21:10:43+02:00 CX600-X2 %%01SRM/2/NODEFAULT(l)[61919]:Slot=7;PIC3 of LPU 7 is failed, perhaps 33M_Exist of CLOCK is abnormal. (Reason="P4CFA-FPGA: 33M Clock Test Error.")

Nov 27 2014 21:10:43+02:00 CX600-X2 %%01SRM/2/NODEFAULT(l)[61920]:Slot=7;PIC3 of LPU 7 is failed, perhaps VCO38M_Exist of CLOCK is abnormal. (Reason="P4CFA-FPGA: VCO38.88M Clock Test Error.")

Nov 27 2014 21:10:43+02:00 CX600-X2 %%01SRM/2/NODEFAULT(l)[61921]:Slot=7;PIC3 of LPU 7 is failed, perhaps SGMII_Exist of CLOCK is abnormal. (Reason="P4CFA-FPGA: 125M Clock Test Error.")

Nov 27 2014 21:10:43+02:00 CX600-X2 %%01SRM/2/NODEFAULT(l)[61922]:Slot=7;PIC3 of LPU 7 is failed, perhaps 77M_Exist of CLOCK is abnormal. (Reason="P4CFA-FPGA: 77M Clock Test Error.")

Nov 27 2014 21:10:43+02:00 CX600-X2 %%01SRM/2/NODEFAULT(l)[61923]:Slot=7;PIC3 of LPU 7 is failed, perhaps 19M_Exist of CLOCK is abnormal. (Reason="P4CFA-FPGA: 19M Clock Test Error.")

Nov 27 2014 21:10:43+02:00 CX600-X2 %%01SRM/2/NODEFAULT(l)[61924]:Slot=7;PIC3 of LPU 7 is failed, perhaps FRAME_PL3 of DataLink is abnormal. (Reason="P4CFA-Frame: PL3_LINERX Interface Test Error.")

Nov 27 2014 21:10:43+02:00 CX600-X2 %%01SRM/2/NODEFAULT(l)[61925]:Slot=7;PIC5 of LPU 7 is failed, perhaps RW_TEST of PIC[5] FPGA is abnormal. (Reason="PIC 5 FPGA TEST :ERROR!")

Nov 27 2014 21:10:46+02:00 CX600-X2 %%01SRM/2/NODEFAULT(l)[61981]:Slot=8;PIC9 of LPU 8 is failed, perhaps RW_TEST of PIC[9] FPGA is abnormal. (Reason="SelfTest[4] is failed")

Nov 27 2014 21:11:02+02:00 CX600-X2 %%01SRM/2/NODEFAULT(l)[62042]:Slot=8;PIC9 of LPU 8 is failed, perhaps RW_TEST of PIC[9] FPGA is abnormal. (Reason="SelfTest[11] is failed")

For the above fault , take pic7/3 for example ,all the test items on the pic became fault, there must be a fault on the power system on pic7/3. As all the pics became fault at the same time ,so there must be a faulty on the together-power system.

Above is the power supply system of the X2 device ,we can see that the NPUs were supplied by two paths of 12V and the pics was supplied by two path of 48V . there must be something wrong with the two paths of 48V input on each pics, which leaded to the fault alarms. as the 12Vwas separated form the 48V ,the NPUs were not affected.

Pulled out the spare part of pic4 ,we find the power connector on the board was Blackened

Just as the following picture shows. It means that there was a big current on the connecter, which breakdown the 48V fuse on both of the PSU_A and PSU_B. then 48V_1 and 48V_2 to all the pics were down. so all the pics stopped to work ,all the interface physically down and could not register anymore;

After that , the traffic was restored after change the PSU power boards.

In order to find out werther the backplane was fault ,we got the following picture, it was the connector on the backplane ,which looked ok.

So form above picture ,we can almost arrive at the short cut point was not on the backplane. So the short cut point should be on the pic4 ,when the board was pluged in ,

In order to make sue ,we tried to find out where was the short point. We made the following test on the short cut pic4. Comparing the test result ,we found all the test result were the same as in the lab.

 

 

So we coulds arrive at the short cut was not on the PCB of the pic4.  The short cut point must be on the power connector of the pic4.

As the short cut point was found ,which had no relationship with the backplane ,so we change the fault pic with a new one ,after that ,all resumed.

Root Cause
There was a short cut point on the power connector of the pic4 spare part ,when the pic was pluged in slot 4 , the sudden big current of 48V breakdown the 48V fuse on both of the PSU_A and PSU_B. then 48V_1 and 48V_2 to all the pics were down. so all the pics stopped to work ,all the interface physically down and could not register anymore. The sudden big current also Blackened the pic4’s power connector.
Solution

PLS Change the PIC4 with a new spare part. And pls send the faulty pic to R&D for deep analyzing .

END