1. Log in to the SMM on the CLI, and run the df -ah command to check the current flash usage for the SMM. The root directory usage is 85%, as shown in Figure 2. The flash alarm threshold is 85%, which is the alarm threshold.
Figure 2 Viewing the current flash usage
2. Run the du -sh * command in the root directory to view the sizes of folders and files in the root directory, as shown in Figure 3.
Figure 3 Sizes of files for a faulty SMM
3. Find an SMM for which no alarm on the too high flash usage exists, and view the sizes of folders and files in the root directory by using the same method, as shown in Figure 4.
Figure 4 Sizes of files for a normal SMM
4. Compare the sizes of folders and files in the root directory between two SMMs, and find that the flash usages for each folder and files are the same, and no abnormal file exists.
5.. Since no abnormal file exists, the largest possibility is that a process constantly runs, and resources are not released in time. Processes run on the SMM are as follows:
Component monitoring and log output processes include vpem.out, vfan.out, and vnem.out.
Processes related to the web process include httpd and snmpd.
6. Run the top command to view the CPU usage, and find that the load average is lower than 5, and the CPU usage is lower than 80%, indicating that the CPU load is small, as shown in Figure 5. No abnormal process exists based on the CPU usage. You need to use other methods to check all processes.
Figure 5 CPU usage
7. To check key processes, run the ps-ef command to query the program IDs (PIDs) of the monitoring processes for fans, power supplies, and switch modules, as shown in Figure 6. Numbers in red boxes indicate PIDs.
Only an active SMM can be used to query the monitoring processes of fans, power supplies, and switch blades.
Figure 6 PIDs of fans, power supplies, and switch blades
8. Run the kill command to end the monitoring processes of fans, power supplies, and switch blades. Then run the df-ah command to check that the root directory usage is 85%.
9. After you use the same method to end the httpd process, the root directory usage is 85%.
10. When you use the same method to end the snmpd process, the root directory usage is reduced from 85% to 75%. After running the df-ah command repeatedly, find that the flash usage for the SMM gradually falls to 64% (about 65% in normal condition), as shown in Figure 7. Therefore, the snmpd process is abnormal.
Figure 7 Flash usage for the SMM
11. Simulate the problem by using fault injection. By using a dedicated software tool, a large number of web files are found opened and then closed; however, the snmpd process does not release occupied flash resources in time.