Providing System OOM Information

Function Introduction

Due to special causes, the memory of the product or platform service software or OS may be insufficient, resulting in the OOM event. The kbox can record the OOM event information such as the time of occurrence, information about the process that encounters the OOM event, and system process information, in the storage device, and therefore facilitating fault locating.

Information List

The recorded OOM information includes three parts.

1. OOM exception information. Up to 256 KB information can be recorded. This area mainly records the following information:
   • Time of occurrence (UTC time)
   • PID and name of the current process, kernel version, and CPU number
   • Track of the called abnormal process and stack information (up to 150 lines of the stack information can be output)
   • Memory statistics
   • Slab allocation information
   • Information about current processes in the system
   • Vmalloc information
   • File information of the memory file systems such as rootfs, ramfs, and tmpfs
   • Information about memory resources consumed by files in the memory file systems such as rootfs, ramfs, and tmpfs
   • Actions after the OOM event (0: no operation; 1: calling the panic function)
2. Message output logs of the kernel. Up to 64 KB information can be recorded. This area mainly records the following information:
   • Time of occurrence (UTC time)
   • 64 KB of latest logs generated when the exception occurs, that is, the last 64 KB logs output by the kernel to the circular buffer
3. Message output logs of the console. Up to 32 KB information can be recorded. This area mainly records the following information:
   • Time of occurrence (UTC time)
   • Message output logs of other kernels

Examples

A log file includes the following information:

1. OOM exception information

   //OOM exception storage area, the current OOM log is located in the OOM exception storage area with index 1
   area type: 0-panic 1-reboot 2-emerge 3-intermit 4-oom 5-oops/die 6-rlock 7-message 8-console 
   ---------------area type:4, record num:1, unit_size:64 ------------
   
   
   ###num:0, record len:83407
   ---------KBOX_START----------
   //Time when kernel panic occurs. The time is in UTC format. Beijing time = UTC time + 8 hours
   oom time:20160218155821-e0387
   //PID and name of the current process that triggers the OOM process, kernel version, and CPU number
   Current Pid: 44, comm: kworker/1:1 Tainted: G           O E ---- -------   3.10.0-229.20.1.22.hulk.x86_64 #1
   //Calling track of abnormal process functions
   Call Trace:
    [<ffffffffa037ca53>] kbox_dump_oom+0x13/0x30 [kbox]
    [<ffffffffa037bb17>] kbox_oom_callback+0x107/0x210 [kbox]
    [<ffffffff810a8c20>] ? wake_up_state+0x20/0x20
    [<ffffffff8109ffe8>] ? __wake_up_common+0x58/0x90
    [<ffffffff8160a2fc>] notifier_call_chain+0x4c/0x70
    [<ffffffff8109d07d>] __blocking_notifier_call_chain+0x4d/0x70
    [<ffffffff8109d0b6>] blocking_notifier_call_chain+0x16/0x20
    [<ffffffff81159e6e>] out_of_memory+0x4e/0x4f0
    [<ffffffff81390b1c>] ? tty_ldisc_deref+0x3c/0xa0
    [<ffffffff81394fad>] moom_callback+0x4d/0x50
    [<ffffffff8108f445>] process_one_work+0x175/0x430
    [<ffffffff8109028b>] worker_thread+0x11b/0x3a0
    [<ffffffff81090170>] ? manage_workers.isra.25+0x2a0/0x2a0
    [<ffffffff81096fbf>] kthread+0xcf/0xe0
    [<ffffffff81096ef0>] ? insert_kthread_work+0x40/0x40
    [<ffffffff8160e858>] ret_from_fork+0x58/0x90
    [<ffffffff81096ef0>] ? insert_kthread_work+0x40/0x40
   oom stack:
    ffff8801385bfbf0 ffff8801385bfc48 ffff8801385bfc6c 0000000000000000
    ffff8801385bfdc8 0000000000000000 ffff8801385bfc48 ffffffffa037c04a
    0000000000000002 0000000045746160 0000000000000000 ffff8801385bfc58
    ffffffffa037ca53 ffff8801385bfce8 ffffffffa037bb17 36313032810a8c32
    3835353138313230 37383330652d3132 0000000000000000 ffff880035ec2280
    ffffffff810a8c20 0000000000000000 0000000000000000 0000000045746160
    ffff8801385bfcf8 ffffffff8109ffe8 0000000100000000 0000000000000000
    0000000045746160 00000000ffffffff 0000000000000000 ffff8801385bfd20
    ffffffff8160a2fc ffffffff81977f80 0000000000000000 0000000000000000
    ffff8801385bfdc8 00000000ffffffff ffff8801385bfd60 ffffffff8109d07d
    0000000000000000 0000000000000000 0000000000000001 ffff88013ec92f00
    ffff88013efd8300 00000000000000d0 ffff8801385bfd70 ffffffff8109d0b6
    ffff8801385bfe08 ffffffff81159e6e 00000000fffc4c8b 0000000000000000
    ffff8801385bfde0 0000000000000286 0000000000000286 0000000000000286
    ffff8801385bfdd0 ffffffff81390b1c 0000000000000282 0000000000000000
    ffff8801385bfe18 0000000045746160 ffffffff819bd9a0 ffff88013846a000
    ffff88013ec92f00 ffff88013ec96f00 0000000000000040 ffff8801385bfe18
    ffffffff81394fad ffff8801385bfe60 ffffffff8108f445 000000003ec92f00
    0000000000000000 ffff88013ec92f00 ffff88013ec92f18 ffff88013846a030
    ffff880138589700 ffff88013846a000 ffff8801385bfec0 ffffffff8109028b
    ffff8801385bffd8 ffff8801385bffd8 ffff880138589700 ffff880138589700
    ffff880138589700 ffff880138bafd38 ffff88013846a000 ffffffff81090170
    0000000000000000 0000000000000000 ffff8801385bff48 ffffffff81096fbf
    0000000000000000 0000000000000000 ffff88013846a000 0000000000000000
    0000000000000000 ffff8801385bfef8 ffff8801385bfef8 ffff880100000000
    ffff880100000000 ffff8801385bff18 ffff8801385bff18 0000000045746160
    ffffffff81096ef0 0000000000000000 0000000000000000 ffff880138bafd38
    ffffffff8160e858 0000000000000000 0000000000000000 0000000000000000
    0000000000000000 ffff880138bafd38 ffffffff81096ef0 0000000000000000
    0000000000000000 0000000000000000 0000000000000000 0000000000000000
    0000000000000000 0000000000000000 0000000000000000 0000000000000000
    ffffffffffffffff 0000000000000000 0000000000000010 0000000000000202
    ffff8801385bff58 0000000000000018
   //Memory statistics
   *******  meminfo  *******
   MemTotal:     3768752 kB
   MemFree:     3522112 kB
   TotalHigh:          0 kB
   FreeHigh:           0 kB
   Buffers:         764 kB
   SwapTotal:     4063228 kB
   SwapFree:     4063228 kB
   //Slab allocation
   *******  slabinfo  *******
   # name            <active_objs> <num_objs> <objsize> <objperslab> <pagesperslab>: tunables <limit> <batchcount> <sharedfactor>: slabdata <active_slabs> <num_slabs> <sharedavail>
   nf_conntrack_ffffffff819dcb40     50     50    320   25    2 : tunables    0    0    0 : slabdata      2      2      0
   xfs_icr                        0      0    144   28    1 : tunables    0    0    0 : slabdata      0      0      0
   xfs_ili                      104    104    152   26    1 : tunables    0    0    0 : slabdata      4      4      0
   xfs_inode                   1760   1760   1024   16    4 : tunables    0    0    0 : slabdata    110    110      0
   xfs_efd_item                  80     80    400   20    2 : tunables    0    0    0 : slabdata      4      4      0
   xfs_da_state                  64     64    488   16    2 : tunables    0    0    0 : slabdata      4      4      0
   xfs_btree_cur                 76     76    208   19    1 : tunables    0    0    0 : slabdata      4      4      0
   xfs_log_ticket                88     88    184   22    1 : tunables    0    0    0 : slabdata      4      4      0
   scsi_cmd_cache                72     72    448   18    2 : tunables    0    0    0 : slabdata      4      4      0
   kcopyd_job                     0      0   3312    9    8 : tunables    0    0    0 : slabdata      0      0      0
   dm_uevent                      0      0   2608   12    8 : tunables    0    0    0 : slabdata      0      0      0
   dm_rq_target_io                0      0    424   19    2 : tunables    0    0    0 : slabdata      0      0      0
   UDPLITEv6                      0      0   1152   28    8 : tunables    0    0    0 : slabdata      0      0      0
   UDPv6                         84     84   1152   28    8 : tunables    0    0    0 : slabdata      3      3      0
   tw_sock_TCPv6                  0      0    256   16    1 : tunables    0    0    0 : slabdata      0      0      0
   TCPv6                         16     16   2048   16    8 : tunables    0    0    0 : slabdata      1      1      0
   cfq_queue                    102    102    232   17    1 : tunables    0    0    0 : slabdata      6      6      0
   bsg_cmd                        0      0    312   26    2 : tunables    0    0    0 : slabdata      0      0      0
   mqueue_inode_cache            18     18    896   18    4 : tunables    0    0    0 : slabdata      1      1      0
   hugetlbfs_inode_cache         52     52    608   26    4 : tunables    0    0    0 : slabdata      2      2      0
   configfs_dir_cache             0      0     88   46    1 : tunables    0    0    0 : slabdata      0      0      0
   dquot                          0      0    256   16    1 : tunables    0    0    0 : slabdata      0      0      0
   kioctx                         0      0    576   28    4 : tunables    0    0    0 : slabdata      0      0      0
   pid_namespace                  0      0   2176   15    8 : tunables    0    0    0 : slabdata      0      0      0
   posix_timers_cache             0      0    248   16    1 : tunables    0    0    0 : slabdata      0      0      0
   UDP-Lite                       0      0   1024   16    4 : tunables    0    0    0 : slabdata      0      0      0
   ......
   
   //Process information of the system at that time
   *******  kbox show all tasks  *******
   [ pid ]   uid  ppid  tgid total_vm      rss cpu stat name
   [    1]     0     0     1    14373     1847   3    S                          systemd
   [  482]     0     1   482    10753      663   1    R                  systemd-journal
   [  497]     0     1   497    12521     1073   1    S                          lvmetad
   [  503]     0     1   503    10676      546   1    S                    systemd-udevd
   [  585]     0     1   585    29179      406   0    S                           auditd
   [  607]     0     1   607    66139     5887   0    S                        firewalld
   [  612]     0     1   612    35618      693   3    S                         rsyslogd
   [  614]     0     1   614    88477     4062   0    S                            tuned
   [  615]     0     1   615     4787      289   3    S                       irqbalance
   [  616]     0     1   616     8677      419   0    S                   systemd-logind
   [  617]    81     1   617     8738      450   1    S                      dbus-daemon
   [  625]     0     1   625    31582      414   2    S                            crond
   [  630]     0     1   630     1628       85   0    S                           mcelog
   [  633]     0     1   633    27507      198   2    S                           agetty
   [  776]     0     1   776    73003     1861   1    S                   NetworkManager
   [  873]   999     1   873    46674     2780   3    S                          polkitd
   [  913]     0     1   913    13262      673   0    S                   wpa_supplicant
   [ 1227]     0     1  1227    16627      814   0    S                             sshd
   [ 2436]     0  1227  2436    31363     1154   2    S                             sshd
   [ 2440]     0  2436  2440    28844      473   0    S                             bash
   [ 2472]     0  2440  2472    28844      196   2    R                             bash
   total real task number: 21
   *****************Start oom extend info.*****************
   Vmallocinfo Start >>>>>>>>>>>>>>>>>>>>//通过内核接口申请的内存
   0xffffc90000000000-0xffffc90000002000       8192 hpet_enable+0x2d/0x2e9 phys=0xfed00000 ioremap
   0xffffc90000002000-0xffffc90000403000    4198400 alloc_large_system_hash+0x171/0x239 pages=1024 vmalloc vpages
   0xffffc90000403000-0xffffc90000406000      12288 alloc_large_system_hash+0x171/0x239 pages=2 vmalloc
   0xffffc90000406000-0xffffc90000607000    2101248 alloc_large_system_hash+0x171/0x239 pages=512 vmalloc
   0xffffc90000608000-0xffffc9000060b000      12288 acpi_os_map_memory+0xea/0x142 phys=0xbffdf000 ioremap
   0xffffc9000060c000-0xffffc9000060f000      12288 acpi_os_map_memory+0xea/0x142 phys=0xbffe0000 ioremap
   0xffffc90000610000-0xffffc90000612000       8192 acpi_os_map_memory+0xea/0x142 phys=0xfed00000 ioremap
   0xffffc90000612000-0xffffc90000653000     266240 alloc_large_system_hash+0x171/0x239 pages=64 vmalloc
   0xffffc90000653000-0xffffc900006d4000     528384 alloc_large_system_hash+0x171/0x239 pages=128 vmalloc
   0xffffc900006d4000-0xffffc900006e5000      69632 alloc_large_system_hash+0x171/0x239 pages=16 vmalloc
   0xffffc900006e5000-0xffffc900006f6000      69632 alloc_large_system_hash+0x171/0x239 pages=16 vmalloc
   0xffffc900006f6000-0xffffc90000707000      69632 alloc_large_system_hash+0x171/0x239 pages=16 vmalloc
   0xffffc90000708000-0xffffc9000070a000       8192 cirrus_device_init+0x86/0x130 [cirrus] phys=0xfebf4000 ioremap
   0xffffc9000070a000-0xffffc9000070c000       8192 dm_table_create+0x9e/0x130 [dm_mod] pages=1 vmalloc
   0xffffc9000070c000-0xffffc9000072d000     135168 raw_init+0x41/0x141 pages=32 vmalloc
   0xffffc9000072d000-0xffffc90000736000      36864 drm_ht_create+0x55/0x80 [drm] pages=8 vmalloc
   0xffffc90000736000-0xffffc90000738000       8192 dm_table_create+0x9e/0x130 [dm_mod] pages=1 vmalloc
   0xffffc90000739000-0xffffc9000073c000      12288 cirrusfb_create+0xde/0x390 [cirrus] pages=2 vmalloc
   0xffffc9000073c000-0xffffc9000073e000       8192 swap_cgroup_swapon+0x48/0x160 pages=1 vmalloc
   0xffffc9000073e000-0xffffc90000740000       8192 ebt_register_table+0xa2/0x380 [ebtables] pages=1 vmalloc
   0xffffc90000740000-0xffffc90000761000     135168 pci_ioremap_bar+0x46/0x80 phys=0xfebc0000 ioremap
   0xffffc90000761000-0xffffc900009a2000    2363392 cirrusfb_create+0xde/0x390 [cirrus] pages=576 vmalloc vpages
   ......
   Vmallocinfo End <<<<<<<<<<<<<<<<<<<<
   
   //Memory File System
   Filesystem            1K-blocks    Used   Available Use(%)   Mounted on
   tmpfs                  1884376        0  1884376        0%   /dev/shm  
   tmpfs                  1884376     8504  1875872        0%   /run      
   tmpfs                  1884376        0  1884376        0%   /sys/fs/cgroup
   ********    mem info     ********?Total:                3768752 kB
   ?Total free:           3521908 kB
   ?User space:            157772 kB
   ?Mlock:                      0 kB
   ?Kernel space:           89072 kB
   ?Bootmem reserved:      409164 kB
   ?HugePages_Total:            0 kB
   ?HugePages_Free:             0 kB
   ?HugePages_Rsvd:             0 kB
   ?HugePages_Surp:             0 kB
   ?Hugepagesize:            2048 kB
   //Files that occupy memory in descending order
   ******     pagecache_info:     ******
     /run/log/journal/ad0c8c7dc64c4ecb860400450d3d0825/system.journal : nrpages = 1971.
     /run/udev/data/c116:33 : nrpages = 1.
     /run/udev/data/c116:3 : nrpages = 1.
     /run/udev/data/b8:2 : nrpages = 1.
     /run/systemd/system/session-1.scope.d/90-SendSIGHUP.conf : nrpages = 1.
     /run/udev/data/b253:1 : nrpages = 1.
     /run/mcelog.pid : nrpages = 1.
     /run/udev/data/n2 : nrpages = 1.
     /run/systemd/system/session-1.scope.d/90-Slice.conf : nrpages = 1.
     /run/udev/data/b8:1 : nrpages = 1.
     /run/syslogd.pid : nrpages = 1.
     /run/udev/data/b8:0 : nrpages = 1.
     /unknown : nrpages = 1.
     /run/udev/data/b11:0 : nrpages = 1.
     /run/crond.pid : nrpages = 1.
     /run/udev/data/c13:33 : nrpages = 1.
     /run/systemd/system/session-1.scope : nrpages = 1.
     /run/udev/data/c13:67 : nrpages = 1.
     /run/auditd.pid : nrpages = 1.
     ......
   *****************End oom extend info.*****************
   //Operations After OOM Exceptions Are Recorded (0: return the call chain; 1: call panic)
   action after oom is:1
   ---------KBOX_END------------

2. Message output logs of the kernel

   ---------------area type:7, record num:1, unit_size:64 ------------
   
   
   ###num:0, record len:39434
   oom time:20160218155821-e0387
   [    0.     0] Initializing cgroup subsys cpuset
   [    0.     0] Initializing cgroup subsys cpu
   [    0.     0] Initializing cgroup subsys cpuacct
   [    0.     0] Linux version 3.10.0-229.20.1.22.hulk.x86_64 (abuild@HGH1000006709) (gcc version 4.8.3 20140911 (EulerOS 4.8.3-10) (GCC) ) #1 SMP Thu Feb 4 14:30:09 UTC 2016
   [    0.     0] Command line: BOOT_IMAGE=/vmlinuz-3.10.0-229.20.1.22.hulk.x86_64 root=/dev/mapper/euleros-root ro reserve_kbox_mem=16M crash_kexec_post_notifiers panic=3 8250.nr_uarts=8 efi=old_map rd.lvm.lv=euleros/swap crashkernel=2G-32G:256M,32G-1T:512M,1T-8T:1G,8T-:4G rd.lvm.lv=euleros/root rhgb quiet
   [    0.     0] e820: BIOS-provided physical RAM map:
   [    0.     0] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable
   [    0.     0] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved
   [    0.     0] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved
   [    0.     0] BIOS-e820: [mem 0x0000000000100000-0x00000000bffdefff] usable
   [    0.     0] BIOS-e820: [mem 0x00000000bffdf000-0x00000000bfffffff] reserved
   [    0.     0] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved
   [    0.     0] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved
   [    0.     0] BIOS-e820: [mem 0x0000000100000000-0x000000013fffffff] usable
   [    0.     0] e820: reserve kbox memory size: 16M
   [    0.     0] e820: reserve kbox memory success. [mem 0x000000013f000000-0x000000013fffffff]
   [    0.     0] NX (Execute Disable) protection: active
   [    0.     0] e820: user-defined physical RAM map:
   [    0.     0] user: [mem 0x0000000000000000-0x000000000009fbff] usable
   [    0.     0] user: [mem 0x000000000009fc00-0x000000000009ffff] reserved
   [    0.     0] user: [mem 0x00000000000f0000-0x00000000000fffff] reserved
   [    0.     0] user: [mem 0x0000000000100000-0x00000000bffdefff] usable
   [    0.     0] user: [mem 0x00000000bffdf000-0x00000000bfffffff] reserved
   [    0.     0] user: [mem 0x00000000feffc000-0x00000000feffffff] reserved
   [    0.     0] user: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved
   [    0.     0] user: [mem 0x0000000100000000-0x000000013effffff] usable
   [    0.     0] user: [mem 0x000000013f000000-0x000000013fffffff] reserved
   [    0.     0] SMBIOS 2.8 present.
   [    0.     0] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.7.5-0-ge51488c-20140602_164612-nilsson.home.kraxel.org 04/01/2014
   [    0.     0] Hypervisor detected: KVM
   [    0.     0] e820: update [mem 0x00000000-0x00000fff] usable ==> reserved
   [    0.     0] e820: remove [mem 0x000a0000-0x000fffff] usable
   [    0.     0] No AGP bridge found
   [    0.     0] e820: last_pfn = 0x13f000 max_arch_pfn = 0x400000000
   [    0.     0] MTRR default type: write-back
   [    0.     0] MTRR fixed ranges enabled:
   [    0.     0]   00000-9FFFF write-back
   [    0.     0]   A0000-BFFFF uncachable
   [    0.     0]   C0000-FFFFF write-protect
   [    0.     0] MTRR variable ranges enabled:
   [    0.     0]   0 base 000C0000000 mask 3FFC0000000 uncachable
   [    0.     0]   1 disabled
   [    0.     0]   2 disabled
   [    0.     0]   3 disabled
   [    0.     0]   4 disabled
   [    0.     0]   5 disabled
   [    0.     0]   6 disabled
   [    0.     0]   7 disabled
   [    0.     0] PAT not supported by CPU.
   [    0.     0] e820: last_pfn = 0xbffdf max_arch_pfn = 0x400000000
   [    0.     0] found SMP MP-table at [mem 0x000f0e50-0x000f0e5f] mapped at [ffff8800000f0e50]
   [    0.     0] Base memory trampoline at [ffff880000099000] 99000 size 24576
   [    0.     0] init_memory_mapping: [mem 0x00000000-0x000fffff]
   [    0.     0]  [mem 0x00000000-0x000fffff] page 4k
   [    0.     0] BRK [0x01ee3000, 0x01ee3fff] PGTABLE
   [    0.     0] BRK [0x01ee4000, 0x01ee4fff] PGTABLE
   [    0.     0] BRK [0x01ee5000, 0x01ee5fff] PGTABLE
   [    0.     0] init_memory_mapping: [mem 0x13ee00000-0x13effffff]
   [    0.     0]  [mem 0x13ee00000-0x13effffff] page 2M
   [    0.     0] BRK [0x01ee6000, 0x01ee6fff] PGTABLE
   [    0.     0] init_memory_mapping: [mem 0x13c000000-0x13edfffff]
   [    0.     0]  [mem 0x13c000000-0x13edfffff] page 2M
   [    0.     0] init_memory_mapping: [mem 0x100000000-0x13bffffff]
   [    0.     0]  [mem 0x100000000-0x13bffffff] page 2M
   [    0.     0] init_memory_mapping: [mem 0x00100000-0xbffdefff]
   [    0.     0]  [mem 0x00100000-0x001fffff] page 4k
   [    0.     0]  [mem 0x00200000-0xbfdfffff] page 2M
   [    0.     0]  [mem 0xbfe00000-0xbffdefff] page 4k
   [    0.     0] RAMDISK: [mem 0x35ea8000-0x36f4bfff]
   [    0.     0] ACPI: RSDP 00000000000f0e00 00014 (v00 BOCHS )
   [    0.     0] ACPI: RSDT 00000000bffe16cf 00034 (v01 BOCHS  BXPCRSDT 00000001 BXPC 00000001)
   [    0.     0] ACPI: FACP 00000000bffe0877 00074 (v01 BOCHS  BXPCFACP 00000001 BXPC 00000001)
   [    0.     0] ACPI: DSDT 00000000bffdfd80 00AF7 (v01 BOCHS  BXPCDSDT 00000001 BXPC 00000001)
   [    0.     0] ACPI: FACS 00000000bffdfd40 00040
   [    0.     0] ACPI: SSDT 00000000bffe08eb 00D1C (v01 BOCHS  BXPCSSDT 00000001 BXPC 00000001)
   [    0.     0] ACPI: APIC 00000000bffe1607 00090 (v01 BOCHS  BXPCAPIC 00000001 BXPC 00000001)
   [    0.     0] ACPI: HPET 00000000bffe1697 00038 (v01 BOCHS  BXPCHPET 00000001 BXPC 00000001)
   [    0.     0] ACPI: Local APIC address 0xfee00000
   [    0.     0] No NUMA configuration found
   [    0.     0] Faking a node at [mem 0x0000000000000000-0x000000013effffff]
   [    0.     0] Initmem setup node 0 [mem 0x00000000-0x13effffff]
   [    0.     0]   NODE_DATA [mem 0x13efd6000-0x13effcfff]
   [    0.     0] Reserving 256MB of memory at 592MB for crashkernel (System RAM: 4079MB)
   [    0.     0] kvm-clock: Using msrs 4b564d01 and 4b564d00
   [    0.     0] kvm-clock: cpu 0, msr 1:3ef86001, primary cpu clock
   [    0.     0]  [ffffea0000000000-ffffea0004ffffff] PMD -> [ffff88013a600000-ffff88013e5fffff] on node 0
   [    0.     0] Zone ranges:
   [    0.     0]   DMA      [mem 0x00001000-0x00ffffff]
   [    0.     0]   DMA32    [mem 0x01000000-0xffffffff]
   [    0.     0]   Normal   [mem 0x100000000-0x13effffff]
   [    0.     0] Movable zone start for each node
   [    0.     0] Mirror info
   [    0.     0] Early memory node ranges
   [    0.     0]   node   0: [mem 0x00001000-0x0009efff]
   [    0.     0]   node   0: [mem 0x00100000-0xbffdefff]
   [    0.     0]   node   0: [mem 0x100000000-0x13effffff]
   [    0.     0] On node 0 totalpages: 1044349
   [    0.     0]   DMA zone: 64 pages used for memmap
   [    0.     0]   DMA zone: 21 pages reserved
   [    0.     0]   DMA zone: 3998 pages, LIFO batch:0
   [    0.     0]   DMA32 zone: 12224 pages used for memmap
   [    0.     0]   DMA32 zone: 782303 pages, LIFO batch:31
   [    0.     0]   Normal zone: 4032 pages used for memmap
   [    0.     0]   Normal zone: 258048 pages, LIFO batch:31
   [    0.     0] ACPI: PM-Timer IO Port: 0x608
   [    0.     0] ACPI: Local APIC address 0xfee00000
   [    0.     0] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])
   [    0.     0] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23
   [    0.     0] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
   [    0.     0] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)
   [    0.     0] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
   [    0.     0] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)
   [    0.     0] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)
   [    0.     0] ACPI: IRQ0 used by override.
   [    0.     0] ACPI: IRQ2 used by override.
   [    0.     0] ACPI: IRQ5 used by override.
   [    0.     0] ACPI: IRQ9 used by override.
   [    0.     0] ACPI: IRQ10 used by override.
   [    0.     0] ACPI: IRQ11 used by override.
   [    0.     0] Using ACPI (MADT) for SMP configuration information
   [    0.     0] ACPI: HPET id: 0x8086a201 base: 0xfed00000
   [    0.     0] smpboot: Allowing 4 CPUs, 0 hotplug CPUs
   [    0.     0] nr_irqs_gsi: 40
   [    0.     0] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff]
   [    0.     0] PM: Registered nosave memory: [mem 0x000a0000-0x000effff]
   [    0.     0] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff]
   [    0.     0] PM: Registered nosave memory: [mem 0xbffdf000-0xbfffffff]
   [    0.     0] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff]
   [    0.     0] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff]
   [    0.     0] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff]
   [    0.     0] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff]
   [    0.     0] e820: [mem 0xc0000000-0xfeffbfff] available for PCI devices
   [    0.     0] Booting paravirtualized kernel on KVM
   [    0.     0] setup_percpu: NR_CPUS:5120 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1
   [    0.     0] PERCPU: Embedded 28 pages/cpu @ffff88013ec00000 s82880 r8192 d23616 u524288
   [    0.     0] pcpu-alloc: s82880 r8192 d23616 u524288 alloc=1*2097152
   [    0.     0] pcpu-alloc: [0] 0 1 2 3 
   [    0.     0] KVM setup async PF for cpu 0
   [    0.     0] kvm-stealtime: cpu 0, msr 13ec0d000
   [    0.     0] Built 1 zonelists in Node order, mobility grouping on.  Total pages: 1028008
   [    0.     0] Policy zone: Normal
   [    0.     0] Kernel command line: BOOT_IMAGE=/vmlinuz-3.10.0-229.20.1.22.hulk.x86_64 root=/dev/mapper/euleros-root ro reserve_kbox_mem=16M crash_kexec_post_notifiers panic=3 8250.nr_uarts=8 efi=old_map rd.lvm.lv=euleros/swap crashkernel=2G-32G:256M,32G-1T:512M,1T-8T:1G,8T-:4G rd.lvm.lv=euleros/root rhgb quiet
   [    0.     0] PID hash table entries: 4096 (order: 3, 32768 bytes)
   [    0.     0] Checking aperture...
   [    0.     0] No AGP bridge found
   [    0.     0] Memory: 3750084k/5226496k available (6220k kernel code, 1049100k absent, 427312k reserved, 4174k data, 1604k init)
   [    0.     0] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1
   [    0.     0] Hierarchical RCU implementation.
   [    0.     0] ?RCU restricting CPUs from NR_CPUS=5120 to nr_cpu_ids=4.
   [    0.     0] ?Experimental no-CBs for all CPUs
   [    0.     0] ?Experimental no-CBs CPUs: 0-3.
   [    0.     0] NR_IRQS:327936 nr_irqs:712 0
   [    0.     0] Console: colour VGA+ 80x25
   [    0.     0] console [tty0] enabled
   [    0.     0] allocated 16777216 bytes of page_cgroup
   [    0.     0] please try 'cgroup_disable=memory' option if you don't want memory cgroups
   [    0.     0] hpet clockevent registered
   [    0.     0] tsc: Detected 2099.998 MHz processor
   [    0.  2000] Calibrating delay loop (skipped) preset value.. 4199.99 BogoMIPS (lpj=2099998)
   [    0.  2000] pid_max: default: 32768 minimum: 301
   [    0.  2000] Security Framework initialized
   [    0.  2000] SELinux:  Initializing.
   [    0.  2000] SELinux:  Starting in permissive mode
   [    0.  2000] Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes)
   [    0.  4006] Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes)
   [    0.  4512] Mount-cache hash table entries: 4096
   [    0.  4709] Initializing cgroup subsys memory
   [    0.  4717] Initializing cgroup subsys devices
   [    0.  4719] Initializing cgroup subsys freezer
   [    0.  4720] Initializing cgroup subsys net_cls
   [    0.  4722] Initializing cgroup subsys blkio
   [    0.  4724] Initializing cgroup subsys perf_event
   [    0.  4726] Initializing cgroup subsys hugetlb
   [    0.  5052] mce: CPU supports 10 MCE banks
   [    0.  5096] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0
   Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0
   tlb_flushall_shift: 6
   [    0.  5263] Freeing SMP alternatives: 24k freed
   [    0.  9184] ACPI: Core revision 20130517
   [    0. 10146] ACPI: All ACPI Tables successfully acquired
   [    0. 10187] ftrace: allocating 24008 entries in 94 pages
   [    0. 17279] Enabling x2apic
   [    0. 17286] Enabled x2apic
   [    0. 17486] Switched APIC routing to physical x2apic.
   [    0. 18975] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
   [    0. 18977] smpboot: CPU0: Intel QEMU Virtual CPU version 2.1.0 (fam: 06, model: 06, stepping: 03)
   [    0. 19000] Performance Events: Broken PMU hardware detected, using software events only.
   [    0. 19007] Failed to access perfctr msr (MSR c1 is 0)
   [    0. 20737] NMI watchdog: disabled (cpu0): hardware events not enabled
   [    0. 20729] kvm-clock: cpu 1, msr 1:3ef86041, secondary cpu clock
   [    0. 33015] KVM setup async PF for cpu 1
   [    0. 33021] kvm-stealtime: cpu 1, msr 13ec8d000
   [    0. 33023] kvm-clock: cpu 2, msr 1:3ef86081, secondary cpu clock
   [    0. 45016] KVM setup async PF for cpu 2
   [    0. 45023] kvm-stealtime: cpu 2, msr 13ed0d000
   [    0. 20814] smpboot: Booting Node   0, Processors  #1 #2 #3 OK
   [    0. 45023] kvm-clock: cpu 3, msr 1:3ef860c1, secondary cpu clock
   [    0. 58021] Brought up 4 CPUs
   [    0. 58025] smpboot: Total of 4 processors activated (16799.98 BogoMIPS)
   [    0. 58014] KVM setup async PF for cpu 3
   [    0. 58018] kvm-stealtime: cpu 3, msr 13ed8d000
   [    0. 59002] devtmpfs: initialized
   [    0. 61851] EVM: security.selinux
   [    0. 61853] EVM: security.ima
   [    0. 61854] EVM: security.capability
   [    0. 62993] atomic64 test passed for x86-64 platform with CX8 and with SSE
   [    0. 63103] NET: Registered protocol family 16
   [    0. 63377] ACPI: bus type PCI registered
   [    0. 63379] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5
   [    0. 63503] PCI: Using configuration type 1 for base access
   [    0. 64399] ACPI: Added _OSI(Module Device)
   [    0. 64399] ACPI: Added _OSI(Processor Device)
   [    0. 64399] ACPI: Added _OSI(3.0 _SCP Extensions)
   [    0. 64399] ACPI: Added _OSI(Processor Aggregator Device)
   [    0. 64399] ACPI: Added _OSI(EulerOS RAS Extensions)
   [    0. 65301] ACPI: EC: Look up EC in DSDT
   [    0. 66195] ACPI: Interpreter enabled
   [    0. 66201] ACPI Exception: AE_NOT_FOUND, While evaluating Sleep State [\_S1_] (20130517/hwxface-571)
   [    0. 66205] ACPI Exception: AE_NOT_FOUND, While evaluating Sleep State [\_S2_] (20130517/hwxface-571)
   [    0. 66216] ACPI: (supports S0 S3 S4 S5)
   [    0. 66217] ACPI: Using IOAPIC for interrupt routing
   [    0. 66230] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug
   [    0. 68961] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])
   [    0. 68967] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI]
   [    0. 68972] acpi PNP0A03:00: _OSC failed (AE_NOT_FOUND); disabling ASPM
   [    0. 69063] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge.
   [    0. 69322] acpiphp: Slot [3] registered
   [    0. 69346] acpiphp: Slot [4] registered
   [    0. 69370] acpiphp: Slot [5] registered
   [    0. 69394] acpiphp: Slot [6] registered
   [    0. 69416] acpiphp: Slot [7] registered
   [    0. 69437] acpiphp: Slot [8] registered
   [    0. 69457] acpiphp: Slot [9] registered
   [    0. 69478] acpiphp: Slot [10] registered
   [    0. 69499] acpiphp: Slot [11] registered
   [    0. 69519] acpiphp: Slot [12] registered
   [    0. 69541] acpiphp: Slot [13] registered
   [    0. 69563] acpiphp: Slot [14] registered
   [    0. 69583] acpiphp: Slot [15] registered
   [    0. 69604] acpiphp: Slot [16] registered
   [    0. 69625] acpiphp: Slot [17] registered
   [    0. 69645] acpiphp: Slot [18] registered
   [    0. 69666] acpiphp: Slot [19] registered
   [    0. 69688] acpiphp: Slot [20] registered
   [    0. 69710] acpiphp: Slot [21] registered
   [    0. 69730] acpiphp: Slot [22] registered
   [    0. 69751] acpiphp: Slot [23] registered
   [    0. 69771] acpiphp: Slot [24] registered
   [    0. 69792] acpiphp: Slot [25] registered
   [    0. 69813] acpiphp: Slot [26] registered
   [    0. 69835] acpiphp: Slot [27] registered
   [    0. 69856] acpiphp: Slot [28] registered
   [    0. 69877] acpiphp: Slot [29] registered
   [    0. 69898] acpiphp: Slot [30] registered
   [    0. 69919] acpiphp: Slot [31] registered
   [    0. 69931] PCI host bridge to bus 0000:00
   [    0. 69934] pci_bus 0000:00: root bus resource [bus 00-ff]
   [    0. 69937] pci_bus 0000:00: root bus resource [io  0x0000-0x0cf7]
   [    0. 69939] pci_bus 0000:00: root bus resource [io  0x0d00-0xadff]
   [    0. 69941] pci_bus 0000:00: root bus resource [io  0xae0f-0xaeff]
   [    0. 69943] pci_bus 0000:00: root bus resource [io  0xaf20-0xafdf]
   [    0. 69945] pci_bus 0000:00: root bus resource [io  0xafe4-0xffff]
   [    0. 69947] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff]
   [    0. 69949] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff]
   [    0. 69991] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000
   [    0. 70387] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100
   [    0. 70934] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180
   [    0. 75005] pci 0000:00:01.1: reg 0x20: [io  0xc080-0xc08f]
   [    0. 76432] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io  0x01f0-0x01f7]
   [    0. 76435] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io  0x03f6]
   [    0. 76437] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io  0x0170-0x0177]
   [    0. 76439] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io  0x0376]
   [    0. 76594] pci 0000:00:01.2: [8086:7020] type 00 class 0x0c0300
   [    0. 80787] pci 0000:00:01.2: reg 0x20: [io  0xc040-0xc05f]
   [    0. 82574] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000
   [    0. 83098] pci 0000:00:01.3: quirk: [io  0x0600-0x063f] claimed by PIIX4 ACPI
   [    0. 83111] pci 0000:00:01.3: quirk: [io  0x0700-0x070f] claimed by PIIX4 SMB
   [    0. 83336] pci 0000:00:02.0: [1013:00b8] type 00 class 0x030000
   [    0. 85006] pci 0000:00:02.0: reg 0x10: [mem 0xfc000000-0xfdffffff pref]
   [    0. 87006] pci 0000:00:02.0: reg 0x14: [mem 0xfebf4000-0xfebf4fff]
   [    0. 97005] pci 0000:00:02.0: reg 0x30: [mem 0xfebe0000-0xfebeffff pref]
   [    0. 97204] pci 0000:00:03.0: [8086:100e] type 00 class 0x020000
   [    0. 99005] pci 0000:00:03.0: reg 0x10: [mem 0xfebc0000-0xfebdffff]
   [    0.100004] pci 0000:00:03.0: reg 0x14: [io  0xc000-0xc03f]
   [    0.108005] pci 0000:00:03.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref]
   [    0.108209] pci 0000:00:04.0: [8086:2668] type 00 class 0x040300
   [    0.109004] pci 0000:00:04.0: reg 0x10: [mem 0xfebf0000-0xfebf3fff]
   [    0.114288] pci 0000:00:06.0: [1af4:1002] type 00 class 0x00ff00
   [    0.115005] pci 0000:00:06.0: reg 0x10: [io  0xc060-0xc07f]
   [    0.120244] pci 0000:00:1e.0: [1af4:1110] type 00 class 0x050000
   [    0.122005] pci 0000:00:1e.0: reg 0x10: [mem 0xfebf5000-0xfebf50ff]
   [    0.126005] pci 0000:00:1e.0: reg 0x18: [mem 0xfe000000-0xfe7fffff 64bit pref]
   [    0.132465] ACPI: PCI Interrupt Link [LNKA] (IRQs 5 *10 11)
   [    0.132576] ACPI: PCI Interrupt Link [LNKB] (IRQs 5 *10 11)
   [    0.132673] ACPI: PCI Interrupt Link [LNKC] (IRQs 5 10 *11)
   [    0.132777] ACPI: PCI Interrupt Link [LNKD] (IRQs 5 10 *11)
   [    0.132827] ACPI: PCI Interrupt Link [LNKS] (IRQs *9)
   [    0.133399] ACPI: Enabled 16 GPEs in block 00 to 0F
   [    0.133478] vgaarb: device added: PCI:0000:00:02.0,decodes=io+mem,owns=io+mem,locks=none
   [    0.133478] vgaarb: loaded
   [    0.133478] vgaarb: bridge control possible 0000:00:02.0
   [    0.133478] SCSI subsystem initialized
   [    0.133478] ACPI: bus type USB registered
   [    0.133478] usbcore: registered new interface driver usbfs
   [    0.133478] usbcore: registered new interface driver hub
   [    0.133478] usbcore: registered new device driver usb
   [    0.133478] PCI: Using ACPI for IRQ routing
   [    0.133478] PCI: pci_cache_line_size set to 64 bytes
   [    0.134030] e820: reserve RAM buffer [mem 0x0009fc00-0x0009ffff]
   [    0.134032] e820: reserve RAM buffer [mem 0xbffdf000-0xbfffffff]
   [    0.134034] e820: reserve RAM buffer [mem 0x13f000000-0x13fffffff]
   [    0.134136] NetLabel: Initializing
   [    0.134138] NetLabel:  domain hash size = 128
   [    0.134139] NetLabel:  protocols = UNLABELED CIPSOv4
   [    0.134155] NetLabel:  unlabeled traffic allowed by default
   [    0.134225] HPET: 3 timers in total, 0 timers will be used for per-cpu timer
   [    0.134244] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0
   [    0.134248] hpet0: 3 comparators, 64-bit 100.000000 MHz counter
   [    0.138016] Switching to clocksource kvm-clock
   [    0.143436] pnp: PnP ACPI init
   [    0.143448] ACPI: bus type PNP registered
   [    0.143515] pnp 00:00: Plug and Play ACPI device, IDs PNP0b00 (active)
   [    0.143565] pnp 00:01: Plug and Play ACPI device, IDs PNP0303 (active)
   [    0.143605] pnp 00:02: Plug and Play ACPI device, IDs PNP0f13 (active)
   [    0.143646] pnp 00:03: [dma 2]
   [    0.143660] pnp 00:03: Plug and Play ACPI device, IDs PNP0700 (active)
   [    0.143757] pnp 00:04: Plug and Play ACPI device, IDs PNP0501 (active)
   [    0.143905] pnp 00:05: Plug and Play ACPI device, IDs PNP0103 (active)
   [    0.144030] pnp: PnP ACPI: found 6 devices
   [    0.144032] ACPI: bus type PNP unregistered
   [    0.151861] pci_bus 0000:00: resource 4 [io  0x0000-0x0cf7]
   [    0.151864] pci_bus 0000:00: resource 5 [io  0x0d00-0xadff]
   [    0.151866] pci_bus 0000:00: resource 6 [io  0xae0f-0xaeff]
   [    0.151868] pci_bus 0000:00: resource 7 [io  0xaf20-0xafdf]
   [    0.151870] pci_bus 0000:00: resource 8 [io  0xafe4-0xffff]
   [    0.151872] pci_bus 0000:00: resource 9 [mem 0x000a0000-0x000bffff]
   [    0.151874] pci_bus 0000:00: resource 10 [mem 0xc0000000-0xfebfffff]
   [    0.151901] NET: Registered protocol family 2
   [    0.152048] TCP established hash table entries: 32768 (order: 6, 262144 bytes)
   [    0.152139] TCP bind hash table entries: 32768 (order: 7, 524288 bytes)
   [    0.152193] TCP: Hash tables configured (established 32768 bind 32768)
   [    0.152209] TCP: reno registered
   [    0.152217] UDP hash table entries: 2048 (order: 4, 65536 bytes)
   [    0.152235] UDP-Lite hash table entries: 2048 (order: 4, 65536 bytes)
   [    0.152296] NET: Registered protocol family 1
   [    0.152307] pci 0000:00:00.0: Limiting direct PCI/PCI transfers
   [    0.152328] pci 0000:00:01.0: PIIX3: Enabling Passive Release
   [    0.152348] pci 0000:00:01.0: Activating ISA DMA hang workarounds
   [    0.182906] ACPI: PCI Interrupt Link [LNKD] enabled at IRQ 11
   [    0.213623] pci 0000:00:02.0: Boot video device
   [    0.213672] PCI: CLS 0 bytes, default 64
   [    0.213724] Unpacking initramfs...
   [    0.523944] Freeing initrd memory: 17040k freed
   [    0.528748] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
   [    0.528753] software IO TLB [mem 0xbbfdf000-0xbffdf000] (64MB) mapped at [ffff8800bbfdf000-ffff8800bffdefff]
   [    0.529517] microcode: CPU0 sig=0x663, pf=0x1, revision=0x1
   [    0.529534] microcode: CPU1 sig=0x663, pf=0x1, revision=0x1
   [    0.529552] microcode: CPU2 sig=0x663, pf=0x1, revision=0x1
   [    0.529569] microcode: CPU3 sig=0x663, pf=0x1, revision=0x1
   [    0.529616] microcode: Microcode Update Driver: v2.00 <tigran@aivazian.fsnet.co.uk>, Peter Oruba
   [    0.529912] futex hash table entries: 1024 (order: 4, 65536 bytes)
   [    0.529933] Initialise system trusted keyring
   [    0.529988] audit: initializing netlink socket (disabled)
   [    0.529999] type=2000 audit(1455811046.013:1): initialized
   [    0.555547] HugeTLB registered 2 MB page size, pre-allocated 0 pages
   [    0.556760] zbud: loaded
   [    0.556971] VFS: Disk quotas dquot_6.5.2
   [    0.557018] Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
   [    0.557237] msgmni has been set to 7357
   [    0.557294] Key type big_key registered
   [    0.557298] SELinux:  Registering netfilter hooks
   [    0.558946] NET: Registered protocol family 38
   [    0.558956] Key type asymmetric registered
   [    0.558959] Asymmetric key parser 'x509' registered
   [    0.558994] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 252)
   [    0.559078] io scheduler noop registered
   [    0.559083] io scheduler deadline registered (default)
   [    0.559115] io scheduler cfq registered
   [    0.559204] pci_hotplug: PCI Hot Plug PCI Core version: 0.5
   [    0.559218] pciehp: PCI Express Hot Plug Controller Driver version: 0.4
   [    0.559277] intel_idle: does not run on family 6 model 6
   [    0.559346] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0
   [    0.559350] ACPI: Power Button [PWRF]
   [    0.559565] GHES: HEST is not enabled!
   [    0.559652] Serial: 8250/16550 driver, 8 ports, IRQ sharing enabled
   [    0.584666] 00:04: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
   [    0.585463] Non-volatile memory driver v1.3
   [    0.585469] Linux agpgart interface v0.103
   [    0.585612] crash memory driver: version 1.1
   [    0.585645] rdac: device handler registered
   [    0.585699] hp_sw: device handler registered
   [    0.585701] emc: device handler registered
   [    0.585704] alua: device handler registered
   [    0.585736] libphy: Fixed MDIO Bus: probed
   [    0.585798] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
   [    0.585803] ehci-pci: EHCI PCI platform driver
   [    0.585813] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
   [    0.585816] ohci-pci: OHCI PCI platform driver
   [    0.585824] uhci_hcd: USB Universal Host Controller Interface driver
   [    0.617200] uhci_hcd 0000:00:01.2: UHCI Host Controller
   [    0.617270] uhci_hcd 0000:00:01.2: new USB bus registered, assigned bus number 1
   [    0.617296] uhci_hcd 0000:00:01.2: detected 2 ports
   [    0.617395] uhci_hcd 0000:00:01.2: irq 11, io base 0x0000c040
   [    0.617488] usb usb1: New USB device found, idVendor=1d6b, idProduct=0001
   [    0.617491] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
   [    0.617493] usb usb1: Product: UHCI Host Controller
   [    0.617495] usb usb1: Manufacturer: Linux 3.10.0-229.20.1.22.hulk.x86_64 uhci_hcd
   [    0.617496] usb usb1: SerialNumber: 0000:00:01.2
   [    0.617613] hub 1-0:1.0: USB hub found
   [    0.617620] hub 1-0:1.0: 2 ports detected
   [    0.617762] usbcore: registered new interface driver usbserial
   [    0.617768] usbcore: registered new interface driver usbserial_generic
   [    0.617774] usbserial: USB Serial support registered for generic
   [    0.617800] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12
   [    0.618540] serio: i8042 KBD port at 0x60,0x64 irq 1
   [    0.618545] serio: i8042 AUX port at 0x60,0x64 irq 12
   [    0.618647] mousedev: PS/2 mouse device common for all mice
   [    0.618954] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1
   [    0.620143] rtc_cmos 00:00: RTC can wake from S4
   [    0.620527] rtc_cmos 00:00: rtc core: registered rtc_cmos as rtc0
   [    0.620742] rtc_cmos 00:00: alarms up to one day, 114 bytes nvram, hpet irqs
   [    0.620805] cpuidle: using governor menu
   [    0.620873] hidraw: raw HID events driver (C) Jiri Kosina
   [    0.620984] usbcore: registered new interface driver usbhid
   [    0.620986] usbhid: USB HID core driver
   [    0.621049] drop_monitor: Initializing network drop monitor service
   [    0.621131] TCP: cubic registered
   [    0.621136] Initializing XFRM netlink socket
   [    0.621226] NET: Registered protocol family 10
   [    0.621378] NET: Registered protocol family 17
   [    0.621721] Loading compiled-in X.509 certificates
   [    0.622345] Loaded X.509 cert 'Red Hat Enterprise Linux Driver Update Program (key 3): bf57f3e87362bc7229d9f465321773dfd1f77a80'
   [    0.622928] Loaded X.509 cert 'Red Hat Enterprise Linux kpatch signing key: 4d38fd864ebe18c5f0b72e3852e2014c3a676fc8'
   [    0.623888] Loaded X.509 cert 'Magrathea: Glacier signing key: 6e93856ba079c51674540346c06d7c031d25d30d'
   [    0.623906] registered taskstats version 1
   [    0.626078] Key type trusted registered
   [    0.627915] Key type encrypted registered
   [    0.629712] IMA: No TPM chip found, activating TPM-bypass!
   [    0.630129] rtc_cmos 00:00: setting system clock to 2016-02-18 15:57:25 UTC (1455811045)
   [    0.630912] Freeing unused kernel memory: 1604k freed
   [    0.634541] systemd[1]: systemd 208 running in system mode. (+PAM +LIBWRAP +AUDIT +SELINUX +IMA +SYSVINIT +LIBCRYPTSETUP +GCRYPT +ACL +XZ)
   [    0.634589] systemd[1]: Detected virtualization 'kvm'.
   [    0.634592] systemd[1]: Running in initial RAM disk.
   [    0.634651] systemd[1]: Set hostname to <localhost.localdomain>.
   [    0.668185] systemd[1]: Expecting device dev-mapper-euleros\x2droot.device...
   [    0.668201] systemd[1]: Starting -.slice.
   [    0.668336] systemd[1]: Created slice -.slice.
   [    0.668382] systemd[1]: Starting System Slice.
   [    0.668458] systemd[1]: Created slice System Slice.
   [    0.668500] systemd[1]: Starting Slices.
   [    0.668512] systemd[1]: Reached target Slices.
   [    0.668546] systemd[1]: Starting Timers.
   [    0.668558] systemd[1]: Reached target Timers.
   [    0.668592] systemd[1]: Starting Journal Socket.
   [    0.668660] systemd[1]: Listening on Journal Socket.
   [    0.668754] systemd[1]: Started dracut ask for additional cmdline parameters.
   [    0.668925] systemd[1]: Starting dracut cmdline hook...
   [    0.669394] systemd[1]: Started Load Kernel Modules.
   [    0.669411] systemd[1]: Starting Setup Virtual Console...
   [    0.669714] systemd[1]: Starting Journal Service...
   [    0.670110] systemd[1]: Started Journal Service.
   [    0.676193] NMI watchdog: disabled (cpu0): hardware events not enabled
   [    0.805130] device-mapper: uevent: version 1.0.3
   [    0.805248] device-mapper: ioctl: 4.29.0-ioctl (2014-10-28) initialised: dm-devel@redhat.com
   [    0.840316] systemd-udevd[257]: starting version 208
   [    0.870632] e1000: Intel(R) PRO/1000 Network Driver - version 7.3.21-k8-NAPI
   [    0.870635] e1000: Copyright (c) 1999-2006 Intel Corporation.
   [    0.875066] [drm] Initialized drm 1.1.0 20060810
   [    0.875462] FDC 0 is a S82078B
   [    0.883437] libata version 3.00 loaded.
   [    0.911357] ACPI: PCI Interrupt Link [LNKB] enabled at IRQ 10
   [    0.920031] usb 1-1: new full-speed USB device number 2 using uhci_hcd
   [    0.943102] ACPI: PCI Interrupt Link [LNKC] enabled at IRQ 11
   [    1.267681] e1000 0000:00:03.0 eth0: (PCI:33MHz:32-bit) 90:e2:ba:20:0d:21
   [    1.267689] e1000 0000:00:03.0 eth0: Intel(R) PRO/1000 Network Connection
   [    1.267744] ata_piix 0000:00:01.1: version 2.13
   [    1.269451] scsi host0: ata_piix
   [    1.269706] scsi host1: ata_piix
   [    1.269745] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc080 irq 14
   [    1.269748] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc088 irq 15
   [    1.270098] [TTM] Zone  kernel: Available graphics memory: 1884376 kiB
   [    1.270100] [TTM] Initializing pool allocator
   [    1.270105] [TTM] Initializing DMA pool allocator
   [    1.270581] [drm] fb mappable at 0xFC000000
   [    1.270583] [drm] vram aper at 0xFC000000
   [    1.270584] [drm] size 4194304
   [    1.270585] [drm] fb depth is 24
   [    1.270586] [drm]    pitch is 3072
   [    1.271204] fbcon: cirrusdrmfb (fb0) is primary device
   [    1.290116] Console: switching to colour frame buffer device 128x48
   [    1.303472] cirrus 0000:00:02.0: fb0: cirrusdrmfb frame buffer device
   [    1.303475] cirrus 0000:00:02.0: registered panic notifier
   [    1.303481] [drm] Initialized cirrus 1.0.0 20110418 for 0000:00:02.0 on minor 0
   [    1.310106] systemd-udevd[259]: renamed network interface eth0 to ens3
   [    1.421760] ata1.00: ATAPI: QEMU DVD-ROM, 2.1.0, max UDMA/100
   [    1.421767] ata1.01: ATA-7: QEMU HARDDISK, 2.1.0, max UDMA/100
   [    1.421769] ata1.01: 83886080 sectors, multi 16: LBA48 
   [    1.422428] ata1.00: configured for MWDMA2
   [    1.422878] ata1.01: configured for MWDMA2
   [    1.423499] scsi 0:0:0:0: CD-ROM            QEMU     QEMU DVD-ROM     2.1. PQ: 0 ANSI: 5
   [    1.424024] scsi 0:0:1:0: Direct-Access     ATA      QEMU HARDDISK    2.1. PQ: 0 ANSI: 5
   [    1.432649] sr 0:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray
   [    1.432653] cdrom: Uniform CD-ROM driver Revision: 3.20
   [    1.432760] sr 0:0:0:0: Attached scsi CD-ROM sr0
   [    1.435856] sd 0:0:1:0: [sda] 83886080 512-byte logical blocks: (42.9 GB/40.0 GiB)
   [    1.446975] sd 0:0:1:0: [sda] Write Protect is off
   [    1.446979] sd 0:0:1:0: [sda] Mode Sense: 00 3a 00 00
   [    1.447138] sd 0:0:1:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
   [    1.448504]  sda: sda1 sda2
   [    1.448780] sd 0:0:1:0: [sda] Attached SCSI disk
   [    1.451475] input: ImExPS/2 Generic Explorer Mouse as /devices/platform/i8042/serio1/input/input2
   [    1.531051] tsc: Refined TSC clocksource calibration: 2100.005 MHz
   [    4.922621] usb 1-1: New USB device found, idVendor=0627, idProduct=0001
   [    4.922627] usb 1-1: New USB device strings: Mfr=1, Product=3, SerialNumber=5
   [    4.922629] usb 1-1: Product: QEMU USB Tablet
   [    4.922631] usb 1-1: Manufacturer: QEMU
   [    4.922633] usb 1-1: SerialNumber: 42
   [    5.823527] input: QEMU QEMU USB Tablet as /devices/pci0000:00/0000:00:01.2/usb1/1-1/1-1:1.0/input/input3
   [    5.823757] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v0.01 Pointer [QEMU QEMU USB Tablet] on usb-0000:00:01.2-1/input0
   [    6. 86844] SGI XFS with ACLs, security attributes, large block/inode numbers, no debug enabled
   [    6. 88977] XFS (dm-1): Mounting V4 Filesystem
   [    6.155803] XFS (dm-1): Starting recovery (logdev: internal)
   [    6.187844] XFS (dm-1): Ending recovery (logdev: internal)
   [    6.422561] systemd-journald[106]: Received SIGTERM
   [    6.812415] type=1404 audit(1455811051.682:2): enforcing=1 old_enforcing=0 auid=4294967295 ses=4294967295
   [    7. 24071] SELinux: 2048 avtab hash slots, 111514 rules.
   [    7. 44938] SELinux: 2048 avtab hash slots, 111514 rules.
   [    7. 79940] SELinux:  8 users, 102 roles, 4977 types, 295 bools, 1 sens, 1024 cats
   [    7. 79944] SELinux:  83 classes, 111514 rules
   [    7. 86403] SELinux:  Completing initialization.
   [    7. 86406] SELinux:  Setting up existing superblocks.
   [    7. 86412] SELinux: initialized (dev sysfs, type sysfs), uses genfs_contexts
   [    7. 86417] SELinux: initialized (dev rootfs, type rootfs), uses genfs_contexts
   [    7. 86426] SELinux: initialized (dev bdev, type bdev), uses genfs_contexts
   [    7. 86431] SELinux: initialized (dev proc, type proc), uses genfs_contexts
   [    7. 86480] SELinux: initialized (dev tmpfs, type tmpfs), uses transition SIDs
   [    7. 86496] SELinux: initialized (dev devtmpfs, type devtmpfs), uses transition SIDs
   [    7. 86976] SELinux: initialized (dev sockfs, type sockfs), uses task SIDs
   [    7. 86980] SELinux: initialized (dev debugfs, type debugfs), uses genfs_contexts
   [    7. 87872] SELinux: initialized (dev pipefs, type pipefs), uses task SIDs
   [    7. 87877] SELinux: initialized (dev anon_inodefs, type anon_inodefs), uses genfs_contexts
   [    7. 87880] SELinux: initialized (dev aio, type aio), not configured for labeling
   [    7. 87883] SELinux: initialized (dev devpts, type devpts), uses transition SIDs
   [    7. 87895] SELinux: initialized (dev hugetlbfs, type hugetlbfs), uses transition SIDs
   [    7. 87900] SELinux: initialized (dev mqueue, type mqueue), uses transition SIDs
   [    7. 87905] SELinux: initialized (dev selinuxfs, type selinuxfs), uses genfs_contexts
   [    7. 87915] SELinux: initialized (dev securityfs, type securityfs), uses genfs_contexts
   [    7. 87919] SELinux: initialized (dev sysfs, type sysfs), uses genfs_contexts
   [    7. 88245] SELinux: initialized (dev tmpfs, type tmpfs), uses transition SIDs
   [    7. 88252] SELinux: initialized (dev tmpfs, type tmpfs), uses transition SIDs
   [    7. 88317] SELinux: initialized (dev tmpfs, type tmpfs), uses transition SIDs
   [    7. 88342] SELinux: initialized (dev cgroup, type cgroup), uses genfs_contexts
   [    7. 88351] SELinux: initialized (dev pstore, type pstore), uses genfs_contexts
   [    7. 88354] SELinux: initialized (dev cgroup, type cgroup), uses genfs_contexts
   [    7. 88359] SELinux: initialized (dev cgroup, type cgroup), uses genfs_contexts
   [    7. 88364] SELinux: initialized (dev cgroup, type cgroup), uses genfs_contexts
   [    7. 88372] SELinux: initialized (dev cgroup, type cgroup), uses genfs_contexts
   [    7. 88376] SELinux: initialized (dev cgroup, type cgroup), uses genfs_contexts
   [    7. 88380] SELinux: initialized (dev cgroup, type cgroup), uses genfs_contexts
   [    7. 88384] SELinux: initialized (dev cgroup, type cgroup), uses genfs_contexts
   [    7. 88391] SELinux: initialized (dev cgroup, type cgroup), uses genfs_contexts
   [    7. 88395] SELinux: initialized (dev cgroup, type cgroup), uses genfs_contexts
   [    7. 88400] SELinux: initialized (dev configfs, type configfs), uses genfs_contexts
   [    7. 88402] SELinux: initialized (dev drm, type drm), not configured for labeling
   [    7. 88409] SELinux: initialized (dev dm-1, type xfs), uses xattr
   [    7. 98755] type=1403 audit(1455811051.967:3): policy loaded auid=4294967295 ses=4294967295
   [    7.110951] systemd[1]: Successfully loaded SELinux policy in 317.874ms.
   [    7.212349] systemd[1]: Relabelled /dev and /run in 23.634ms.
   [    7.862304] SELinux: initialized (dev autofs, type autofs), uses genfs_contexts
   [    8.131675] SELinux: initialized (dev hugetlbfs, type hugetlbfs), uses transition SIDs
   [    8.382401] systemd-udevd[503]: starting version 208
   [    8.742381] input: PC Speaker as /devices/platform/pcspkr/input/input4
   [    8.877826] ppdev: user-space parallel port driver
   [    8.961245] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0
   [    9.261314] XFS (sda1): Mounting V4 Filesystem
   [    9.518676] Adding 4063228k swap on /dev/mapper/euleros-swap.  Priority:-1 extents:1 across:4063228k FS
   [    9.602697] snd_hda_intel 0000:00:04.0: irq 40 for MSI
   [    9.636085] sound hdaudioC0D0: autoconfig: line_outs=1 (0x3/0x0/0x0/0x0/0x0) type:line
   [    9.636090] sound hdaudioC0D0:    speaker_outs=0 (0x0/0x0/0x0/0x0/0x0)
   [    9.636092] sound hdaudioC0D0:    hp_outs=0 (0x0/0x0/0x0/0x0/0x0)
   [    9.636094] sound hdaudioC0D0:    mono: mono_out=0x0
   [    9.636096] sound hdaudioC0D0:    inputs:
   [    9.636098] sound hdaudioC0D0:      Line=0x5
   [    9.747980] XFS (sda1): Ending clean mount
   [    9.747993] SELinux: initialized (dev sda1, type xfs), uses xattr
   [    9.755453] systemd-journald[482]: Received request to flush runtime journal from PID 1
   [   10. 37592] type=1305 audit(1455811054.906:4): audit_pid=585 old=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:auditd_t:s0 res=1
   [   11.482452] kbox: module verification failed: signature and/or  required key missing - tainting kernel
   [   11.485337] num of online cpus: 4
   [   11.485340] KBOX: the config 0x59 is ok.
   [   11.485343] KBOX: the config info is:
   Product          : euler
   Soft version     : V200R002C10
   Frame No         : 1
   Slot No          : 0
   Location No      : 0
   Hardware version : Unknown
   
   [   11.604279] console [kbox_console0] enabled
   [   11.649070] KBOX: load OK
   [   11.858046] ip_tables: (C) 2000-2006 Netfilter Core Team
   [   11.899038] kbox_status=0 Changing to=0(maintain(0),working(1))
   [   11.918847] hmem_driver: module verification failed: signature and/or  required key missing - tainting kernel
   [   11.930164] reserve_kbox_mem_start sym address: 0xffffffff81beab50, physical start address: 0x13f000000
   [   11.938631] reserve_kbox_mem_len sym address: 0xffffffff81beab48, physical size: 0x1000000
   [   11.938634] hmem_init parameters check ok. start_paddr: 0x13f000000, mem_size: 0x1000000
   [   11.939410] the manage area checksum.
   [   11.939927] no init or error!! magic number=0x0, flag=0x0.
   [   11.939929] init the manage area.
   [   11.939930] init the manage area finish!.
   [   11.944070] kbox:format dev hmem
   [   11.944074] file_supports=20
   [   11.998256] nf_conntrack version 0.5.0 (16384 buckets, 65536 max)
   [   12. 95253] ip6_tables: (C) 2000-2006 Netfilter Core Team
   [   12.253913] Ebtables v2.0 registered
   [   12.278419] Bridge firewalling registered
   [   12.852228] IPv6: ADDRCONF(NETDEV_UP): ens3: link is not ready
   [   12.856737] IPv6: ADDRCONF(NETDEV_UP): ens3: link is not ready
   [   12.860846] IPv6: ADDRCONF(NETDEV_UP): ens3: link is not ready
   [   13.796136] IPv6: ADDRCONF(NETDEV_UP): ens3: link is not ready
   [   14.855520] e1000: ens3 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX
   [   14.856049] IPv6: ADDRCONF(NETDEV_CHANGE): ens3: link becomes ready
   [   15. 14908] kbox_status=0 Changing to=1(maintain(0),working(1))
   [   15. 14913] dev_cb:ffff8800368b8680
   [   15. 14915] dev_cb: hmem
   [   57. 47985] SysRq : Manual OOM execution

3. Logs generated by the console

   ---------------area type:8, record num:1, unit_size:8 ------------
   
   
   ###num:0, record len:61
   oom time:20160218155821-e0387
   [ 2455.550908] Mem-Info:
   [ 2455.553173] Node 0 DMA per-cpu:
   [ 2455.556302] CPU    0: hi:    0, btch:   1 usd:   0
   [ 2455.561064] CPU    1: hi:    0, btch:   1 usd:   0
   [ 2455.565825] CPU    2: hi:    0, btch:   1 usd:   0
   [ 2455.570586] CPU    3: hi:    0, btch:   1 usd:   0
   [ 2455.575347] Node 0 DMA32 per-cpu:
   [ 2455.578651] CPU    0: hi:  186, btch:  31 usd:   0
   [ 2455.583412] CPU    1: hi:  186, btch:  31 usd:   0
   [ 2455.588173] CPU    2: hi:  186, btch:  31 usd:   0
   [ 2455.592934] CPU    3: hi:  186, btch:  31 usd:   0
   [ 2455.597698] active_anon:315623 inactive_anon:23 isolated_anon:0
   [ 2455.597699]  active_file:44 inactive_file:40 isolated_file:0
   [ 2455.597700]  unevictable:63860 dirty:22 writeback:0 unstable:0
   [ 2455.597701]  free:490 slab_reclaimable:3727 slab_unreclaimable:7388
   [ 2455.597703]  mapped:1188 shmem:27 pagetables:976 bounce:0
   [ 2455.626584] Node 0 DMA free:24kB min:16kB low:20kB high:24kB active_anon:15852kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15676kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
   [ 2455.662953] lowmem_reserve[]: 0 2 3072 128
   [ 2455.667074] Node 0 DMA32 free:1936kB min:2028kB low:2532kB high:3040kB active_anon:1246640kB inactive_anon:92kB active_file:176kB inactive_file:0kB unevictable:255440kB isolated(anon):0kB isolated(file):0kB present:1764800kB mlocked:0kB dirty:88kB writeback:0kB mapped:4752kB shmem:108kB slab_reclaimable:14908kB slab_unreclaimable:29552kB kernel_stack:792kB pagetables:3904kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:4640 all_unreclaimable? yes
   [ 2455.707242] lowmem_reserve[]: 0 0 256 256
   [ 2455.711283] Node 0 DMA: 1*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 4kB
   [ 2455.721622] Node 0 DMA32: 226*4kB 67*8kB 8*16kB 3*32kB 4*64kB 0*128kB 1*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 2176kB
   [ 2455.732650] 64028 total pagecache pages
   [ 2455.736460] 0 pages in swap cache
   [ 2455.739752] Swap cache stats: add 0, delete 0, find 0/0
   [ 2455.744944] Free swap  = 0kB
   [ 2455.747804] Total swap = 0kB
   [ 2455.755068] 487408 pages RAM
   [ 2455.757938] 78981 pages reserved
   [ 2455.761143] 7202 pages shared
   [ 2455.764089] 397079 pages non-shared
   [ 2455.767672] calling kbox_sync :begin
   [ 2455.771223] sync kbox :begin
   [ 2455.774085] open all redirect device :begin
   [ 2455.778242] open all redirect device :end
   [ 2455.782224] flush kbox regions :begin
   [ 2455.785864] kbox region (oom) is writing into (hmem), action is 202
   [ 2455.792525] test write len : 21748
   [ 2455.795902] first start addr : ffff88007e686000
   [ 2455.800403] second start addr : ffff88007e6a6000
   [ 2455.804990] cur_index : 1, offset : 131072, third start addr : ffff88007e6a6008
   [ 2455.812252] first length : 21748
   [ 2455.815508] bios_write_file_data write data len *ptr_length : 21748
   [ 2455.821735] cur_index : 2, record_number : 2, total_number : 4
   [ 2455.827538] kbox region (oom) has been written into (hmem)
   [ 2455.833420] dev hmem is dirty
   oom time:20160218155821-e0387