Skip to content

Instantly share code, notes, and snippets.

@omgjlk
Created April 15, 2019 21:23
Show Gist options
  • Save omgjlk/580de626eb070bd223310d5e23836f25 to your computer and use it in GitHub Desktop.
Save omgjlk/580de626eb070bd223310d5e23836f25 to your computer and use it in GitHub Desktop.
[2694520.622354] CPU 3/KVM invoked oom-killer: gfp_mask=0x24280ca(GFP_HIGHUSER_MOVABLE|__GFP_ZERO), nodemask=0, order=0, oom_score_adj=0
[2694520.622482] CPU 3/KVM cpuset=vcpu3 mems_allowed=0
[2694520.622737] CPU: 14 PID: 169362 Comm: CPU 3/KVM Not tainted 4.9.0-0.bpo.6-amd64 #1 Debian 4.9.88-1+deb9u1~bpo8+1
[2694520.622859] Hardware name: Dell Inc. PowerEdge FC640/05YC4P, BIOS 1.6.11 11/20/2018
[2694520.622977] 0000000000000000 ffffffff9b53047b ffffab422331f838 ffff8e400aa50800
[2694520.623402] ffffffff9b402ea5 0000000000000000 0000000000000000 ffffffff9b39b4ab
[2694520.623827] ffff8e413fffccc0 0000000c3fffb5c0 ffff8e413fffccf0 ffffffff9b38a0f4
[2694520.624250] Call Trace:
[2694520.624350] [<ffffffff9b53047b>] ? dump_stack+0x5a/0x6f
[2694520.624454] [<ffffffff9b402ea5>] ? dump_header+0x85/0x212
[2694520.624557] [<ffffffff9b39b4ab>] ? do_try_to_free_pages+0x2ab/0x320
[2694520.624661] [<ffffffff9b38a0f4>] ? get_page_from_freelist+0x144/0xa90
[2694520.624764] [<ffffffff9b386268>] ? oom_kill_process+0x228/0x3e0
[2694520.624867] [<ffffffff9b2838ea>] ? has_capability_noaudit+0x1a/0x20
[2694520.624969] [<ffffffff9b385eee>] ? oom_badness+0xee/0x160
[2694520.625069] [<ffffffff9b3866cc>] ? out_of_memory+0x10c/0x4b0
[2694520.625171] [<ffffffff9b38b847>] ? __alloc_pages_slowpath+0xa47/0xb80
[2694520.625274] [<ffffffff9b38bb8a>] ? __alloc_pages_nodemask+0x20a/0x270
[2694520.625378] [<ffffffff9b3de780>] ? alloc_pages_vma+0xb0/0x240
[2694520.625481] [<ffffffff9b2666df>] ? pte_alloc_one+0x2f/0x40
[2694520.625584] [<ffffffff9b3bac50>] ? handle_mm_fault+0x1220/0x15b0
[2694520.625687] [<ffffffff9b22fbc5>] ? sched_clock+0x5/0x10
[2694520.625788] [<ffffffff9b3b300a>] ? follow_page_pte+0x4aa/0x660
[2694520.625889] [<ffffffff9b3b37c2>] ? __get_user_pages+0x142/0x670
[2694520.625991] [<ffffffff9b2678e3>] ? gup_pud_range+0x123/0x220
[2694520.626092] [<ffffffff9b3b4116>] ? get_user_pages+0x56/0x60
[2694520.626217] [<ffffffffc0bf9ccb>] ? __gfn_to_pfn_memslot+0x18b/0x430 [kvm]
[2694520.626328] [<ffffffffc0c1a718>] ? mmu_set_spte+0x118/0x280 [kvm]
[2694520.626438] [<ffffffffc0c1f21a>] ? try_async_pf+0x6a/0x240 [kvm]
[2694520.626547] [<ffffffffc0c36be4>] ? kvm_mtrr_check_gfn_range_consistency+0x64/0x120 [kvm]
[2694520.626671] [<ffffffffc0bfbb6d>] ? kvm_host_page_size+0x8d/0xa0 [kvm]
[2694520.626781] [<ffffffffc0c21005>] ? tdp_page_fault+0x135/0x290 [kvm]
[2694520.626886] [<ffffffffc3bfd35d>] ? vmx_vcpu_run+0x2cd/0x8b0 [kvm_intel]
[2694520.626996] [<ffffffffc0c1887f>] ? kvm_mmu_page_fault+0x5f/0xe0 [kvm]
[2694520.627099] [<ffffffffc3bf76ef>] ? vmx_handle_exit+0x1cf/0x1460 [kvm_intel]
[2694520.627217] [<ffffffffc3bfd351>] ? vmx_vcpu_run+0x2c1/0x8b0 [kvm_intel]
[2694520.627320] [<ffffffffc3bfd35d>] ? vmx_vcpu_run+0x2cd/0x8b0 [kvm_intel]
[2694520.627424] [<ffffffffc3bfd351>] ? vmx_vcpu_run+0x2c1/0x8b0 [kvm_intel]
[2694520.627527] [<ffffffffc3bfd35d>] ? vmx_vcpu_run+0x2cd/0x8b0 [kvm_intel]
[2694520.627630] [<ffffffffc3bfd351>] ? vmx_vcpu_run+0x2c1/0x8b0 [kvm_intel]
[2694520.627734] [<ffffffff9b2eb055>] ? hrtimer_cancel+0x15/0x20
[2694520.627842] [<ffffffffc0c135c1>] ? kvm_arch_vcpu_ioctl_run+0x711/0x15e0 [kvm]
[2694520.627967] [<ffffffffc0c0d3dd>] ? kvm_arch_vcpu_load+0x5d/0x290 [kvm]
[2694520.628074] [<ffffffffc0bf9206>] ? kvm_vcpu_ioctl+0x316/0x5c0 [kvm]
[2694520.628178] [<ffffffff9b41beae>] ? do_vfs_ioctl+0x9e/0x610
[2694520.628280] [<ffffffff9b2fd723>] ? SyS_futex+0x83/0x180
[2694520.628380] [<ffffffff9b41c496>] ? SyS_ioctl+0x76/0x90
[2694520.628483] [<ffffffff9b203746>] ? exit_to_usermode_loop+0xa6/0xc0
[2694520.628585] [<ffffffff9b203bd1>] ? do_syscall_64+0x91/0x1a0
[2694520.628688] [<ffffffff9b8137ce>] ? entry_SYSCALL_64_after_swapgs+0x58/0xc6
[2694520.628798] Mem-Info:
[2694520.628900] active_anon:19484182 inactive_anon:976805 isolated_anon:0
active_file:1259064 inactive_file:742110 isolated_file:0
unevictable:0 dirty:17 writeback:8 unstable:0
slab_reclaimable:103531 slab_unreclaimable:44915
mapped:53878 shmem:1856432 pagetables:40512 bounce:0
free:1480897 free_pcp:150 free_cma:0
[2694520.629121] Node 0 active_anon:44722192kB inactive_anon:2229732kB active_file:364kB inactive_file:556kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:153576kB dirty:52kB writeback:48kB shmem:4630156kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 36163584kB writeback_tmp:0kB unstable:0kB pages_scanned:12074 all_unreclaimable? yes
[2694520.629318] Node 1 active_anon:33214536kB inactive_anon:1677488kB active_file:5035892kB inactive_file:2967884kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:61936kB dirty:16kB writeback:0kB shmem:2795572kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 30715904kB writeback_tmp:0kB unstable:0kB pages_scanned:0 all_unreclaimable? no
[2694520.629515] Node 0 DMA free:15836kB min:12kB low:24kB high:36kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:15920kB managed:15836kB mlocked:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
[2694520.629789] lowmem_reserve[]: 0 1461 46758 46758 46758
[2694520.630350] Node 0 DMA32 free:182392kB min:1400kB low:2896kB high:4392kB active_anon:1309300kB inactive_anon:4kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:1562400kB managed:1496832kB mlocked:0kB slab_reclaimable:0kB slab_unreclaimable:80kB kernel_stack:0kB pagetables:0kB bounce:0kB free_pcp:168kB local_pcp:168kB free_cma:0kB
[2694520.630625] lowmem_reserve[]: 0 0 45296 45296 45296
[2694520.631186] Node 0 Normal free:43164kB min:43424kB low:89804kB high:136184kB active_anon:43412956kB inactive_anon:2229728kB active_file:364kB inactive_file:556kB unevictable:0kB writepending:100kB present:47185920kB managed:46388056kB mlocked:0kB slab_reclaimable:41988kB slab_unreclaimable:94568kB kernel_stack:11416kB pagetables:38000kB bounce:0kB free_pcp:432kB local_pcp:300kB free_cma:0kB
[2694520.631467] lowmem_reserve[]: 0 0 0 0 0
[2694520.632029] Node 0 DMA: 1*4kB (U) 1*8kB (U) 1*16kB (U) 0*32kB 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (M) 3*4096kB (M) = 15836kB
[2694520.633846] Node 0 DMA32: 2*4kB (UM) 6*8kB (U) 16*16kB (UM) 18*32kB (UM) 20*64kB (UM) 16*128kB (UM) 16*256kB (UM) 10*512kB (UM) 7*1024kB (UM) 3*2048kB (UM) 38*4096kB (UE) = 182392kB
[2694520.635840] Node 0 Normal: 11004*4kB (U) 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 44016kB
[2694520.637039] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB
[2694520.637158] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
[2694520.637277] Node 1 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB
[2694520.637396] Node 1 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
[2694520.637515] 3857679 total pagecache pages
[2694520.637613] 0 pages in swap cache
[2694520.637710] Swap cache stats: add 0, delete 0, find 0/0
[2694520.637809] Free swap = 0kB
[2694520.637904] Total swap = 0kB
[2694520.638001] 24773972 pages RAM
[2694520.638096] 0 pages HighMem/MovableOnly
[2694520.638194] 418304 pages reserved
[2694520.638290] 0 pages hwpoisoned
[2694520.638386] [ pid ] uid tgid total_vm rss nr_ptes nr_pmds swapents oom_score_adj name
[2694520.638527] [ 534] 0 534 450159 438149 884 4 0 0 systemd-journal
[2694520.638649] [ 537] 0 537 10407 903 23 3 0 -1000 systemd-udevd
[2694520.638774] [ 4120] 0 4120 6876 711 18 3 0 0 cron
[2694520.638894] [ 4127] 0 4127 13873 1143 31 3 0 0 lldpd
[2694520.639014] [ 4133] 0 4133 6081 948 16 3 0 0 smartd
[2694520.639135] [ 4140] 107 4140 13873 804 28 3 0 0 lldpd
[2694520.639255] [ 4152] 0 4152 21329 410 11 4 0 0 numad
[2694520.639374] [ 4174] 106 4174 13634 955 27 3 0 -900 dbus-daemon
[2694520.639495] [ 4204] 0 4204 65785 2542 36 3 0 0 rsyslogd
[2694520.639615] [ 4212] 0 4212 1064 392 8 3 0 0 acpid
[2694520.639740] [ 4304] 0 4304 4980 688 14 3 0 0 irqbalance
[2694520.639877] [ 4331] 0 4331 9662 830 21 3 0 0 systemd-logind
[2694520.639998] [ 4342] 0 4342 22566 885 45 3 0 0 login
[2694520.640118] [ 4377] 0 4377 39303 2211 78 3 0 0 sssd
[2694520.640238] [ 4385] 0 4385 47116 4362 89 3 0 0 sssd_be
[2694520.640359] [ 4391] 0 4391 38705 8409 81 3 0 0 sssd_nss
[2694520.640479] [ 4393] 0 4393 33094 2115 69 3 0 0 sssd_pam
[2694520.643578] [ 4396] 0 4396 36074 2643 73 3 0 0 sssd_ssh
[2694520.643702] [ 4424] 0 4424 13795 1365 32 3 0 -1000 sshd
[2694520.643821] [ 4747] 0 4747 288387 31705 116 4 0 0 dsm_sa_datamgrd
[2694520.643943] [ 4802] 0 4802 6851 911 20 3 0 0 systemd
[2694520.644062] [ 4804] 0 4804 14160 619 33 3 0 0 (sd-pam)
[2694520.644182] [ 4806] 0 4806 5472 956 15 3 0 0 bash
[2694520.644301] [ 4885] 0 4885 49129 2092 38 3 0 0 dsm_sa_eventmgr
[2694520.644422] [ 4891] 0 4891 147634 4548 58 3 0 0 dsm_sa_datamgrd
[2694520.644544] [ 4907] 0 4907 85326 2386 49 3 0 0 dsm_sa_snmpd
[2694520.644664] [ 4932] 0 4932 7924 441 18 3 0 0 dsm_om_connsvcd
[2694520.644786] [ 4933] 0 4933 1825659 59988 369 11 0 0 dsm_om_connsvcd
[2694520.644907] [ 5090] 0 5090 24008 590 19 3 0 0 chronyd
[2694520.645027] [ 5095] 0 5095 24525 922 45 3 0 0 sfcbd
[2694520.645146] [ 5096] 0 5096 20395 645 42 3 0 0 sfcbd
[2694520.645265] [ 5098] 0 5098 22444 240 43 3 0 0 sfcbd
[2694520.645385] [ 5099] 0 5099 57812 1103 47 3 0 0 sfcbd
[2694520.645504] [ 5103] 0 5103 39380 896 46 3 0 0 sfcbd
[2694520.645624] [ 5109] 0 5109 58327 1179 48 3 0 0 sfcbd
[2694520.645743] [ 5469] 999 5469 772251 6400 134 8 0 0 node_exporter
[2694520.645864] [ 6174] 0 6174 18132 5326 39 3 0 0 gunicorn
[2694520.645984] [ 6177] 0 6177 118255 9612 94 3 0 0 gunicorn
[2694520.646104] [ 6199] 0 6199 41815 1743 51 3 0 -900 virtlogd
[2694520.646224] [ 6213] 0 6213 3360 561 13 3 0 0 systemd-machine
[2694520.646346] [ 6321] 64055 6321 2851050 1226801 2542 14 0 0 qemu-system-x86
[2694520.646470] [ 6736] 998 6736 10351 5573 25 5 0 0 consul
[2694520.646589] [ 6749] 109 6749 1262714 27335 251 10 0 0 agent
[2694520.646709] [181514] 0 181514 285817 6549 157 5 0 0 libvirtd
[2694520.646831] [169357] 64055 169357 3581358 2495043 5128 17 0 0 qemu-system-x86
[2694520.646953] Out of memory: Kill process 169357 (qemu-system-x86) score 194 or sacrifice child
[2694520.647262] Killed process 169357 (qemu-system-x86) total-vm:14325432kB, anon-rss:9961732kB, file-rss:40kB, shmem-rss:18400kB
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment