1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
| Sep 21 16:12:28 localhost kernel: mysqld invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0, oom_score_adj=0
Sep 21 16:12:28 localhost kernel: mysqld cpuset=/ mems_allowed=0
Sep 21 16:12:28 localhost kernel: Pid: 20824, comm: mysqld Not tainted 2.6.32-573.el6.x86_64
Sep 21 16:12:28 localhost kernel: Call Trace:
Sep 21 16:12:28 localhost kernel: [<ffffffff810d6dd1>] ? cpuset_print_task_mems_allowed+0x91/0xb0
Sep 21 16:12:28 localhost kernel: [<ffffffff8112a5d0>] ? dump_header+0x90/0x1b0
Sep 21 16:12:28 localhost kernel: [<ffffffff81232a7c>] ? security_real_capable_noaudit+0x3c/0x70
Sep 21 16:12:28 localhost kernel: [<ffffffff8112aa52>] ? oom_kill_process+0x82/0x2a0
Sep 21 16:12:28 localhost kernel: [<ffffffff8112a991>] ? select_bad_process+0xe1/0x120
Sep 21 16:12:28 localhost kernel: [<ffffffff8112ae90>] ? out_of_memory+0x220/0x3c0
Sep 21 16:12:28 localhost kernel: [<ffffffff8113786c>] ? __alloc_pages_nodemask+0x93c/0x950
Sep 21 16:12:28 localhost kernel: [<ffffffff8117035a>] ? alloc_pages_vma+0x9a/0x150
Sep 21 16:12:28 localhost kernel: [<ffffffff81152b1d>] ? handle_pte_fault+0x73d/0xb20
Sep 21 16:12:28 localhost kernel: [<ffffffff810537b7>] ? pte_alloc_one+0x37/0x50
Sep 21 16:12:28 localhost kernel: [<ffffffff8118be29>] ? do_huge_pmd_anonymous_page+0xb9/0x3b0
Sep 21 16:12:28 localhost kernel: [<ffffffff81153199>] ? handle_mm_fault+0x299/0x3d0
Sep 21 16:12:28 localhost kernel: [<ffffffff81066053>] ? perf_event_task_sched_out+0x33/0x70
Sep 21 16:12:28 localhost kernel: [<ffffffff8104f156>] ? __do_page_fault+0x146/0x500
Sep 21 16:12:28 localhost kernel: [<ffffffff8153817e>] ? thread_return+0x4e/0x7d0
Sep 21 16:12:28 localhost kernel: [<ffffffff8153e78e>] ? do_page_fault+0x3e/0xa0
Sep 21 16:12:28 localhost kernel: [<ffffffff8153bb35>] ? page_fault+0x25/0x30
Sep 21 16:12:28 localhost kernel: Mem-Info:
Sep 21 16:12:28 localhost kernel: Node 0 DMA per-cpu:
Sep 21 16:12:28 localhost kernel: CPU 0: hi: 0, btch: 1 usd: 0
Sep 21 16:12:28 localhost kernel: CPU 1: hi: 0, btch: 1 usd: 0
Sep 21 16:12:28 localhost kernel: CPU 2: hi: 0, btch: 1 usd: 0
Sep 21 16:12:28 localhost kernel: CPU 3: hi: 0, btch: 1 usd: 0
Sep 21 16:12:28 localhost kernel: Node 0 DMA32 per-cpu:
Sep 21 16:12:28 localhost kernel: CPU 0: hi: 186, btch: 31 usd: 0
Sep 21 16:12:28 localhost kernel: CPU 1: hi: 186, btch: 31 usd: 0
Sep 21 16:12:28 localhost kernel: CPU 2: hi: 186, btch: 31 usd: 0
Sep 21 16:12:28 localhost kernel: CPU 3: hi: 186, btch: 31 usd: 0
Sep 21 16:12:28 localhost kernel: Node 0 Normal per-cpu:
Sep 21 16:12:28 localhost kernel: CPU 0: hi: 186, btch: 31 usd: 0
Sep 21 16:12:28 localhost kernel: CPU 1: hi: 186, btch: 31 usd: 30
Sep 21 16:12:28 localhost kernel: CPU 2: hi: 186, btch: 31 usd: 0
Sep 21 16:12:28 localhost kernel: CPU 3: hi: 186, btch: 31 usd: 0
Sep 21 16:12:28 localhost kernel: active_anon:1555339 inactive_anon:332686 isolated_anon:0
Sep 21 16:12:28 localhost kernel: active_file:21589 inactive_file:21442 isolated_file:0
Sep 21 16:12:28 localhost kernel: unevictable:0 dirty:39195 writeback:0 unstable:0
Sep 21 16:12:28 localhost kernel: free:25816 slab_reclaimable:8890 slab_unreclaimable:10707
Sep 21 16:12:28 localhost kernel: mapped:3508 shmem:37 pagetables:11523 bounce:0
Sep 21 16:12:28 localhost kernel: Node 0 DMA free:15728kB min:124kB low:152kB high:184kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15336kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Sep 21 16:12:28 localhost kernel: lowmem_reserve[]: 0 2970 8020 8020
Sep 21 16:12:28 localhost kernel: Node 0 DMA32 free:45136kB min:24980kB low:31224kB high:37468kB active_anon:2112200kB inactive_anon:584284kB active_file:10564kB inactive_file:10124kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3041728kB mlocked:0kB dirty:19804kB writeback:0kB mapped:68kB shmem:4kB slab_reclaimable:5484kB slab_unreclaimable:1668kB kernel_stack:176kB pagetables:1800kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:10688 all_unreclaimable? no
Sep 21 16:12:28 localhost kernel: lowmem_reserve[]: 0 0 5050 5050
Sep 21 16:12:28 localhost kernel: Node 0 Normal free:42400kB min:42472kB low:53088kB high:63708kB active_anon:4109156kB inactive_anon:746460kB active_file:75792kB inactive_file:75644kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:5171200kB mlocked:0kB dirty:136976kB writeback:0kB mapped:13964kB shmem:144kB slab_reclaimable:30076kB slab_unreclaimable:41160kB kernel_stack:6240kB pagetables:44292kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:37952 all_unreclaimable? no
Sep 21 16:12:28 localhost kernel: lowmem_reserve[]: 0 0 0 0
Sep 21 16:12:28 localhost kernel: Node 0 DMA: 2*4kB 1*8kB 2*16kB 2*32kB 2*64kB 1*128kB 0*256kB 0*512kB 1*1024kB 1*2048kB 3*4096kB = 15728kB
Sep 21 16:12:28 localhost kernel: Node 0 DMA32: 599*4kB 470*8kB 304*16kB 140*32kB 60*64kB 40*128kB 51*256kB 13*512kB 1*1024kB 0*2048kB 0*4096kB = 45196kB
Sep 21 16:12:28 localhost kernel: Node 0 Normal: 1635*4kB 1018*8kB 574*16kB 304*32kB 93*64kB 20*128kB 2*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 42620kB
Sep 21 16:12:28 localhost kernel: 59465 total pagecache pages
Sep 21 16:12:28 localhost kernel: 16337 pages in swap cache
Sep 21 16:12:28 localhost kernel: Swap cache stats: add 8300762, delete 8284425, find 1851709760/1852280532
Sep 21 16:12:28 localhost kernel: Free swap = 9912132kB
Sep 21 16:12:28 localhost kernel: Total swap = 10239996kB
Sep 21 16:12:28 localhost kernel: 2097151 pages RAM
Sep 21 16:12:28 localhost kernel: 89559 pages reserved
Sep 21 16:12:28 localhost kernel: 84494 pages shared
Sep 21 16:12:28 localhost kernel: 1927295 pages non-shared
Sep 21 16:12:28 localhost kernel: [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Sep 21 16:12:28 localhost kernel: [ 636] 0 636 2663 67 0 -17 -1000 udevd
Sep 21 16:12:28 localhost kernel: [ 1477] 0 1477 23289 81 1 -17 -1000 auditd
Sep 21 16:12:28 localhost kernel: [ 1511] 0 1511 62991 1202 2 0 0 rsyslogd
Sep 21 16:12:28 localhost kernel: [ 1602] 0 1602 4561 77 1 0 0 irqbalance
Sep 21 16:12:28 localhost kernel: [ 1620] 32 1620 4744 20 1 0 0 rpcbind
Sep 21 16:12:28 localhost kernel: [ 1642] 29 1642 5837 2 0 0 0 rpc.statd
Sep 21 16:12:28 localhost kernel: [ 1657] 0 1657 1197 90 1 0 0 mdadm
Sep 21 16:12:28 localhost kernel: [ 1684] 81 1684 5391 96 2 0 0 dbus-daemon
Sep 21 16:12:28 localhost kernel: [ 1705] 0 1705 47232 2 0 0 0 cupsd
Sep 21 16:12:28 localhost kernel: [ 1737] 0 1737 1020 1 0 0 0 acpid
Sep 21 16:12:28 localhost kernel: [ 1749] 68 1749 9494 175 0 0 0 hald
Sep 21 16:12:28 localhost kernel: [ 1750] 0 1750 5100 2 2 0 0 hald-runner
Sep 21 16:12:28 localhost kernel: [ 1782] 0 1782 5630 2 0 0 0 hald-addon-inpu
Sep 21 16:12:28 localhost kernel: [ 1805] 68 1805 4502 2 1 0 0 hald-addon-acpi
Sep 21 16:12:28 localhost kernel: [ 1824] 0 1824 96534 103 0 0 0 automount
Sep 21 16:12:28 localhost kernel: [ 1981] 0 1981 1566 1 1 0 0 mcelog
Sep 21 16:12:28 localhost kernel: [ 2089] 0 2089 20218 47 3 0 0 master
Sep 21 16:12:28 localhost kernel: [ 2096] 89 2096 20280 133 2 0 0 qmgr
Sep 21 16:12:28 localhost kernel: [ 2118] 0 2118 45214 2 1 0 0 abrtd
Sep 21 16:12:28 localhost kernel: [ 2156] 0 2156 5276 9 1 0 0 atd
Sep 21 16:12:28 localhost kernel: [ 2172] 0 2172 16082 46 2 0 0 certmonger
Sep 21 16:12:28 localhost kernel: [ 2195] 0 2195 1016 2 2 0 0 mingetty
Sep 21 16:12:28 localhost kernel: [ 2197] 0 2197 1016 2 0 0 0 mingetty
Sep 21 16:12:28 localhost kernel: [ 5013] 0 5013 92272 314 3 0 0 php
Sep 21 16:12:28 localhost kernel: [ 5030] 0 5030 91990 314 1 0 0 php
Sep 21 16:12:28 localhost kernel: [ 5140] 0 5140 92185 325 0 0 0 php
Sep 21 16:12:28 localhost kernel: [ 1427] 0 1427 2868 116 1 0 0 mysqld_safe
Sep 21 16:12:28 localhost kernel: [13252] 0 13252 25628 137 1 0 0 sshd
Sep 21 16:12:28 localhost kernel: [13279] 511 13279 26549 226 0 0 0 sshd
Sep 21 16:12:28 localhost kernel: [13284] 511 13284 14359 155 0 0 0 sftp-server
Sep 21 16:12:28 localhost kernel: [17029] 0 17029 25628 137 1 0 0 sshd
Sep 21 16:12:28 localhost kernel: [17042] 511 17042 25715 156 0 0 0 sshd
Sep 21 16:12:28 localhost kernel: [17044] 511 17044 14348 135 2 0 0 sftp-server
Sep 21 16:12:28 localhost kernel: [18981] 0 18981 97300 171 0 0 0 httpd
Sep 21 16:12:28 localhost kernel: [19000] 48 19000 100856 4499 3 0 0 httpd
Sep 21 16:12:28 localhost kernel: [19001] 48 19001 101126 5024 3 0 0 httpd
Sep 21 16:12:28 localhost kernel: [19002] 48 19002 101128 5015 3 0 0 httpd
Sep 21 16:12:28 localhost kernel: [19003] 48 19003 101184 5138 3 0 0 httpd
Sep 21 16:12:28 localhost kernel: [19004] 48 19004 101262 5173 3 0 0 httpd
Sep 21 16:12:28 localhost kernel: [19005] 48 19005 101113 5181 3 0 0 httpd
Sep 21 16:12:28 localhost kernel: [19006] 48 19006 103955 7278 0 0 0 httpd
Sep 21 16:12:28 localhost kernel: [19007] 48 19007 103892 5011 0 0 0 httpd
Sep 21 16:12:28 localhost kernel: [ 7033] 48 7033 101150 4990 0 0 0 httpd
Sep 21 16:12:28 localhost kernel: [ 9907] 48 9907 103906 5150 3 0 0 httpd
Sep 21 16:12:28 localhost kernel: [ 9943] 48 9943 100195 4298 0 0 0 httpd
Sep 21 16:12:28 localhost kernel: [ 9944] 48 9944 103931 4977 3 0 0 httpd
Sep 21 16:12:28 localhost kernel: [26057] 48 26057 101127 4790 1 0 0 httpd
Sep 21 16:12:28 localhost kernel: [26921] 48 26921 101067 4969 3 0 0 httpd
Sep 21 16:12:28 localhost kernel: [31980] 0 31980 25629 10 1 0 0 sshd
Sep 21 16:12:28 localhost kernel: [31982] 500 31982 25629 55 3 0 0 sshd
Sep 21 16:12:28 localhost kernel: [31983] 500 31983 27084 2 1 0 0 bash
Sep 21 16:12:28 localhost kernel: [32369] 500 32369 41514 2 3 0 0 su
Sep 21 16:12:28 localhost kernel: [ 355] 0 355 27104 146 2 0 0 bash
Sep 21 16:12:28 localhost kernel: [27397] 99 27397 3635 180 1 0 0 nmon
Sep 21 16:12:28 localhost kernel: [15221] 0 15221 25628 138 3 0 0 sshd
Sep 21 16:12:28 localhost kernel: [15496] 511 15496 25699 145 2 0 0 sshd
Sep 21 16:12:28 localhost kernel: [15497] 511 15497 14349 153 1 0 0 sftp-server
Sep 21 16:12:28 localhost kernel: [ 2380] 0 2380 25628 138 2 0 0 sshd
Sep 21 16:12:28 localhost kernel: [ 2627] 511 2627 25663 121 2 0 0 sshd
Sep 21 16:12:28 localhost kernel: [ 2629] 511 2629 14312 116 2 0 0 sftp-server
Sep 21 16:12:28 localhost kernel: [17337] 0 17337 29215 122 0 0 0 crond
Sep 21 16:12:28 localhost kernel: [17464] 0 17464 16558 44 0 -17 -1000 sshd
Sep 21 16:12:28 localhost kernel: [14863] 497 14863 1043341 101433 3 0 0 java
Sep 21 16:12:28 localhost kernel: [26148] 0 26148 1184739 199553 2 0 0 java
Sep 21 16:12:28 localhost kernel: [ 3941] 0 3941 2662 92 2 -17 -1000 udevd
Sep 21 16:12:28 localhost kernel: [ 3942] 0 3942 2662 92 1 -17 -1000 udevd
Sep 21 16:12:28 localhost kernel: [ 408] 498 408 2251205 1478808 0 0 0 mysqld
Sep 21 16:12:28 localhost kernel: [ 978] 0 978 25628 697 0 0 0 sshd
Sep 21 16:12:28 localhost kernel: [ 1029] 503 1029 25628 347 3 0 0 sshd
Sep 21 16:12:28 localhost kernel: [ 1038] 503 1038 27084 252 2 0 0 bash
Sep 21 16:12:28 localhost kernel: [ 1147] 503 1147 41511 362 1 0 0 su
Sep 21 16:12:28 localhost kernel: [ 1517] 0 1517 27084 266 1 0 0 bash
Sep 21 16:12:28 localhost kernel: [ 4617] 0 4617 35003 325 0 0 0 crond
Sep 21 16:12:28 localhost kernel: [ 4618] 0 4618 35003 325 1 0 0 crond
Sep 21 16:12:28 localhost kernel: [ 4619] 0 4619 35003 325 2 0 0 crond
Sep 21 16:12:28 localhost kernel: [ 4620] 0 4620 35003 325 0 0 0 crond
Sep 21 16:12:28 localhost kernel: [ 4621] 0 4621 35003 325 2 0 0 crond
Sep 21 16:12:28 localhost kernel: [ 4665] 0 4665 95366 7116 3 0 0 php
Sep 21 16:12:28 localhost kernel: [ 4666] 0 4666 20233 649 2 0 0 sendmail
Sep 21 16:12:28 localhost kernel: [ 4675] 0 4675 20224 650 2 0 0 postdrop
Sep 21 16:12:28 localhost kernel: [ 4689] 0 4689 95496 7123 3 0 0 php
Sep 21 16:12:28 localhost kernel: [ 4691] 0 4691 95431 7174 3 0 0 php
Sep 21 16:12:28 localhost kernel: [ 4692] 0 4692 95455 7122 2 0 0 php
Sep 21 16:12:28 localhost kernel: [ 4695] 0 4695 20233 651 0 0 0 sendmail
Sep 21 16:12:28 localhost kernel: [ 4696] 0 4696 20233 650 3 0 0 sendmail
Sep 21 16:12:28 localhost kernel: [ 4697] 0 4697 20233 649 2 0 0 sendmail
Sep 21 16:12:28 localhost kernel: [ 4699] 0 4699 20224 651 2 0 0 postdrop
Sep 21 16:12:28 localhost kernel: [ 4700] 0 4700 20224 651 3 0 0 postdrop
Sep 21 16:12:28 localhost kernel: [ 4701] 0 4701 20224 652 0 0 0 postdrop
Sep 21 16:12:28 localhost kernel: [ 4707] 0 4707 95507 7232 3 0 0 php
Sep 21 16:12:28 localhost kernel: [ 4709] 0 4709 20233 650 0 0 0 sendmail
Sep 21 16:12:28 localhost kernel: [ 4710] 0 4710 20224 651 0 0 0 postdrop
Sep 21 16:12:28 localhost kernel: [22878] 89 22878 20238 666 0 0 0 pickup
Sep 21 16:12:28 localhost kernel: [18462] 89 18462 20240 661 0 0 0 trivial-rewrite
Sep 21 16:12:28 localhost kernel: [18573] 89 18573 20247 673 0 0 0 bounce
Sep 21 16:12:28 localhost kernel: [18574] 0 18574 20312 793 0 0 0 local
Sep 21 16:12:28 localhost kernel: [20055] 0 20055 20312 793 0 0 0 local
Sep 21 16:12:28 localhost kernel: [20056] 89 20056 20274 714 1 0 0 cleanup
Sep 21 16:12:28 localhost kernel: [20058] 89 20058 20274 716 3 0 0 cleanup
Sep 21 16:12:28 localhost kernel: [20063] 89 20063 20247 671 3 0 0 bounce
Sep 21 16:12:28 localhost kernel: [20064] 89 20064 20274 714 0 0 0 cleanup
Sep 21 16:12:28 localhost kernel: [25666] 0 25666 20312 793 1 0 0 local
Sep 21 16:12:28 localhost kernel: [27049] 89 27049 20247 672 1 0 0 bounce
Sep 21 16:12:28 localhost kernel: [31585] 0 31585 92315 3716 0 0 0 php
Sep 21 16:12:28 localhost kernel: [31586] 0 31586 58126 993 3 0 0 php
Sep 21 16:12:28 localhost kernel: [31587] 0 31587 58126 993 0 0 0 php
Sep 21 16:12:28 localhost kernel: Out of memory: Kill process 408 (mysqld) score 324 or sacrifice child
Sep 21 16:12:28 localhost kernel: Killed process 408, UID 498, (mysqld) total-vm:9004820kB, anon-rss:5910380kB, file-rss:4852kB
|
