-
Incident report
-
Resolution: Fixed
-
Critical
-
3.0.0alpha4
Seems to be broken in ZBXNEXT-444.
Trigger expression:
{svnzbx-rw:log[/var/log/syslog,"oom-killer",,,skip].strlen()} <> 0
Log file:
Nov 10 19:17:01 ********* /USR/SBIN/CRON[21216]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly) Nov 10 20:17:01 ********* /USR/SBIN/CRON[21712]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly) Nov 10 21:17:01 ********* /USR/SBIN/CRON[22207]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly) Nov 10 22:17:01 ********* /USR/SBIN/CRON[22704]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly) Nov 10 23:17:01 ********* /USR/SBIN/CRON[23199]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly) Nov 11 00:17:01 ********* /USR/SBIN/CRON[23697]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly) Nov 11 01:17:01 ********* /USR/SBIN/CRON[24195]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly) Nov 11 02:17:01 ********* /USR/SBIN/CRON[24691]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly) Nov 11 03:17:01 ********* /USR/SBIN/CRON[25359]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly) Nov 11 04:06:34 ********* dhclient: DHCPREQUEST on eth0 to *********** port 67 Nov 11 04:06:34 ********* dhclient: DHCPACK from *********** Nov 11 04:06:34 ********* dhclient: bound to *********** -- renewal in 43001 seconds. Nov 11 04:17:01 ********* /USR/SBIN/CRON[25864]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly) Nov 11 05:17:01 ********* /USR/SBIN/CRON[26360]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly) Nov 11 06:17:01 ********* /USR/SBIN/CRON[26857]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly) Nov 11 06:25:01 ********* /USR/SBIN/CRON[26936]: (root) CMD (test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.daily )) Nov 11 06:25:02 ********* rsyslogd: [origin software="rsyslogd" swVersion="4.6.4" x-pid="489" x-info="http://www.rsyslog.com"] rsyslogd was HUPed, type 'lightweight'. Nov 11 07:17:01 ********* /USR/SBIN/CRON[27443]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly) Nov 11 08:17:01 ********* /USR/SBIN/CRON[27939]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly) Nov 11 09:17:01 ********* /USR/SBIN/CRON[28453]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly) Nov 11 10:17:01 ********* /USR/SBIN/CRON[28993]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly)
Events:
| Time | Host | Description | Status | Severity | Duration |
|---|---|---|---|---|---|
| 2015-11-11 10:17:19 | ********* | Process has been killed on ********* by OOM Killer | PROBLEM | Disaster | 42m 57s |
| 2015-11-11 09:17:19 | ********* | Process has been killed on ********* by OOM Killer | PROBLEM | Disaster | 1h |
| 2015-11-10 21:17:18 | ********* | Process has been killed on ********* by OOM Killer | PROBLEM | Disaster | 12h 1s |
| 2015-11-10 20:17:18 | ********* | Process has been killed on ********* by OOM Killer | PROBLEM | Disaster | 1h |
| 2015-11-10 19:17:18 | ********* | Process has been killed on ********* by OOM Killer | PROBLEM | Disaster | 1h |
History:
+-------+--------+------------+-----------+--------+----------+----------------------------------------------------------------------------------------------------------------------------+------------+-----------+ | id | itemid | clock | timestamp | source | severity | value | logeventid | ns | +-------+--------+------------+-----------+--------+----------+----------------------------------------------------------------------------------------------------------------------------+------------+-----------+ | 14774 | 29729 | 1446733421 | 0 | | 0 | Nov 5 16:23:37 ********* kernel: [15306428.325179] zabbix_agentd invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0 | 0 | 521845088 | +-------+--------+------------+-----------+--------+----------+----------------------------------------------------------------------------------------------------------------------------+------------+-----------+