[ZBX-5421] alert rate limiting Created: 2012 Aug 06  Updated: 2017 May 30  Resolved: 2012 Aug 07

Status: Closed
Project: ZABBIX BUGS AND ISSUES
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Problem report Priority: Critical
Reporter: Arun Sasidhar Assignee: Unassigned
Resolution: Duplicate Votes: 0
Labels: alerts, duplicates, logmonitoring
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
duplicates ZBXNEXT-39 Recurring triggers - limit notifications Open

 Description   

The Log monitoring feature of Zabbix server is working fine. But there is one major issue in the system. The below step will help you to reproduce the issue.

To reproduce:

(1) Create a new Log monitoring Item with following values.

log[/tmp/test.log,ERROR,,100]

(2) Then create a trigger for this Item.
(3) Create an action to send mail from the Trigger.

(4) Insert the below line in the /tmp/test.log file.

2012-08-06 13:35:05,237 ERROR [nucleusNamespace.atg.commerce.catalog.CatalogTools] (ajp-0.0.0.0-8509-4) Not parent organization for HOST found.

(5) This will trigger a mail to the configured email ID.

(6) Append the log with same line in 100 times ( The Time stamp can be different (2012-08-06 13:35:05,237))
(7) This will send 100 mails to the configured email ID.

We need a mechanisam to limit this kind of Duplicate alerts. In this situation, instead of sending 100 mails, the Zabbix server first send a mail about the ERROR. Then, if the ERROR repeats, it should send another message something like below.

"The above message repeated 100 times."

This will prevent Alert storms, in case of duplicate ERROR occured in Application logs. We have similar situation happended recently with our JBOSS application server. Due to some Bug in our application, aroung 1 lakhs ERROR is generated in the log file and this will fired same amount of emails.

So as a work around we are Using policyd to control mails from Zabbix Log monitoring.



 Comments   
Comment by richlv [ 2012 Aug 06 ]

hmm. i was sure this was reported already, but i just can't find it now

Comment by Oleksii Zagorskyi [ 2012 Aug 07 ]

Here is it - ZBXNEXT-39.
Feel free to add there any ideas which present here but missing there.

This closed as a duplicate.

Generated at Thu Apr 25 15:42:26 EEST 2024 using Jira 9.12.4#9120004-sha1:625303b708afdb767e17cb2838290c41888e9ff0.