The Log monitoring feature of Zabbix server is working fine. But there is one major issue in the system. The below step will help you to reproduce the issue.
(1) Create a new Log monitoring Item with following values.
(2) Then create a trigger for this Item.
(3) Create an action to send mail from the Trigger.
(4) Insert the below line in the /tmp/test.log file.
2012-08-06 13:35:05,237 ERROR [nucleusNamespace.atg.commerce.catalog.CatalogTools] (ajp-0.0.0.0-8509-4) Not parent organization for HOST found.
(5) This will trigger a mail to the configured email ID.
(6) Append the log with same line in 100 times ( The Time stamp can be different (2012-08-06 13:35:05,237))
(7) This will send 100 mails to the configured email ID.
We need a mechanisam to limit this kind of Duplicate alerts. In this situation, instead of sending 100 mails, the Zabbix server first send a mail about the ERROR. Then, if the ERROR repeats, it should send another message something like below.
"The above message repeated 100 times."
This will prevent Alert storms, in case of duplicate ERROR occured in Application logs. We have similar situation happended recently with our JBOSS application server. Due to some Bug in our application, aroung 1 lakhs ERROR is generated in the log file and this will fired same amount of emails.
So as a work around we are Using policyd to control mails from Zabbix Log monitoring.