[ZBX-9428] Low alerter performance on 2.4.2 vs 1.8.15 Created: 2015 Mar 24 Updated: 2017 May 30 Resolved: 2015 Jul 09 |
|
Status: | Closed |
Project: | ZABBIX BUGS AND ISSUES |
Component/s: | Server (S) |
Affects Version/s: | 2.4.2 |
Fix Version/s: | None |
Type: | Incident report | Priority: | Major |
Reporter: | Viktor | Assignee: | Unassigned |
Resolution: | Cannot Reproduce | Votes: | 0 |
Labels: | None | ||
Remaining Estimate: | Not Specified | ||
Time Spent: | Not Specified | ||
Original Estimate: | Not Specified |
Attachments: |
![]() |
Description |
Activation of 700+ triggers make alerter go to 100% cpu load and 5 minutes delay of messages sending. Alerter on zabbix 1.8.15 works fine on this load. Is it possible to improve alerter performance? |
Comments |
Comment by Marc [ 2015 Mar 26 ] |
One option could be |
Comment by Aleksandrs Saveljevs [ 2015 Mar 27 ] |
What kind of notifications are you using? Does "5 minutes delay of messages sending" mean that at most 1 notification is sent in 5 minutes? |
Comment by Viktor [ 2015 Mar 27 ] |
We are using Email media type to 127.0.0.1. |
Comment by Aleksandrs Saveljevs [ 2015 Mar 27 ] |
Would it be possible to provide a DebugLevel=4 log showing what alerter process is doing? Note that since Zabbix 2.4.0 it is possible to change debug level for separate processes at runtime without restarting the server (see http://blog.zabbix.com/zabbix-2-4-features-part-6-runtime-loglevel-changing/). |
Comment by Viktor [ 2015 Mar 30 ] |
I was not able to reproduse it today (i will try later), but I`ve noticed 30-seconds delay after enabling trigger and sending alert. maybe this is the reason? is it normal 'idle 30 sec'? Как вы относитесь к комментариям на русском? |
Comment by richlv [ 2015 Mar 30 ] |
30 seconds sound normal, that's the default frequency at which zabbix tries to send the messages - parameter SenderFrequency in the server config file |
Comment by richlv [ 2015 Mar 30 ] |
also please see https://www.zabbix.org/wiki/Docs/bug_reporting_guidelines#Reporting_an_issue |
Comment by Viktor [ 2015 Mar 31 ] |
I still trying to reproduce this case, and I've noticed that the speed of mail sending decreasing. |
Comment by Viktor [ 2015 Apr 01 ] |
Here is log file alert.log |
Comment by Aleksandrs Saveljevs [ 2015 Apr 01 ] |
According to "alert.log", the slowness is in send_email() function. If we grep the log by this function, we will see the following: ... 1436:20150331:120922.418 In send_email() smtp_server:'127.0.0.1' 1436:20150331:120922.457 End of send_email():SUCCEED 1436:20150331:120922.458 In send_email() smtp_server:'127.0.0.1' 1436:20150331:120922.497 End of send_email():SUCCEED 1436:20150331:120922.497 In send_email() smtp_server:'127.0.0.1' 1436:20150331:120922.537 End of send_email():SUCCEED 1436:20150331:120922.537 In send_email() smtp_server:'127.0.0.1' 1436:20150331:120923.577 End of send_email():SUCCEED 1436:20150331:120923.577 In send_email() smtp_server:'127.0.0.1' 1436:20150331:120924.619 End of send_email():SUCCEED 1436:20150331:120924.620 In send_email() smtp_server:'127.0.0.1' 1436:20150331:120925.657 End of send_email():SUCCEED ... It can be seen that before 12:09:22, it takes around 40 ms to send an email. After 12:09:22, it takes around 1s 40 ms to send an email. So my guess is that at some point the email server is starting to limit the rate at which Zabbix server can send emails. Could you please check that? |