[ZBX-8542] A lot of semop and time functions while processing triggers Created: 2014 Jul 29 Updated: 2017 May 30 Resolved: 2014 Sep 26 |
|
Status: | Closed |
Project: | ZABBIX BUGS AND ISSUES |
Component/s: | Server (S) |
Affects Version/s: | 2.2.5 |
Fix Version/s: | 2.3.4, 2.5.0 |
Type: | Incident report | Priority: | Trivial |
Reporter: | Alexey Pustovalov | Assignee: | Unassigned |
Resolution: | Fixed | Votes: | 0 |
Labels: | performance | ||
Remaining Estimate: | Not Specified | ||
Time Spent: | Not Specified | ||
Original Estimate: | Not Specified |
Attachments: | strace.log | ||||
Issue Links: |
|
Description |
Zabbix server uses a lot of slow time function (next snipped is strace output) (time(NULL) = 1406628281) while processing one iteration of history syncer (triggers evaluation). Please look log file to get more information. |
Comments |
Comment by Andris Zeila [ 2014 Aug 18 ] | ||||||||||||||||
(1) Create user macro cache that will be used to scan trigger expressions for user macros, resolve them with a single configuration cache lock and then use the resolved values in substitute_simple_macros() function. RESOLVED in development branch svn://svn.zabbix.com/branches/dev/ZBX-8542 <dimir> CLOSED | ||||||||||||||||
Comment by Andris Zeila [ 2014 Aug 20 ] | ||||||||||||||||
(2) Create trigger macro cache that will be used for dependency checks and trigger value updates. wiper RESOLVED <dimir> was obsoleted, CLOSED | ||||||||||||||||
Comment by richlv [ 2014 Sep 01 ] | ||||||||||||||||
reopening, there's subissue 2 wiper forgot to change its status | ||||||||||||||||
Comment by dimir [ 2014 Sep 23 ] | ||||||||||||||||
(3) Trigger expressions processing became slower. Some testing before and after these changes. Each host contains 10 items and 10 triggers. Every trigger expression contains all 10 items from the host. Every item contains user macro (from the host) as parameter (system.run[echo {$ZZZ<NUM>}). Check out the time spent on evaluating 2000 triggers. It doubled at least after the changes.
BTW, about 1000 items are queued for 10 seconds. Was tested on my workstation which has not much power and memory resources. wiper On my system with loglevel set to 3 and 5000 hosts the macro cache had slight overhead (hard to evaluate, could be about 5%) during trigger expression processing, but the number of configration cache locks was greatly reduced. Meanwhile I optimized the trigger expression function lookups, which gives nice performance improvement in this test scenario. <dimir> CLOSED | ||||||||||||||||
Comment by dimir [ 2014 Nov 04 ] | ||||||||||||||||
(4) [D] More like a suggestion. Would be great if you could describe in the code using block-scheme or something the way trigger expressions are now evaluated. How macro cache and other helper data structures are generated and used. wiper RESOLVED in r50496 <dimir> Looks great! CLOSED | ||||||||||||||||
Comment by dimir [ 2014 Nov 04 ] | ||||||||||||||||
Great performance improvement on substitute_functions()! Now trigger evaluation time dropped dramatically (I used DebugLevel=3 here):
| ||||||||||||||||
Comment by dimir [ 2014 Nov 06 ] | ||||||||||||||||
Successfully tested. | ||||||||||||||||
Comment by Andris Zeila [ 2014 Nov 07 ] | ||||||||||||||||
Released in:
| ||||||||||||||||
Comment by Andris Zeila [ 2014 Nov 07 ] | ||||||||||||||||
(5) Documented at https://www.zabbix.com/documentation/3.0/manual/introduction/whatsnew300?&#daemon_improvements <dimir> Added bit more details, please check. wiper Thanks, CLOSED | ||||||||||||||||
Comment by Andris Zeila [ 2014 Nov 11 ] | ||||||||||||||||
(6) Whitespace should not be removed during expression evaluations. See the patch:
Index: src/libs/zbxserver/expression.c
===================================================================
--- src/libs/zbxserver/expression.c (revision 50540)
+++ src/libs/zbxserver/expression.c (working copy)
@@ -3865,7 +3865,6 @@
tr = (DC_TRIGGER *)triggers->values[i];
tr->expression = zbx_strdup(NULL, tr->expression_orig);
- zbx_remove_whitespace(tr->expression);
/* add trigger macros to user macro cache */
zbx_umc_add_expression(¯o_cache, tr->triggerid, tr->expression);
The whitespace removal apparently slipped from 2.2 branch, which was the branch the development was based on initially. <dimir> Right, CLOSED | ||||||||||||||||
Comment by Andris Zeila [ 2014 Nov 11 ] | ||||||||||||||||
Fixed in development branch svn://svn.zabbix.com/branches/dev/ZBX-8542 | ||||||||||||||||
Comment by Andris Zeila [ 2014 Nov 11 ] | ||||||||||||||||
Released in:
|