ZABBIX BUGS AND ISSUES
  1. ZABBIX BUGS AND ISSUES
  2. ZBX-7109

Very slow processing of low-level discovered triggers

    Details

      Issue Links

        Activity

        Hide
        Alexander Vladishev added a comment - - edited

        (1) Separated lld source code into several files

        Aleksandrs Saveljevs Looks good. Before closing, please take a look at r40087.

        Alexander Vladishev Thanks! CLOSED

        Available in branches/2.0 r40088.

        Show
        Alexander Vladishev added a comment - - edited (1) Separated lld source code into several files Aleksandrs Saveljevs Looks good. Before closing, please take a look at r40087. Alexander Vladishev Thanks! CLOSED Available in branches/2.0 r40088.
        Hide
        Alexander Vladishev added a comment -

        Related issue: ZBX-7315

        Show
        Alexander Vladishev added a comment - Related issue: ZBX-7315
        Hide
        Alexander Vladishev added a comment - - edited

        (2) moved filtering of lld rows to the one place

        • svn://svn.zabbix.com/branches/dev/ZBX-7109
        • svn://svn.zabbix.com/branches/dev/ZBX-7109_trunk

        Andris Zeila svn://svn.zabbix.com/branches/dev/ZBX-7109 r41040 tested
        svn://svn.zabbix.com/branches/dev/ZBX-7109_trunk r41045 tested

        CLOSED

        Available in pre-2.0.11 r41051, pre-2.2.2 r41053 and pre-2.3.0 (trunk) r41054.

        Show
        Alexander Vladishev added a comment - - edited (2) moved filtering of lld rows to the one place svn://svn.zabbix.com/branches/dev/ZBX-7109 svn://svn.zabbix.com/branches/dev/ZBX-7109_trunk Andris Zeila svn://svn.zabbix.com/branches/dev/ZBX-7109 r41040 tested svn://svn.zabbix.com/branches/dev/ZBX-7109_trunk r41045 tested CLOSED Available in pre-2.0.11 r41051, pre-2.2.2 r41053 and pre-2.3.0 (trunk) r41054.
        Hide
        richlv added a comment - - edited

        (3) documented perf improvement in :
        https://www.zabbix.com/documentation/2.0/manual/introduction/whatsnew2011
        https://www.zabbix.com/documentation/2.2/manual/introduction/whatsnew222

        RESOLVED, please review

        Alexander Vladishev added more details, please review.

        <richlv> clarified a bit more, please, review

        Alexander Vladishev Thank you! CLOSED

        Show
        richlv added a comment - - edited (3) documented perf improvement in : https://www.zabbix.com/documentation/2.0/manual/introduction/whatsnew2011 https://www.zabbix.com/documentation/2.2/manual/introduction/whatsnew222 RESOLVED, please review Alexander Vladishev added more details, please review. <richlv> clarified a bit more, please, review Alexander Vladishev Thank you! CLOSED
        Hide
        richlv added a comment -

        this also fixed ZBX-6315

        Show
        richlv added a comment - this also fixed ZBX-6315
        Hide
        Alexander Vladishev added a comment - - edited

        Whats new:

        1. Better processing of trigger prototypes.
          • Already discovered triggers will be updated only if something changed.
          • Only changed fields will be updated.
          • Decreased number and size of sql queries.
          • Lost triggers will be deleted only after selected period ZBX-6315.
          • Better validation of fields after resolving of LLD macros.
            • validity of UTF-8 sequences
            • validity of fields lengths

        Tests:

        One discovery rule with 2 item prototypes and 2 trigger prototypes. Received 2048 new records.

        2.0.11

        Action Time, sec. Number of SQL requests Total size of SQL statements, bytes
        first discovery (creating of 2 * 2048 = 4096 triggers) 58.192054 20505 4504138
        next discovery (updating) 40.862417 16392 4511767
        next discovery (changed expression in trigger prototypes) 6131.527467 4225040 736623181

        2.0.11 with the fix

        Action Time, sec. Number of SQL requests Total size of SQL statements, bytes
        first discovery (creating of 2 * 2048 = 4096 triggers) 1.065422 34 1071045
        next discovery (updating) 0.129869 9 1265
        next discovery (changed expression in trigger prototypes) 1.128554 17 515629
        Show
        Alexander Vladishev added a comment - - edited Whats new: Better processing of trigger prototypes. Already discovered triggers will be updated only if something changed. Only changed fields will be updated. Decreased number and size of sql queries. Lost triggers will be deleted only after selected period ZBX-6315 . Better validation of fields after resolving of LLD macros. validity of UTF-8 sequences validity of fields lengths Tests: One discovery rule with 2 item prototypes and 2 trigger prototypes. Received 2048 new records. 2.0.11 Action Time, sec. Number of SQL requests Total size of SQL statements, bytes first discovery (creating of 2 * 2048 = 4096 triggers) 58.192054 20505 4504138 next discovery (updating) 40.862417 16392 4511767 next discovery (changed expression in trigger prototypes) 6131.527467 4225040 736623181 2.0.11 with the fix Action Time, sec. Number of SQL requests Total size of SQL statements, bytes first discovery (creating of 2 * 2048 = 4096 triggers) 1.065422 34 1071045 next discovery (updating) 0.129869 9 1265 next discovery (changed expression in trigger prototypes) 1.128554 17 515629
        Hide
        Alexander Vladishev added a comment -

        Fixed in the development branch svn://svn.zabbix.com/branches/dev/ZBX-7109

        Show
        Alexander Vladishev added a comment - Fixed in the development branch svn://svn.zabbix.com/branches/dev/ZBX-7109
        Hide
        Aleksandrs Saveljevs added a comment - - edited

        (4) Parameter "parent_triggerid" in function DBlld_trigger_make() is unused. Also, this function has "DB" prefix, but it does not do anything with the database.

        Alexander Vladishev RESOLVED in r41871

        Aleksandrs Saveljevs CLOSED

        Show
        Aleksandrs Saveljevs added a comment - - edited (4) Parameter "parent_triggerid" in function DBlld_trigger_make() is unused. Also, this function has "DB" prefix, but it does not do anything with the database. Alexander Vladishev RESOLVED in r41871 Aleksandrs Saveljevs CLOSED
        Hide
        Aleksandrs Saveljevs added a comment - - edited

        (5) Please take a look at minor fixes during code review in r41875.

        Alexander Vladishev Thanks! CLOSED

        Show
        Aleksandrs Saveljevs added a comment - - edited (5) Please take a look at minor fixes during code review in r41875. Alexander Vladishev Thanks! CLOSED
        Hide
        Alexander Vladishev added a comment - - edited

        (6) [I] trunk: The fields trigger_discovery.name and trigger_discovery.triggerdiscoveryid should be removed.

        Alexander Vladishev RESOLVED in r41936

        Alexander Vladishev Already tested by Aleksandrs Saveljevs CLOSED

        Show
        Alexander Vladishev added a comment - - edited (6) [I] trunk: The fields trigger_discovery. name and trigger_discovery. triggerdiscoveryid should be removed. Alexander Vladishev RESOLVED in r41936 Alexander Vladishev Already tested by Aleksandrs Saveljevs CLOSED
        Hide
        Alexander Vladishev added a comment -

        Fixed in pre-2.0.11 r41917.

        Show
        Alexander Vladishev added a comment - Fixed in pre-2.0.11 r41917.
        Hide
        Alexander Vladishev added a comment - - edited

        2.2 and trunk are fixed in the development branches:

        • svn://svn.zabbix.com/branches/dev/ZBX-7109-22
        • svn://svn.zabbix.com/branches/dev/ZBX-7109-23 (trunk)

        Aleksandrs Saveljevs In 2.2 branch, I have fixed a bunch of typos in r41984. Please take a look.
        Alexander Vladishev CLOSED.

        Aleksandrs Saveljevs In trunk branch, database index after upgrade from 2.2 has a different name compared to that from a fresh database. REOPENED.
        Alexander Vladishev Fixed in r42019. RESOLVED.
        Aleksandrs Saveljevs CLOSED.

        Show
        Alexander Vladishev added a comment - - edited 2.2 and trunk are fixed in the development branches: svn://svn.zabbix.com/branches/dev/ZBX-7109-22 svn://svn.zabbix.com/branches/dev/ZBX-7109-23 (trunk) Aleksandrs Saveljevs In 2.2 branch, I have fixed a bunch of typos in r41984. Please take a look. Alexander Vladishev CLOSED. Aleksandrs Saveljevs In trunk branch, database index after upgrade from 2.2 has a different name compared to that from a fresh database. REOPENED. Alexander Vladishev Fixed in r42019. RESOLVED. Aleksandrs Saveljevs CLOSED.
        Hide
        yuusou added a comment -

        Will this patch re-create triggers and graphs that were deleted in previous versions?

        Show
        yuusou added a comment - Will this patch re-create triggers and graphs that were deleted in previous versions?
        Hide
        Alexander Vladishev added a comment -

        yuusou,

        We can't create such patch.

        Show
        Alexander Vladishev added a comment - yuusou, We can't create such patch.
        Hide
        yuusou added a comment -

        Alexander,

        If that's the case, then this rewrite doesn't fix ZBX-6315, is that correct? It's marked as a duplicate of this bug.

        Show
        yuusou added a comment - Alexander, If that's the case, then this rewrite doesn't fix ZBX-6315 , is that correct? It's marked as a duplicate of this bug.
        Hide
        richlv added a comment -

        it does fix that - triggers won't get deleted in such case anymore.
        triggers would get recreated based on prototypes next time the entity is discovered again.

        Show
        richlv added a comment - it does fix that - triggers won't get deleted in such case anymore. triggers would get recreated based on prototypes next time the entity is discovered again.
        Hide
        Alexander Vladishev added a comment - - edited

        Fixed in pre-2.2.2 r41993 and pre-2.3.0 r42023.

        Show
        Alexander Vladishev added a comment - - edited Fixed in pre-2.2.2 r41993 and pre-2.3.0 r42023.
        Hide
        richlv added a comment -

        subissues (3) and (6) not closed, reopening

        Show
        richlv added a comment - subissues (3) and (6) not closed, reopening
        Hide
        richlv added a comment - - edited

        (7) i also moved trunk entry "[ZBX-7109] removed unused fields in trigger_discovery table" from new features to bugfixes, please check that it's ok

        Alexander Vladishev Thanks, it will be better! CLOSED

        Show
        richlv added a comment - - edited (7) i also moved trunk entry "[ ZBX-7109 ] removed unused fields in trigger_discovery table" from new features to bugfixes, please check that it's ok Alexander Vladishev Thanks, it will be better! CLOSED

          People

          • Assignee:
            Unassigned
            Reporter:
            Alexey Pustovalov
          • Votes:
            0 Vote for this issue
            Watchers:
            11 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: