ExportXMLWordPrintable

    • 0.5

      Specific template in question:

      https://git.zabbix.com/projects/ZBX/repos/zabbix/browse/templates/app/veeam/backup_replication_http?at=release/7.4

      The "Sessions discovery" LLD rule keys its item prototypes on {#ID}, which comes from the Veeam session's own id field ($.id) which is a value unique to a single execution, not to the job/task itself. Then there's the default filter:

      {$SESSION.RESULT.NOT_MATCHES} = Success

      which means the rule only ever discovers sessions that are not successful.

       

      This means there are two reasons why a discovered and failed session would never clear: first there's the fact that the session ID is a GUID which won't match the next run, and even once that's fixed the default filter means Zabbix isn't ever going to see the subsequent successful run.

      Steps to reproduce:

      1. Link the template to a VBR host with a job or task that will fail at least once or trigger a failure somehow. An easy way is to temporarily disable NFS on a configured backup repo and then do a rescan.
      2. Wait for the failed session to get discovered and alert.
      3. Fix the issue and re-run the task that failed.
      4. The original trigger remains in PROBLEM state. No new item appears reflecting the successful re-run, because {$SESSION.RESULT.NOT_MATCHES}=Success excludes it from discovery.
      5. Keep in mind that even if the {$SESSION.RESULT.NOT_MATCHES}=Success filter is removed, the session failure will still show as a problem because the GUID of the subsequent successful job will be different from the GUID of the failed job, and (improperly) that GUID was used as the key to this alert.

      Expected:
      Once a particular type of task fails but is re-run successfully (presumably because an operator fixed the issue), the trigger should self-resolve.

      Actual:
      A failed job stays as a problem even when it's been fixed and run successfully.

      Suggestions:

      1. Do not key off the unique ID for each job, as it will always be different on subsequent runs. Find something else - some combination of job name, params, and failure message should be sufficient for uniqueness.
      2. Get rid of the "only on failure" filter as its existence guarantees that the alert can never self-resolve. 

            Assignee:
            Evgenii Gordymov
            Reporter:
            Mike Ely
            Team INT
            Votes:
            1 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated:

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - Not Specified
                Not Specified
                Logged:
                Time Spent - 14.5h
                14.5h