Uploaded image for project: 'ZABBIX BUGS AND ISSUES'
  1. ZABBIX BUGS AND ISSUES
  2. ZBX-18418

active nodata() triggers assigned to hosts monitored via proxy resolve after Zabbix Server restart

XMLWordPrintable

    • Icon: Problem report Problem report
    • Resolution: Fixed
    • Icon: Major Major
    • 5.0.2, 5.0.3
    • Proxy (P), Server (S)
    • None
    • Zabbix 5.0.2 and 5.0.3 (tested with both)
      Server, Proxy, Nginx, Frontend all inside Docker
      Passive Agent on Macos.
    • Sprint 71 (Dec 2020)
    • 1

      In a somewhat larger Zabbix environment we encountered the problem that when updating from 4.x to 5.0.2, a huge amount of nodata() based triggers spawned new problems just after the successful upgrade of the environment. Now after a minor upgrade from 5.0.2 to 5.0.3 the same happened again, causing several hundrets of false positive alarms.

      A deeper investigation in an isolated environment made it possible to reproduce this error behavior: Triggers on hosts monitored via a Zabbix proxy (only tested with ACTIVE proxy) based on the nodata() trigger function and being active (in problem state) will RESOLVE after stopping and starting the Zabbix Server. Apparently, this only happens if between stop and start of the Zabbix Server there is above around 1 minute of time gap. Those resolved triggers will then re-generate new problems after their nodata(xx) time is over, and this way generate new problems, new alerts etc.

      Steps to reproduce:

      Preparation:

      1. spawned Zabbix server, an active proxy, one (passive only) agent up.
      2. to the existing template "Template Module Zabbix agent", added a trigger "Agent nodata 5m" - "{Template Module Zabbix agent:agent.ping.nodata(5m)}=1" - see attachment "exported_template.xml"
      3. added the active Zabbix proxy to the configuration and make sure it fetches configuration
      4. added two hosts: macbook-direct and macbook-via-proxy, both using the template "Template Module Zabbix agent" - see attachment "exported_hosts.xml"
      5. wait until both hosts get "available"

      Result:

      Based on the above preparations, note the following procedure:

      15:00 - both hosts "ZBX available", no problems present
      15:10 - stopped Zabbix agent
      15:15 - nodata (and originally existing zabbix-available) triggers spawn problems - see attachment "Screenshot01 - problems active.png"
      15:38 - restart Zabbix server (downtime a bit over a minute)
      15:39 - nodata-Trigger on the host being monitored via proxy recovers (while the one on the host NOT monitored via proxy stays active)
      15:44 - nodata-Trigger on the host monitored via proxy re-triggers -> new problem (Latest data shows clearly, that there have been no values on the item being
      responsible for the nodata trigger on the host monitored via proxy since 15:10) - see attachment "Screenshot02 - resolved and new problem.png"

      Expected:
      Problem on host monitored via proxy should not recover and also not re-spawn. It should just stay active as it was before the Zabbix server restart. See attachment "Screenshot03 - Latest Data.png" to prove there has not been any values on the Item in question during the whole test time frame.

        1. exported_hosts.xml
          2 kB
          Christian Anton
        2. exported_template.xml
          6 kB
          Christian Anton
        3. Screenshot01 - problems active.png
          439 kB
          Christian Anton
        4. Screenshot02 - resolved and new problem.png
          410 kB
          Christian Anton
        5. Screenshot03 - Latest Data.png
          292 kB
          Christian Anton
        6. ZBX-18418-50.patch
          1 kB
          Michael Veksler

            MVekslers Michael Veksler
            christiananton Christian Anton
            Team C
            Votes:
            8 Vote for this issue
            Watchers:
            15 Start watching this issue

              Created:
              Updated:
              Resolved: