Uploaded image for project: 'ZABBIX BUGS AND ISSUES'
  1. ZABBIX BUGS AND ISSUES
  2. ZBX-21011

server log is spammed by flapping "error reason ... changed" messages for zabbix.nodes.* items

    XMLWordPrintable

Details

    • Team INT
    • Sprint 88 (May 2022)
    • 0.125

    Description

      Imagine that you have a standalone zabbix server working. Existing "Zabbix server" host will discover 4 items (rule: High availability cluster node discovery), where node name is empty.

      Then you uncomment and set HANodeName= parameter in server conf file, restart.
      After that an existing standalone entry in "ha_node" DB table will be replaced by a new one, generated for the node.
      So now zabbix installation will get 4 additional discovered items, while previously existing ones will be marked for deletion. This is fine.

      But! Those 4 previously existing items will start to spam zabbix_server.log by messages like:

      356996:20220509:134625.878 error reason for "Zabbix server:zabbix.nodes.lastaccess.time[cl1mhfat80001w98k8vvkvxb9]" changed: Preprocessing failed for: [{"id":"cl2yl9hny00011r8kta7i7063","name":"node1","status":3,"lastaccess":1652093184,"address":"l...
      1. Failed: cannot extract value from json by path "$.[?(@.id=="cl1mhfat80001w98k8vvkvxb9")].lastaccess.first()": no data matches the specified path
      1356996:20220509:134625.878 error reason for "Zabbix server:zabbix.nodes.status[cl1mhfat80001w98k8vvkvxb9]" changed: Preprocessing failed for: [{"id":"cl2yl9hny00011r8kta7i7063","name":"node1","status":3,"lastaccess":1652093184,"address":"l...
      1. Failed: cannot extract value from json by path "$.[?(@.id=="cl1mhfat80001w98k8vvkvxb9")].status.first()": no data matches the specified path
      1356996:20220509:134625.878 error reason for "Zabbix server:zabbix.nodes.lastaccess.age[cl1mhfat80001w98k8vvkvxb9]" changed: Preprocessing failed for: [{"id":"cl2yl9hny00011r8kta7i7063","name":"node1","status":3,"lastaccess":1652093184,"address":"l...
      1. Failed: cannot extract value from json by path "$.[?(@.id=="cl1mhfat80001w98k8vvkvxb9")].lastaccess_age.first()": no data matches the specified path
      1356996:20220509:134625.878 error reason for "Zabbix server:zabbix.nodes.address[cl1mhfat80001w98k8vvkvxb9]" changed: Preprocessing failed for: [{"id":"cl2yl9hny00011r8kta7i7063","name":"node1","status":3,"lastaccess":1652093184,"address":"l...
      1. Failed: cannot extract value from json by path "$.[?(@.id=="cl1mhfat80001w98k8vvkvxb9")].address.first()": no data matches the specified path
      

      Reason for that is returned JSON on master internal item is this:

      [{"id":"cl2yl9hny00011r8kta7i7063","name":"node1","status":3,"lastaccess":1652092584,"address":"localhost:10051","db_timestamp":1652092585,"lastaccess_age":1}]
      

      i.e. on beginning, which is included to the not supported error message, it has a dynamic part - clock for "lastaccess". This causes that each time the item is still evaluated, it gets a new not supported error message, which has to be updated in DB.

      So, such messages in server log may be very unclear where they come from and why.

      Suggestion how to fix in template. For JSONPath preprocessing steps (4 prototypes) like:

      $.[?(@.id=="{#NODE.ID}")].address.first()
      

      enable option "Custom on fail" with "Set value to" and value like - "Node is not discoverable anymore".

      This way error message will be fixed and there should be no flapping in the log.

      Attachments

        Issue Links

          Activity

            People

              zabbix.dev Zabbix Development Team
              zalex_ua Oleksii Zagorskyi
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated: