Uploaded image for project: 'ZABBIX BUGS AND ISSUES'
  1. ZABBIX BUGS AND ISSUES
  2. ZBX-14640

Some items are not refreshed in given frequency for hours

XMLWordPrintable

    • Icon: Incident report Incident report
    • Resolution: Won't fix
    • Icon: Critical Critical
    • None
    • 3.4.10, 3.4.11
    • Server (S)
    • None

      Hello,
        **  About 2 moths ago we upgraded Zabbix 3.0.4 to Zabbix 3.4.9 (and then to 3.4.11). All looks fine but recently we found that some items are not refreshed as they should be. But Zabbix do not report any problems with gathering processes nor with cache nor with Items update Queue.

      Problem is serious for us because some items that should be refreshed every 3-5 minutes are not refreshed for example 12-24 hours.

      I created SQL procedure that will report all items that should be refreshed at least every 60 minutes but it is not refreshed more than 90 minutes. Every time it returns 40-100 items that are  not refreshed. I executed it few times and put all result into attached Excel file "ERROR_not_refreshed_items.xlsx". Each list contains date and time when it was executed and list of all not refreshed items.

      Symptoms:

      1. Items on some hosts are not refreshed. It not happens on all hosts - but list of affected hosts for every check looks very similar.
      2. Items are different in every check but I can see that some items are exists in almost every check
      3. Items type is not important it happens for discovered items, items from template and even for items manually created on the server.
        Also if happens for simple Zabbix checks like CPU load and for custom scripts too.
      4. Not refreshed items are all enabled and the are not in Unsupported state nor in Error state.
      5. It happens on different host types and different Zabbix Agents - Solaris, Linux....
      6. Looks that problem is occurring mainly on two servers that have very big number of items:
        czken3hr.vfcz.dc-ratingen.de - 1070 items
        aczfil10s-z1.vfcz.dc-ratingen.de - 650 items
        l5ucms35 - 571 items
      7. Those delayed (or not refreshed items) are stil refreshed at least once per day (maybe twice).
      8. In Zabbix - Administration - Queue is displayed that only 1-10 items are delayed but not more that 10s.  Simply all those very long delayed items discovered by mine SQL script are ignored by this queue
      9. Zabbix Value Cache is about 50% free
      10. Zabbix pooler processes are busy maximally for 43%, average for 31%
      11. Some statistics: We are monitoring 111 hosts, 20656 items and have 42 values per second.
      12. This problem did not exists in Zabbix 3.0.4 and previous versions
      13. We are using Zabbix 7 years - form version 1.8.8 till current 3.4.11

      Expected:

      1. Please see screenshot from Zabbix GUI that shows that cpu load item is not refreshed (at 11:45 AM): 2018-06-26 11_45_56-Delayed_CPU_Items.png
      2. Second screenshot shows configuration of this item: 2018-06-26 11_46_35-Configuration of items.png
      3. Third screenshot shows graph of items values - you can see that value is refreshed daily instead of every 5 minutes: 2018-06-26 11_48_32-History_CPU_Item.png

       

      Possible root cause:

      • It looks like problem is in calculation of what items should be refreshed.  Maybe some problem in Value cache?
      • Looks like problem occurs if hosts have defined big number of items only (higher that 512 items.

       

      What we trying to solve this problem:

      • We restarted Zabbix server few times.   Sometimes items were refreshed 30 minutes after restart, but after that problem occurred again.  Sometimes problem occurred immediately after restart.
      • We tried to increase number of Process poolers on Zabbix Server - without any success.
      • We tried to increase and decrease number of Process poolers on affected Zabbix Agents - without any success.
      • We tried to go back to old Agents on affected hosts - without any success.

        1. 2018-06-26 11_45_56-Delayed_CPU_Items.png
          19 kB
          Milan Gvozdik
        2. 2018-06-26 11_46_35-Configuration of items.png
          31 kB
          Milan Gvozdik
        3. 2018-06-26 11_48_32-History_CPU_Item.png
          49 kB
          Milan Gvozdik
        4. ERROR_not_refreshed_items.xlsx
          107 kB
          Milan Gvozdik

            Unassigned Unassigned
            nalim27 Milan Gvozdik
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: