Uploaded image for project: 'ZABBIX BUGS AND ISSUES'
  1. ZABBIX BUGS AND ISSUES
  2. ZBX-21032

Preprocessing manager works incorrectly

XMLWordPrintable

    • Icon: Problem report Problem report
    • Resolution: Duplicate
    • Icon: Trivial Trivial
    • None
    • 6.0.4
    • Server (S)
    • None
    • Oracle Linux 8, PostgreSQL 13.6, Timescale 2.3.1

      This issue occurred after update from 5.0.17 to 6.0.4

      Diaginfo

       

      == history cache diagnostic information ==
      Items:0 values:0 time:0.000023
      Memory.data:
        size: free:1073741440 used:0
        chunks: free:1 used:0 min:1073741440 max:1073741440
          buckets:
            256+:1
      Memory.index:
        size: free:1073369232 used:372080
        chunks: free:3 used:5 min:8072 max:1072532296
          buckets:
            256+:3
      Top.values:
      ==
      == preprocessing diagnostic information ==
      Values:403502 done:361959 queued:0 processing:1314 pending:40229 time:2.981349
      Top.values:
        itemid:1296943 values:108 steps:0
        itemid:3276248 values:99 steps:0
        itemid:3382122 values:99 steps:0
        itemid:3380145 values:99 steps:0
        itemid:1402337 values:99 steps:0
        itemid:3276229 values:99 steps:0
        itemid:1296945 values:99 steps:0
        itemid:1056895 values:99 steps:0
        itemid:1402346 values:99 steps:0
        itemid:2014796 values:99 steps:0
        itemid:3276268 values:99 steps:0
        itemid:2078554 values:98 steps:0
        itemid:5488727 values:50 steps:2
        itemid:5488736 values:50 steps:1
        itemid:5488737 values:50 steps:3
        itemid:5488739 values:50 steps:3
        itemid:5488740 values:50 steps:2
        itemid:5488747 values:50 steps:1
        itemid:5488748 values:50 steps:1
        itemid:5488753 values:50 steps:1
        itemid:5488757 values:50 steps:2
        itemid:5488761 values:50 steps:1
        itemid:1410359 values:50 steps:1
        itemid:1410374 values:50 steps:2
        itemid:1410377 values:50 steps:2
      Top.oldest.preproc.values:
      ==
      == locks diagnostic information ==
      Locks:
        ZBX_MUTEX_LOG:0x7f5d60c33000
        ZBX_MUTEX_CACHE:0x7f5d60c33028
        ZBX_MUTEX_TRENDS:0x7f5d60c33050
        ZBX_MUTEX_CACHE_IDS:0x7f5d60c33078
        ZBX_MUTEX_SELFMON:0x7f5d60c330a0
        ZBX_MUTEX_CPUSTATS:0x7f5d60c330c8
        ZBX_MUTEX_DISKSTATS:0x7f5d60c330f0
        ZBX_MUTEX_VALUECACHE:0x7f5d60c33118
        ZBX_MUTEX_VMWARE:0x7f5d60c33140
        ZBX_MUTEX_SQLITE3:0x7f5d60c33168
        ZBX_MUTEX_PROCSTAT:0x7f5d60c33190
        ZBX_MUTEX_PROXY_HISTORY:0x7f5d60c331b8
        ZBX_MUTEX_MODBUS:0x7f5d60c331e0
        ZBX_MUTEX_TREND_FUNC:0x7f5d60c33208
        ZBX_RWLOCK_CONFIG:0x7f5d60c33230
        ZBX_RWLOCK_VALUECACHE:0x7f5d60c33268
      ==
      == value cache diagnostic information ==
      Items:0 values:0 mode:-1 time:0.000000
      Memory:
        size: free:0 used:0
        chunks: free:0 used:0 min:0 max:0
          buckets:
      Top.values:
      Top.request.values:
      ==
      == LLD diagnostic information ==
      Rules:0 values:0 time:0.000134
      Top.values:
      ==
      == alerting diagnostic information ==
      Alerts:0 time:0.000140
      Media.alerts:
      Source.alerts:
      ==
      

       

      Steps to reproduce:

      1. Update Zabbix from 5.0 to 6.0 (just server, proxy hasn't problem)
      2. Watch for the preprocessing manager

      Result:
      Preprocessing manager queue is growing up, after that manager starts using CPU on 100% (and 1 worker uses CPU on 100% of 50 workers, see screenshots) and all zabbix checks are down (e.g. Zabbix agent is DOWN, manual check says UP, but Zabbix marks host is unreachable). When queue is full - OOM kills zabbix-server.

        1. 2022-05-12_12-15-32.jpg
          129 kB
          Igor Sidorenko
        2. 2022-05-12_12-25-54.jpg
          64 kB
          Igor Sidorenko
        3. 2022-05-12_12-26-35.jpg
          333 kB
          Igor Sidorenko
        4. 2022-05-12_12-41-44.jpg
          235 kB
          Igor Sidorenko
        5. 2022-05-12_12-42-31.jpg
          225 kB
          Igor Sidorenko
        6. 2022-06-15 15_07_30-Window.png
          21 kB
          vincent meyer
        7. 2022-06-15 15_08_26-Window.png
          10 kB
          vincent meyer
        8. 2022-06-15 15_09_57-zabbix-val_ Dashboard — Mozilla Firefox.png
          15 kB
          vincent meyer
        9. 2022-06-15 15_10_47-root@dataval652-19_ _home_exploit.png
          33 kB
          vincent meyer
        10. preprocessin.png
          46 kB
          vincent meyer

            elina.kuzyutkina Elina Kuzyutkina (Inactive)
            ownhrd Igor Sidorenko
            Votes:
            5 Vote for this issue
            Watchers:
            7 Start watching this issue

              Created:
              Updated:
              Resolved: