Uploaded image for project: 'ZABBIX BUGS AND ISSUES'
  1. ZABBIX BUGS AND ISSUES
  2. ZBX-20567

Preprocessing queue grows indefinitely on Zabbix Proxy

XMLWordPrintable

    • Sprint 85 (Feb 2022), Sprint 86 (Mar 2022), Sprint 87 (Apr 2022)
    • 2

      This issue occurred before update from 5.4.9 to 5.4.10.

      $ ps aux --sort -pmem | head -n 20 USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND zabbix 1599316 1.4 32.2 13476284 7891020 ? R 06:15 5:13 /usr/sbin/zabbix_proxy: preprocessing manager #1 [queued 21584654, processed 577 values, idle 0.000000 sec during 6.788713 sec]

      When diaginfo running sometimes this appear:

      1599249:20220210:113030.854 cannot obtain diagnostic information: Timeout while waiting for response 1599316:20220210:113056.418 cannot write to IPC socket: Broken pipe

      Diaginfo

      1599249:20220210:113715.414 == history cache diagnostic information ==
1599249:20220210:113715.426 items:10 values:10 time:13.283719
1599249:20220210:113715.439 memory.data:
1599249:20220210:113715.451   size: free:2147482712 used:400
1599249:20220210:113715.463   chunks: free:1 used:10 min:2147482712 max:2147482712
1599249:20220210:113715.475     buckets:
1599249:20220210:113715.486       256+:1
1599249:20220210:113715.498 memory.index:
1599249:20220210:113715.509   size: free:1072513584 used:1227608
1599249:20220210:113715.522   chunks: free:2 used:14 min:2825200 max:1069688384
1599249:20220210:113715.533     buckets:
1599249:20220210:113715.545       256+:2
1599249:20220210:113715.557 top.values:
1599249:20220210:113715.581   itemid:5557378 values:1
1599249:20220210:113715.595   itemid:1237280 values:1
1599249:20220210:113715.611   itemid:5367883 values:1
1599249:20220210:113715.624   itemid:5371741 values:1
1599249:20220210:113715.638   itemid:5649288 values:1
1599249:20220210:113715.651   itemid:5369595 values:1
1599249:20220210:113715.664   itemid:5558434 values:1
1599249:20220210:113715.682   itemid:5559854 values:1
1599249:20220210:113715.694   itemid:1241362 values:1
1599249:20220210:113715.712   itemid:1237150 values:1
1599249:20220210:113715.726 ==
1599249:20220210:113715.748 == preprocessing diagnostic information ==
1599249:20220210:113715.761 values:18731150 done:7491830 queued:1100825 processing:200 pending:10138295 time:109.178839
1599249:20220210:113715.774 top.values:
1599249:20220210:113715.786   itemid:2556516 values:468 steps:0
1599249:20220210:113715.913   itemid:1769589 values:235 steps:0
1599249:20220210:113716.022   itemid:298449 values:235 steps:0
1599249:20220210:113716.055   itemid:246236 values:235 steps:0
1599249:20220210:113716.067   itemid:4966626 values:235 steps:0
1599249:20220210:113716.251   itemid:4417086 values:235 steps:0
1599249:20220210:113716.339   itemid:246253 values:235 steps:0
1599249:20220210:113716.748   itemid:246663 values:235 steps:0
1599249:20220210:113716.846   itemid:1660930 values:235 steps:0
1599249:20220210:113716.886   itemid:2825099 values:235 steps:0
1599249:20220210:113716.910   itemid:340443 values:235 steps:0
1599249:20220210:113716.931   itemid:5068270 values:235 steps:0
1599249:20220210:113716.970   itemid:295390 values:235 steps:0
1599249:20220210:113718.499   itemid:286688 values:235 steps:0
1599249:20220210:113718.532   itemid:2856565 values:235 steps:0
1599249:20220210:113718.555   itemid:2856564 values:235 steps:0
1599249:20220210:113718.585   itemid:2856566 values:235 steps:0
1599249:20220210:113718.603   itemid:2853435 values:235 steps:0
1599249:20220210:113718.627   itemid:2853436 values:235 steps:0
1599249:20220210:113718.649   itemid:2853437 values:235 steps:0
1599249:20220210:113718.676   itemid:2847569 values:235 steps:0
1599249:20220210:113718.703   itemid:2847567 values:235 steps:0
1599249:20220210:113718.731   itemid:2847568 values:235 steps:0
1599249:20220210:113718.786   itemid:2856326 values:235 steps:0
1599249:20220210:113718.809   itemid:2856324 values:235 steps:0
1599249:20220210:113719.453 top.oldest.preproc.values:
1599249:20220210:113719.487 ==
1599249:20220210:113719.537 == locks diagnostic information ==
1599249:20220210:113719.562 locks:
1599249:20220210:113719.642   ZBX_MUTEX_LOG:0x7f4a7da22000
1599249:20220210:113719.716   ZBX_MUTEX_CACHE:0x7f4a7da22028
1599249:20220210:113719.770   ZBX_MUTEX_TRENDS:0x7f4a7da22050
1599249:20220210:113719.804   ZBX_MUTEX_CACHE_IDS:0x7f4a7da22078
1599249:20220210:113719.818   ZBX_MUTEX_SELFMON:0x7f4a7da220a0
1599249:20220210:113719.830   ZBX_MUTEX_CPUSTATS:0x7f4a7da220c8
1599249:20220210:113719.869   ZBX_MUTEX_DISKSTATS:0x7f4a7da220f0
1599249:20220210:113721.055   ZBX_MUTEX_ITSERVICES:0x7f4a7da22118
1599249:20220210:113721.141   ZBX_MUTEX_VALUECACHE:0x7f4a7da22140
1599249:20220210:113721.230   ZBX_MUTEX_VMWARE:0x7f4a7da22168
1599249:20220210:113721.244   ZBX_MUTEX_SQLITE3:0x7f4a7da22190
1599249:20220210:113721.272   ZBX_MUTEX_PROCSTAT:0x7f4a7da221b8
1599249:20220210:113721.447   ZBX_MUTEX_PROXY_HISTORY:0x7f4a7da221e0
1599249:20220210:113721.466   ZBX_MUTEX_MODBUS:0x7f4a7da22208
1599249:20220210:113721.496   ZBX_MUTEX_TREND_FUNC:0x7f4a7da22230
1599249:20220210:113721.522   ZBX_RWLOCK_CONFIG:0x7f4a7da22258
1599249:20220210:113721.541   ZBX_RWLOCK_VALUECACHE:0x7f4a7da22290
1599249:20220210:113721.567 ==

      Proxy cache usage

        1. PROXY-CACHES.PNG
          60 kB
          Aleksey Volodin
        2. screenshot-1.png
          250 kB
          Andris Zeila

            wiper Andris Zeila
            avolodin Aleksey Volodin
            Team A
            Votes:
            2 Vote for this issue
            Watchers:
            8 Start watching this issue

              Created:
              Updated:
              Resolved: