Uploaded image for project: 'ZABBIX BUGS AND ISSUES'
  1. ZABBIX BUGS AND ISSUES
  2. ZBX-19363

Data is not updating to Devices and all option hangs, Zabbix server services required to restart manually every 3 hours.

    XMLWordPrintable

Details

    Description

      Dear Team, 

      Seeking your help with the below issues. 

      We installed the Production Zabbix system and integrated around "1351" Nodes into them and we have to integrated 800 more devices into the production system. So In total, we will have above 2K integrated nodes.

       

      Please let me know if any more information is required. Thank you.

       

      System Information:

       

      CPU : 8 CPU

      RAM: 32 GB

      DISK: 3 TB

      OS: Red Hat Enterprise Linux release 8.3 (Ootpa)

      Zabbix Version: 5.0

       

      Issue:  Data is not updating to Devices and all option hangs, Zabbix server services required to restart manually every 3 hours.  

       

      Troubleshooting:

      • As a part of troubleshooting, I have increased the "Start poller" to 375, and the connection to Mysql increase to =1000.
      • Poller probing in progress but no data updating.
      • Mysql optimizer_switch='index_condition_pushdown=off'; is switch off

       

      Below cache configuration done in the Zabbix_server.conf

       

      VMwareCacheSize=2048M

      HousekeepingFrequency=1

      CacheSize=8144M

      CacheUpdateFrequency=60

      HistoryCacheSize=1024M

      HistoryIndexCacheSize=1024M

      TrendCacheSize=512M

      ValueCacheSize=1024M

      Timeout=10

       

      Mysql Configuration :

       

      MariaDB [(none)]> Show variables like 'max_connections';                                                      ----------------------+

      Variable_name   Value

      ----------------------+

      max_connections 1000 

      ----------------------+

      1 row in set (0.224 sec)

       

      MariaDB [(none)]>

       

      MariaDB [(none)]> SELECT @@optimizer_switch\G                                                                 *************************** 1. row ***************************

      @@optimizer_switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on,split_materialized=on

      1 row in set (0.000 sec)

       

      MariaDB [(none)]>

       

      -----------------------------------+

      database           size in GB     

      -----------------------------------+

      information_schema   0.000167846680
      mysql                0.000945094042
      performance_schema   0.000000000000
      zabbix             67.724426269531

      -----------------------------------+

      4 rows in set (0.050 sec)

       

      MariaDB [zabbix]>

       

       

      [[email protected] ~]# iostat 5 10

      Linux 4.18.0-240.el8.x86_64 (zabbix.mobily.com.sa)      05/12/2021      x86_64        (8 CPU)

       

      avg-cpu:  %user   %nice %system %iowait  %steal   %idle

                 5.56    0.00    3.56   27.36    0.00   63.52

       

      Device             tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn

      sda             932.92     16913.87     20996.90 3072589362 3814314887

      scd0              0.00         0.14         0.00      25612          0

      dm-0            935.99     16859.41     20764.38 3062694894 3772075564

      dm-1             71.31        53.87       231.37    9785804   42030572

      dm-2              0.03         0.38         1.14      68635     206575

       

      avg-cpu:  %user   %nice %system %iowait  %steal   %idle

                 0.33    0.00    0.07    1.12    0.00   98.47

       

      Device             tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn

      sda              17.40       124.80        33.40        624        167

      scd0              0.00         0.00         0.00          0          0

      dm-0              3.60         4.80        33.40         24        167

      dm-1             30.00       120.00         0.00        600          0

      dm-2              0.00         0.00         0.00          0          0

       

      avg-cpu:  %user   %nice %system %iowait  %steal   %idle

                 6.10    0.00    0.68    2.23    0.00   91.00

       

      Device             tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn

      sda              32.40       428.00        59.40       2140        297

      scd0              0.00         0.00         0.00          0          0

      dm-0              5.00         6.40        59.40         32        297

      dm-1            105.40       421.60         0.00       2108          0

      dm-2              0.00         0.00         0.00          0          0

       

      [[email protected] ~]# vmstat 3 5

      procs ----------memory--------- --swap- ----io--- system- -----cpu----

      r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st

      0  0 3209508 10206468    148 15904656    7   29  2121  2633   14   25  6  4 64 27  0

      0  0 3209508 10206216    148 15904728    0    0     0     0  164  278  0  0 100  0  0

      0  0 3209508 10206248    148 15904728    0    0     0    30  175  296  0  0 100  0  0

      1  0 3209508 10206020    148 15904492   84    0    84     4  199  310  0  0 99  0  0

      2  0 3208484 10203100    148 15904632  400    0   400    49  869  805  4  0 94  1  0

      [[email protected] ~]#

       

       

      [[email protected] ~]# free -m

                    total        used        free      shared  buff/cache   available

      Mem:          31202        5744        9926        1583       15532       23467

      Swap:         24575        3132       21443

      [[email protected] ~]#

       

       

       

      Attachements:

       

      • Zabbix_server.conf

       

       ++ 

      Snapshot

       ++ 

      • Logs /var/log/Zabbix/Zabbix_server.log**

       

      Zabbix Service and Poller information

       

       Best Regards

      Satender Sharma

       

       

      Attachments

        Activity

          People

            zabbix.support Zabbix Support Team
            satender1 Satender Kumar Ramniwas Sharma
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: