-
Problem report
-
Resolution: Declined
-
Trivial
-
None
-
5.0.10, 5.0.11
-
Redhat 8 with Maria db
Dear Team,
Seeking your help with the below issues.
We installed the Production Zabbix system and integrated around "1351" Nodes into them and we have to integrated 800 more devices into the production system. So In total, we will have above 2K integrated nodes.
Please let me know if any more information is required. Thank you.
System Information:
CPU : 8 CPU
RAM: 32 GB
DISK: 3 TB
OS: Red Hat Enterprise Linux release 8.3 (Ootpa)
Zabbix Version: 5.0
Issue: Data is not updating to Devices and all option hangs, Zabbix server services required to restart manually every 3 hours.
Troubleshooting:
- As a part of troubleshooting, I have increased the "Start poller" to 375, and the connection to Mysql increase to =1000.
- Poller probing in progress but no data updating.
- Mysql optimizer_switch='index_condition_pushdown=off'; is switch off
Below cache configuration done in the Zabbix_server.conf
VMwareCacheSize=2048M HousekeepingFrequency=1 CacheSize=8144M CacheUpdateFrequency=60 HistoryCacheSize=1024M HistoryIndexCacheSize=1024M TrendCacheSize=512M ValueCacheSize=1024M Timeout=10
Mysql Configuration :
MariaDB [(none)]> Show variables like 'max_connections'; ----------------------+ Variable_name Value ----------------------+ max_connections 1000 ----------------------+ 1 row in set (0.224 sec) MariaDB [(none)]> SELECT @@optimizer_switch\G *************************** 1. row *************************** @@optimizer_switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on,split_materialized=on 1 row in set (0.000 sec) MariaDB [(none)]> -----------------------------------+ database size in GB -----------------------------------+ information_schema 0.000167846680 mysql 0.000945094042 performance_schema 0.000000000000 zabbix 67.724426269531 -----------------------------------+ 4 rows in set (0.050 sec) MariaDB [zabbix]>
[root@zabbix ~]# iostat 5 10 Linux 4.18.0-240.el8.x86_64 (zabbix.mobily.com.sa) 05/12/2021 x86_64 (8 CPU) avg-cpu: %user %nice %system %iowait %steal %idle 5.56 0.00 3.56 27.36 0.00 63.52 Device tps kB_read/s kB_wrtn/s kB_read kB_wrtn sda 932.92 16913.87 20996.90 3072589362 3814314887 scd0 0.00 0.14 0.00 25612 0 dm-0 935.99 16859.41 20764.38 3062694894 3772075564 dm-1 71.31 53.87 231.37 9785804 42030572 dm-2 0.03 0.38 1.14 68635 206575 avg-cpu: %user %nice %system %iowait %steal %idle 0.33 0.00 0.07 1.12 0.00 98.47 Device tps kB_read/s kB_wrtn/s kB_read kB_wrtn sda 17.40 124.80 33.40 624 167 scd0 0.00 0.00 0.00 0 0 dm-0 3.60 4.80 33.40 24 167 dm-1 30.00 120.00 0.00 600 0 dm-2 0.00 0.00 0.00 0 0 avg-cpu: %user %nice %system %iowait %steal %idle 6.10 0.00 0.68 2.23 0.00 91.00 Device tps kB_read/s kB_wrtn/s kB_read kB_wrtn sda 32.40 428.00 59.40 2140 297 scd0 0.00 0.00 0.00 0 0 dm-0 5.00 6.40 59.40 32 297 dm-1 105.40 421.60 0.00 2108 0 dm-2 0.00 0.00 0.00 0 0 [root@zabbix ~]# vmstat 3 5 procs ----------memory--------- --swap- ----io--- system- -----cpu---- r b swpd free buff cache si so bi bo in cs us sy id wa st 0 0 3209508 10206468 148 15904656 7 29 2121 2633 14 25 6 4 64 27 0 0 0 3209508 10206216 148 15904728 0 0 0 0 164 278 0 0 100 0 0 0 0 3209508 10206248 148 15904728 0 0 0 30 175 296 0 0 100 0 0 1 0 3209508 10206020 148 15904492 84 0 84 4 199 310 0 0 99 0 0 2 0 3208484 10203100 148 15904632 400 0 400 49 869 805 4 0 94 1 0 [root@zabbix ~]# free -m total used free shared buff/cache available Mem: 31202 5744 9926 1583 15532 23467 Swap: 24575 3132 21443
