-
Problem report
-
Resolution: Duplicate
-
Trivial
-
None
-
None
-
None
-
information:
- Zabbix server version: 6.0.9
- Version Operating System Zabbix Servers: Ubuntu 20.04.2 LTS (Focal Fossa)
- Hardware Zabbix servers:
.Instance type: r6i.4xlarge
.vCPU: 16
.Memory: 128 GB
.Disk: 100 GB
- Zabbix server runs in HA
- Database version: 8.0.mysql_aurora.3.04.1 (AWS Aurora MySQL)
- Hardware database:
. Instance class: db.r6g.12xlarge
. vCPU: 48
. RAM: 384GBinformation: - Zabbix server version: 6.0.9 - Version Operating System Zabbix Servers: Ubuntu 20.04.2 LTS (Focal Fossa) - Hardware Zabbix servers: .Instance type: r6i.4xlarge .vCPU: 16 .Memory: 128 GB .Disk: 100 GB - Zabbix server runs in HA - Database version: 8.0.mysql_aurora.3.04.1 (AWS Aurora MySQL) - Hardware database: . Instance class: db.r6g.12xlarge . vCPU: 48 . RAM: 384GB
Hello,
We would like a help regarding an unexpected crash of the Zabbix server that we had on July 29, 2024, where a crash and failover of the HA occurred due to this problem, we need help to identify the cause of this problem as we have not found any cause that justifies it What happened seems to be a bug according to the logs.
There was no OS or Zabbix application resource increase detected, it just suddenly crashed.
LOG Zabbix server
zabbix_server [639138]: ERROR [file and function: <cache.c,tfc_index_add>, revision:64721203c07, line:311] Something impossible has just happened. 639138:20240629:130729.792 === Backtrace: === 639138:20240629:130729.793 25: /usr/sbin/zabbix_server: history poller #17 [got 288 values in 3.343837 sec, getting values](zbx_backtrace+0x52) [0x561efdbc836e] 639138:20240629:130729.793 24: /usr/sbin/zabbix_server: history poller #17 [got 288 values in 3.343837 sec, getting values](+0x38d70a) [0x561efdcde70a] 639138:20240629:130729.793 23: /usr/sbin/zabbix_server: history poller #17 [got 288 values in 3.343837 sec, getting values](zbx_tfc_put_value+0x2f8) [0x561efdcdf139] 639138:20240629:130729.794 22: /usr/sbin/zabbix_server: history poller #17 [got 288 values in 3.343837 sec, getting values](zbx_trends_eval_avg+0xa4) [0x561efdcdd7ca] 639138:20240629:130729.794 21: /usr/sbin/zabbix_server: history poller #17 [got 288 values in 3.343837 sec, getting values](+0x23619c) [0x561efdb8719c] 639138:20240629:130729.795 20: /usr/sbin/zabbix_server: history poller #17 [got 288 values in 3.343837 sec, getting values](+0x236be8) [0x561efdb87be8] 639138:20240629:130729.795 19: /usr/sbin/zabbix_server: history poller #17 [got 288 values in 3.343837 sec, getting values](evaluate_function2+0x55c) [0x561efdb8acb3] 639138:20240629:130729.795 18: /usr/sbin/zabbix_server: history poller #17 [got 288 values in 3.343837 sec, getting values](+0x256abd) [0x561efdba7abd] 639138:20240629:130729.796 17: /usr/sbin/zabbix_server: history poller #17 [got 288 values in 3.343837 sec, getting values](+0x258270) [0x561efdba9270] 639138:20240629:130729.796 16: /usr/sbin/zabbix_server: history poller #17 [got 288 values in 3.343837 sec, getting values](+0x26264e) [0x561efdbb364e] 639138:20240629:130729.796 15: /usr/sbin/zabbix_server: history poller #17 [got 288 values in 3.343837 sec, getting values](+0x2646bf) [0x561efdbb56bf] 639138:20240629:130729.797 14: /usr/sbin/zabbix_server: history poller #17 [got 288 values in 3.343837 sec, getting values](+0x2649a6) [0x561efdbb59a6] 639138:20240629:130729.797 13: /usr/sbin/zabbix_server: history poller #17 [got 288 values in 3.343837 sec, getting values](zbx_eval_execute_ext+0x5a) [0x561efdbb5cf4] 639138:20240629:130729.797 12: /usr/sbin/zabbix_server: history poller #17 [got 288 values in 3.343837 sec, getting values](zbx_expression_eval_execute+0x1b4) [0x561efdba9eb2] 639138:20240629:130729.798 11: /usr/sbin/zabbix_server: history poller #17 [got 288 values in 3.343837 sec, getting values](get_value_calculated+0x1bf) [0x561efd9ddc61] 639138:20240629:130729.798 10: /usr/sbin/zabbix_server: history poller #17 [got 288 values in 3.343837 sec, getting values](+0x88895) [0x561efd9d9895] 639138:20240629:130729.798 9: /usr/sbin/zabbix_server: history poller #17 [got 288 values in 3.343837 sec, getting values](zbx_check_items+0xc0) [0x561efd9dc19c] 639138:20240629:130729.799 8: /usr/sbin/zabbix_server: history poller #17 [got 288 values in 3.343837 sec, getting values](+0x8bd6c) [0x561efd9dcd6c] 639138:20240629:130729.799 7: /usr/sbin/zabbix_server: history poller #17 [got 288 values in 3.343837 sec, getting values](poller_thread+0x215) [0x561efd9dd791] 639138:20240629:130729.800 6: /usr/sbin/zabbix_server: history poller #17 [got 288 values in 3.343837 sec, getting values](zbx_thread_start+0x3b) [0x561efdbd9a80] 639138:20240629:130729.800 5: /usr/sbin/zabbix_server: history poller #17 [got 288 values in 3.343837 sec, getting values](+0x68ccf) [0x561efd9b9ccf] 639138:20240629:130729.800 4: /usr/sbin/zabbix_server: history poller #17 [got 288 values in 3.343837 sec, getting values](MAIN_ZABBIX_ENTRY+0xb6f) [0x561efd9bac94] 639138:20240629:130729.801 3: /usr/sbin/zabbix_server: history poller #17 [got 288 values in 3.343837 sec, getting values](daemon_start+0x3ca) [0x561efdbc7e91] 639138:20240629:130729.801 2: /usr/sbin/zabbix_server: history poller #17 [got 288 values in 3.343837 sec, getting values](main+0x34d) [0x561efd9b8ec2] 639138:20240629:130729.801 1: /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3) [0x7fdf8b2d8083] 639138:20240629:130729.802 0: /usr/sbin/zabbix_server: history poller #17 [got 288 values in 3.343837 sec, getting values](_start+0x2e) [0x561efd9b7b2e] 2069241:20240629:130729.813 One child process died (PID:639138,exitcode/signal:1). Exiting ... 639106:20240629:130729.823 sending configuration data to proxy "ZABBIX-PROXY" at "10.10.10.10", datalen 40000986, bytes 2401480 with compression ratio 16.7 639096:20240629:130730.032 sending configuration data to proxy "ZABBIX-PROXY03" at "10.10.10.11", datalen 43488896, bytes 2755171 with compression ratio 15.8 2069241:20240629:130730.587 PROCESS EXIT: 639138 638617:20240629:130730.587 HA manager has been paused 2069241:20240629:130731.674 syncing history data... 2069241:20240629:130834.060 cannot connect to service manager service: Cannot connect to service "service": [111] Connection refused. 643303:20240629:132023.591 Starting Zabbix Server. Zabbix 6.0.9 (revision 64721203c07).
- is duplicated by
-
-
- Closed
-