[ZBX-15567] Zabbix Server 4.0. Crashes Created: 2019 Jan 31  Updated: 2019 Feb 05  Resolved: 2019 Feb 05

Status: Closed
Project: ZABBIX BUGS AND ISSUES
Component/s: Server (S)
Affects Version/s: 4.0.2
Fix Version/s: None

Type: Problem report Priority: Blocker
Reporter: Khaled Gawad Assignee: Zabbix Support Team
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Text File zabbix_server.log    
Issue Links:
Duplicate
duplicates ZBX-14908 Crashing escalator process zabbix_ser... Closed
is duplicated by ZBX-14908 Crashing escalator process zabbix_ser... Closed

 Description   

After two or three days of Zabbix Server Running:

Zabbix Server 4.0 Crashes with the following dump in logs:

-------------------------------------------------------------------------

22698:20190131:151527.187 Got signal [signal:11(SIGSEGV),reason:1,refaddr:(nil)]. Crashing ...
 22698:20190131:151527.187 ====== Fatal information: ======
 22698:20190131:151527.187 Program counter: 0x7f723ed70a81
 22698:20190131:151527.188 === Registers: ===
 22698:20190131:151527.188 r8 = 0 = 0 = 0
 22698:20190131:151527.188 r9 = 10 = 16 = 16
 22698:20190131:151527.188 r10 = fffffffffffff28b = 18446744073709548171 = -3445
 22698:20190131:151527.188 r11 = 8 = 8 = 8
 22698:20190131:151527.188 r12 = 55a76970d1a5 = 94177516900773 = 94177516900773
 22698:20190131:151527.188 r13 = 1 = 1 = 1
 22698:20190131:151527.188 r14 = 0 = 0 = 0
 22698:20190131:151527.188 r15 = 0 = 0 = 0
 22698:20190131:151527.188 rdi = 0 = 0 = 0
 22698:20190131:151527.188 rsi = cb6 = 3254 = 3254
 22698:20190131:151527.188 rbp = 7fffdc2b5500 = 140736887215360 = 140736887215360
 22698:20190131:151527.188 rbx = 0 = 0 = 0
 22698:20190131:151527.188 rdx = 0 = 0 = 0
 22698:20190131:151527.188 rax = 0 = 0 = 0
 22698:20190131:151527.188 rcx = 0 = 0 = 0
 22698:20190131:151527.188 rsp = 7fffdc2b54a8 = 140736887215272 = 140736887215272
 22698:20190131:151527.188 rip = 7f723ed70a81 = 140128657279617 = 140128657279617
 22698:20190131:151527.188 efl = 10283 = 66179 = 66179
 22698:20190131:151527.188 csgsfs = 33 = 51 = 51
 22698:20190131:151527.188 err = 4 = 4 = 4
 22698:20190131:151527.188 trapno = e = 14 = 14
 22698:20190131:151527.188 oldmask = 0 = 0 = 0
 22698:20190131:151527.188 cr2 = 0 = 0 = 0
 22698:20190131:151527.188 === Backtrace: ===
 22698:20190131:151527.189 18: /usr/sbin/zabbix_server: escalator #1 [processed 0 escalations in 0.000000 sec, processing escalations](zbx_log_fatal_info+0x142) [0x55a769637706]
 22698:20190131:151527.189 17: /usr/sbin/zabbix_server: escalator #1 [processed 0 escalations in 0.000000 sec, processing escalations](+0xf1b19) [0x55a769637b19]
 22698:20190131:151527.189 16: /lib64/libpthread.so.0(+0xf680) [0x7f7242074680]
 22698:20190131:151527.189 15: /lib64/libc.so.6(+0x8ca81) [0x7f723ed70a81]
 22698:20190131:151527.189 14: /lib64/libc.so.6(__strdup+0xe) [0x7f723ed7078e]

 

------------------------------------------------------------------------------------------

The issue is repeating after some time:

we have the following configurations

 * CacheSize=2G
 * CacheUpdateFrequency=300
 * HistoryCacheSize=2G
 * HistoryIndexCacheSize=128M
 * TrendCacheSize=2G
 * ValueCacheSize=2G
 * StartPollers=400
 * StartPollersUnreachable=20
 * StartHTTPPollers=20
 * StartTrappers=50

 

Other configurations are default ones.

Zabbix Server RAM Memory : 16 GB

After crash:

======

top - 15:45:47 up 156 days, 6:00, 3 users, load average: 0.00, 0.03, 0.14
Tasks: 318 total, 1 running, 310 sleeping, 0 stopped, 7 zombie
%Cpu(s): 0.1 us, 0.1 sy, 0.0 ni, 99.8 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
KiB Mem : 16250548 total, 14786196 free, 339052 used, 1125300 buff/cache
KiB Swap: 2097148 total, 1459716 free, 637432 used. 14898176 avail Mem


 Comments   
Comment by Khaled Gawad [ 2019 Jan 31 ]

Please assist here

Comment by Alexey Pustovalov [ 2019 Jan 31 ]

Please attach full log for 22698 PID.

Comment by Alexey Pustovalov [ 2019 Jan 31 ]

Also please show us zabbix_server -V output.

Comment by Khaled Gawad [ 2019 Feb 01 ]

Attached log file contains two restarts and two crashes.

Comment by Khaled Gawad [ 2019 Feb 01 ]

$ zabbix_server -V
zabbix_server (Zabbix) 4.0.0
Revision 85308 1 October 2018, compilation time: Oct 1 2018 08:41:15

Copyright (C) 2018 Zabbix SIA
License GPLv2+: GNU GPL version 2 or later <http://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it according to
the license. There is NO WARRANTY, to the extent permitted by law.

Comment by Khaled Gawad [ 2019 Feb 01 ]

There are 12+ Active Proxy Servers connected to same Zabbix Sever.

Comment by Alexey Pustovalov [ 2019 Feb 01 ]

You told you have 4.0.2 Please update Zabbix server to latest 4.0.3 version. It is known issue and fixed since 4.0.2 version: ZBX-14908.

Comment by Khaled Gawad [ 2019 Feb 01 ]

When we open the ticket, 4.0.0 is not shown in the list we could not open the ticket before we give a version . this why we put least 4.0.2, 

What are steps / Guide to upgrade to latest version ?

 

Thanks

 

 

Comment by Alexey Pustovalov [ 2019 Feb 01 ]

Upgrade procedure is described in our official documentation: https://www.zabbix.com/documentation/4.0/manual/installation/upgrade

Comment by Arturs Lontons [ 2019 Feb 05 ]

Hi,

I will be closing the ticket as a duplicate of ZBX-14908

Generated at Fri Mar 29 17:29:20 EET 2024 using Jira 9.12.4#9120004-sha1:625303b708afdb767e17cb2838290c41888e9ff0.