ZABBIX BUGS AND ISSUES

Zabbix Server constantly crash

Details

  • Type: Bug Bug
  • Status: Closed Closed
  • Priority: Blocker Blocker
  • Resolution: Duplicate
  • Affects Version/s: 1.8.7
  • Fix Version/s: None
  • Component/s: Server (S)
  • Labels:
    None
  • Zabbix ID:
    NA

Description

My Zabbix server constantly crash every 10 minutes.
Log:
 17749:20111026:154510.774 server #521 started [self-monitoring #1]
 16706:20111026:154510.774 server #0 started [main process]
 17400:20111026:154510.775 server #351 started [unreachable poller #49]
 17466:20111026:154521.097 Got signal [signal:11(SIGSEGV),reason:1,refaddr:(nil)]. Crashing ...
 17466:20111026:154521.097 ====== Fatal information: ======
 17466:20111026:154521.097 Program counter: 0x3df48796f0
 17466:20111026:154521.097 === Registers: ===
 17466:20111026:154521.097 r8 = fefefefefefefeff = 18374403900871474943 = -72340172838076673
 17466:20111026:154521.097 r9 = 101010101010101 = 72340172838076673 = 72340172838076673
 17466:20111026:154521.097 r10 = 393133313d3c6b63 = 4121131420594498403 = 4121131420594498403
 17466:20111026:154521.097 r11 = 3df487aa36 = 266095536694 = 266095536694
 17466:20111026:154521.097 r12 = 81f = 2079 = 2079
 17466:20111026:154521.097 r13 = 4732de = 4666078 = 4666078
 17466:20111026:154521.097 r14 = 11b11e0 = 18551264 = 18551264
 17466:20111026:154521.097 r15 = 4ea7ba8d = 1319615117 = 1319615117
 17466:20111026:154521.097 rdi = 0 = 0 = 0
 17466:20111026:154521.097 rsi = 81f = 2079 = 2079
 17466:20111026:154521.097 rbp = 0 = 0 = 0
 17466:20111026:154521.097 rbx = 9 = 9 = 9
 17466:20111026:154521.097 rdx = 0 = 0 = 0
 17466:20111026:154521.097 rax = 0 = 0 = 0
 17466:20111026:154521.097 rcx = 0 = 0 = 0
 17466:20111026:154521.098 rsp = 7fff3e972098 = 140734243479704 = 140734243479704
 17466:20111026:154521.098 rip = 3df48796f0 = 266095531760 = 266095531760
 17466:20111026:154521.098 efl = 10246 = 66118 = 66118
 17466:20111026:154521.098 csgsfs = 33 = 51 = 51
 17466:20111026:154521.098 err = 4 = 4 = 4
 17466:20111026:154521.098 trapno = e = 14 = 14
 17466:20111026:154521.098 oldmask = 0 = 0 = 0
 17466:20111026:154521.098 cr2 = 0 = 0 = 0
 17466:20111026:154521.098 === Backtrace: ===
 17466:20111026:154521.098 15: /usr/local/zabbix/sbin/zabbix_server(print_fatal_info+0xdb) [0x43978b]
 17466:20111026:154521.098 14: /usr/local/zabbix/sbin/zabbix_server [0x438fb8]
 17466:20111026:154521.098 13: /lib64/libc.so.6 [0x3df48302d0]
 17466:20111026:154521.098 12: /lib64/libc.so.6(strlen+0x30) [0x3df48796f0]
 17466:20111026:154521.098 11: /lib64/libc.so.6(__strdup+0x16) [0x3df4879426]
 17466:20111026:154521.098 10: /usr/local/zabbix/sbin/zabbix_server(zbx_strdup2+0x3e) [0x43d90e]
 17466:20111026:154521.098 9: /usr/local/zabbix/sbin/zabbix_server(DBget_history+0x268) [0x455d28]
 17466:20111026:154521.098 8: /usr/local/zabbix/sbin/zabbix_server(evaluate_function+0xb33) [0x44a5a3]
 17466:20111026:154521.098 7: /usr/local/zabbix/sbin/zabbix_server(evaluate_expression+0x4cd) [0x44f9cd]
 17466:20111026:154521.099 6: /usr/local/zabbix/sbin/zabbix_server [0x42f473]
 17466:20111026:154521.099 5: /usr/local/zabbix/sbin/zabbix_server(DCsync_history+0xc55) [0x430225]
 17466:20111026:154521.099 4: /usr/local/zabbix/sbin/zabbix_server(main_dbsyncer_loop+0xc9) [0x40fc09]
 17466:20111026:154521.099 3: /usr/local/zabbix/sbin/zabbix_server(MAIN_ZABBIX_ENTRY+0x950) [0x40f550]
 17466:20111026:154521.099 2: /usr/local/zabbix/sbin/zabbix_server(daemon_start+0x245) [0x438c05]
 17466:20111026:154521.099 1: /lib64/libc.so.6(__libc_start_main+0xf4) [0x3df481d994]
 17466:20111026:154521.099 0: /usr/local/zabbix/sbin/zabbix_server [0x40b849]
 17466:20111026:154521.099 === Memory map: ===
 17466:20111026:154521.099 00400000-0047f000 r-xp 00000000 08:02 1109085 /usr/local/zabbix/sbin/zabbix_server
 17466:20111026:154521.099 0067f000-006a4000 rw-p 0007f000 08:02 1109085 /usr/local/zabbix/sbin/zabbix_server
 17466:20111026:154521.099 006a4000-006aa000 rw-p 006a4000 00:00 0
 17466:20111026:154521.099 01174000-011cf000 rw-p 01174000 00:00 0 [heap]
 17466:20111026:154521.099 3958600000-3958631000 r-xp 00000000 08:02 703875 /usr/lib64/libidn.so.11.5.19
 17466:20111026:154521.099 3958631000-3958830000 ---p 00031000 08:02 703875 /usr/lib64/libidn.so.11.5.19
 17466:20111026:154521.099 3958830000-3958831000 rw-p 00030000 08:02 703875 /usr/lib64/libidn.so.11.5.19
 17466:20111026:154521.099 3959200000-395923b000 r-xp 00000000 08:02 703876 /usr/lib64/libcurl.so.3.0.0
 17466:20111026:154521.099 395923b000-395943b000 ---p 0003b000 08:02 703876 /usr/lib64/libcurl.so.3.0.0
 17466:20111026:154521.099 395943b000-395943d000 rw-p 0003b000 08:02 703876 /usr/lib64/libcurl.so.3.0.0
 17466:20111026:154521.099 3959600000-3959690000 r-xp 00000000 08:02 703878 /usr/lib64/libnetsnmp.so.10.0.3
 17466:20111026:154521.099 3959690000-395988f000 ---p 00090000 08:02 703878 /usr/lib64/libnetsnmp.so.10.0.3
 17466:20111026:154521.099 395988f000-3959893000 rw-p 0008f000 08:02 703878 /usr/lib64/libnetsnmp.so.10.0.3
 17466:20111026:154521.099 3959893000-39598c7000 rw-p 3959893000 00:00 0
 17466:20111026:154521.099 3df4400000-3df441c000 r-xp 00000000 08:02 2606372 /lib64/ld-2.5.so
 17466:20111026:154521.099 3df461b000-3df461c000 r--p 0001b000 08:02 2606372 /lib64/ld-2.5.so
 17466:20111026:154521.099 3df461c000-3df461d000 rw-p 0001c000 08:02 2606372 /lib64/ld-2.5.so
 17466:20111026:154521.099 3df4800000-3df494d000 r-xp 00000000 08:02 2606373 /lib64/libc-2.5.so
 17466:20111026:154521.099 3df494d000-3df4b4d000 ---p 0014d000 08:02 2606373 /lib64/libc-2.5.so
 17466:20111026:154521.099 3df4b4d000-3df4b51000 r--p 0014d000 08:02 2606373 /lib64/libc-2.5.so
 17466:20111026:154521.099 3df4b51000-3df4b52000 rw-p 00151000 08:02 2606373 /lib64/libc-2.5.so
 17466:20111026:154521.099 3df4b52000-3df4b57000 rw-p 3df4b52000 00:00 0
 17466:20111026:154521.099 3df4c00000-3df4c02000 r-xp 00000000 08:02 2606374 /lib64/libdl-2.5.so
 17466:20111026:154521.099 3df4c02000-3df4e02000 ---p 00002000 08:02 2606374 /lib64/libdl-2.5.so
 17466:20111026:154521.099 3df4e02000-3df4e03000 r--p 00002000 08:02 2606374 /lib64/libdl-2.5.so
 17466:20111026:154521.099 3df4e03000-3df4e04000 rw-p 00003000 08:02 2606374 /lib64/libdl-2.5.so
 17466:20111026:154521.099 3df5400000-3df5482000 r-xp 00000000 08:02 2606379 /lib64/libm-2.5.so
 17466:20111026:154521.099 3df5482000-3df5681000 ---p 00082000 08:02 2606379 /lib64/libm-2.5.so
 17466:20111026:154521.099 3df5681000-3df5682000 r--p 00081000 08:02 2606379 /lib64/libm-2.5.so
 17466:20111026:154521.099 3df5682000-3df5683000 rw-p 00082000 08:02 2606379 /lib64/libm-2.5.so
 17466:20111026:154521.099 3df5c00000-3df5c15000 r-xp 00000000 08:02 2606385 /lib64/libselinux.so.1
 17466:20111026:154521.099 3df5c15000-3df5e15000 ---p 00015000 08:02 2606385 /lib64/libselinux.so.1
 17466:20111026:154521.099 3df5e15000-3df5e17000 rw-p 00015000 08:02 2606385 /lib64/libselinux.so.1
 17466:20111026:154521.099 3df5e17000-3df5e18000 rw-p 3df5e17000 00:00 0
 17466:20111026:154521.100 3df6000000-3df6014000 r-xp 00000000 08:02 692681 /usr/lib64/libz.so.1.2.3
 17466:20111026:154521.100 3df6014000-3df6213000 ---p 00014000 08:02 692681 /usr/lib64/libz.so.1.2.3
 17466:20111026:154521.100 3df6213000-3df6214000 rw-p 00013000 08:02 692681 /usr/lib64/libz.so.1.2.3
 17466:20111026:154521.100 3df6400000-3df643b000 r-xp 00000000 08:02 2606384 /lib64/libsepol.so.1
 17466:20111026:154521.100 3df643b000-3df663b000 ---p 0003b000 08:02 2606384 /lib64/libsepol.so.1
 17466:20111026:154521.100 3df663b000-3df663c000 rw-p 0003b000 08:02 2606384 /lib64/libsepol.so.1
 17466:20111026:154521.100 3df663c000-3df6646000 rw-p 3df663c000 00:00 0
 17466:20111026:154521.100 3df7000000-3df7009000 r-xp 00000000 08:02 2606386 /lib64/libcrypt-2.5.so
 17466:20111026:154521.100 3df7009000-3df7208000 ---p 00009000 08:02 2606386 /lib64/libcrypt-2.5.so
 17466:20111026:154521.100 3df7208000-3df7209000 r--p 00008000 08:02 2606386 /lib64/libcrypt-2.5.so
 17466:20111026:154521.100 3df7209000-3df720a000 rw-p 00009000 08:02 2606386 /lib64/libcrypt-2.5.so
 17466:20111026:154521.100 3df720a000-3df7238000 rw-p 3df720a000 00:00 0
 17466:20111026:154521.100 3df7c00000-3df7c15000 r-xp 00000000 08:02 2606364 /lib64/libnsl-2.5.so
 17466:20111026:154521.100 3df7c15000-3df7e14000 ---p 00015000 08:02 2606364 /lib64/libnsl-2.5.so
 17466:20111026:154521.100 3df7e14000-3df7e15000 r--p 00014000 08:02 2606364 /lib64/libnsl-2.5.so
 17466:20111026:154521.100 3df7e15000-3df7e16000 rw-p 00015000 08:02 2606364 /lib64/libnsl-2.5.so
 17466:20111026:154521.100 3df7e16000-3df7e18000 rw-p 3df7e16000 00:00 0
 17466:20111026:154521.100 3df8400000-3df840d000 r-xp 00000000 08:02 2606135 /lib64/libgcc_s-4.1.2-20080825.so.1
 17466:20111026:154521.100 3df840d000-3df860d000 ---p 0000d000 08:02 2606135 /lib64/libgcc_s-4.1.2-20080825.so.1
 17466:20111026:154521.100 3df860d000-3df860e000 rw-p 0000d000 08:02 2606135 /lib64/libgcc_s-4.1.2-20080825.so.1
 17466:20111026:154521.100 3dfa800000-3dfa92d000 r-xp 00000000 08:02 2606387 /lib64/libcrypto.so.0.9.8e
 17466:20111026:154521.100 3dfa92d000-3dfab2c000 ---p 0012d000 08:02 2606387 /lib64/libcrypto.so.0.9.8e
 17466:20111026:154521.100 3dfab2c000-3dfab4d000 rw-p 0012c000 08:02 2606387 /lib64/libcrypto.so.0.9.8e
 17466:20111026:154521.100 3dfab4d000-3dfab51000 rw-p 3dfab4d000 00:00 0
 17466:20111026:154521.100 3dfb000000-3dfb011000 r-xp 00000000 08:02 2606391 /lib64/libresolv-2.5.so
 17466:20111026:154521.100 3dfb011000-3dfb211000 ---p 00011000 08:02 2606391 /lib64/libresolv-2.5.so
 17466:20111026:154521.100 3dfb211000-3dfb212000 r--p 00011000 08:02 2606391 /lib64/libresolv-2.5.so
 17466:20111026:154521.100 3dfb212000-3dfb213000 rw-p 00012000 08:02 2606391 /lib64/libresolv-2.5.so
 17466:20111026:154521.100 3dfb213000-3dfb215000 rw-p 3dfb213000 00:00 0
 17466:20111026:154521.100 3dfcc00000-3dfcc02000 r-xp 00000000 08:02 2606392 /lib64/libcom_err.so.2.1
 17466:20111026:154521.100 3dfcc02000-3dfce01000 ---p 00002000 08:02 2606392 /lib64/libcom_err.so.2.1
 17466:20111026:154521.100 3dfce01000-3dfce02000 rw-p 00001000 08:02 2606392 /lib64/libcom_err.so.2.1
 17466:20111026:154521.100 3dfe400000-3dfe402000 r-xp 00000000 08:02 2606139 /lib64/libkeyutils-1.2.so
 17466:20111026:154521.100 3dfe402000-3dfe601000 ---p 00002000 08:02 2606139 /lib64/libkeyutils-1.2.so
 17466:20111026:154521.100 3dfe601000-3dfe602000 rw-p 00001000 08:02 2606139 /lib64/libkeyutils-1.2.so
 17466:20111026:154521.100 3dfec00000-3dfec24000 r-xp 00000000 08:02 701239 /usr/lib64/libk5crypto.so.3.1
 17466:20111026:154521.100 3dfec24000-3dfee23000 ---p 00024000 08:02 701239 /usr/lib64/libk5crypto.so.3.1
 17466:20111026:154521.100 3dfee23000-3dfee25000 rw-p 00023000 08:02 701239 /usr/lib64/libk5crypto.so.3.1
 17466:20111026:154521.100 3dff400000-3dff491000 r-xp 00000000 08:02 701240 /usr/lib64/libkrb5.so.3.3
 17466:20111026:154521.100 3dff491000-3dff691000 ---p 00091000 08:02 701240 /usr/lib64/libkrb5.so.3.3
 17466:20111026:154521.100 3dff691000-3dff695000 rw-p 00091000 08:02 701240 /usr/lib64/libkrb5.so.3.3
 17466:20111026:154521.100 3dff800000-3dff808000 r-xp 00000000 08:02 701238 /usr/lib64/libkrb5support.so.0.1
 17466:20111026:154521.100 3dff808000-3dffa07000 ---p 00008000 08:02 701238 /usr/lib64/libkrb5support.so.0.1
 17466:20111026:154521.100 3dffa07000-3dffa08000 rw-p 00007000 08:02 701238 /usr/lib64/libkrb5support.so.0.1
 17466:20111026:154521.100 3e01000000-3e0102c000 r-xp 00000000 08:02 701241 /usr/lib64/libgssapi_krb5.so.2.2
 17466:20111026:154521.100 3e0102c000-3e0122c000 ---p 0002c000 08:02 701241 /usr/lib64/libgssapi_krb5.so.2.2
 17466:20111026:154521.101 3e0122c000-3e0122e000 rw-p 0002c000 08:02 701241 /usr/lib64/libgssapi_krb5.so.2.2
 17466:20111026:154521.101 3e04600000-3e04645000 r-xp 00000000 08:02 2606393 /lib64/libssl.so.0.9.8e
 17466:20111026:154521.101 3e04645000-3e04844000 ---p 00045000 08:02 2606393 /lib64/libssl.so.0.9.8e
 17466:20111026:154521.101 3e04844000-3e0484a000 rw-p 00044000 08:02 2606393 /lib64/libssl.so.0.9.8e
 17466:20111026:154521.101 2b4d13bd6000-2b4d13bd8000 rw-p 2b4d13bd6000 00:00 0
 17466:20111026:154521.101 2b4d13be5000-2b4d13be9000 rw-p 2b4d13be5000 00:00 0
 17466:20111026:154521.101 2b4d13be9000-2b4d13bf1000 r-xp 00000000 08:02 2606366 /lib64/libwrap.so.0.7.6
 17466:20111026:154521.101 2b4d13bf1000-2b4d13df0000 ---p 00008000 08:02 2606366 /lib64/libwrap.so.0.7.6
 17466:20111026:154521.101 2b4d13df0000-2b4d13df2000 rw-p 00007000 08:02 2606366 /lib64/libwrap.so.0.7.6
 17466:20111026:154521.101 2b4d13df2000-2b4d13df3000 rw-p 2b4d13df2000 00:00 0
 17466:20111026:154521.101 2b4d13df3000-2b4d13f1d000 r-xp 00000000 08:02 985915 /usr/lib64/mysql/libmysqlclient.so.15.0.0
 17466:20111026:154521.101 2b4d13f1d000-2b4d1411d000 ---p 0012a000 08:02 985915 /usr/lib64/mysql/libmysqlclient.so.15.0.0
 17466:20111026:154521.101 2b4d1411d000-2b4d14164000 rw-p 0012a000 08:02 985915 /usr/lib64/mysql/libmysqlclient.so.15.0.0
 17466:20111026:154521.101 2b4d14164000-2b4d1416a000 rw-p 2b4d14164000 00:00 0
 17466:20111026:154521.101 2b4d1416a000-2b4d14174000 r-xp 00000000 08:02 2606105 /lib64/libnss_files-2.5.so
 17466:20111026:154521.101 2b4d14174000-2b4d14373000 ---p 0000a000 08:02 2606105 /lib64/libnss_files-2.5.so
 17466:20111026:154521.101 2b4d14373000-2b4d14374000 r--p 00009000 08:02 2606105 /lib64/libnss_files-2.5.so
 17466:20111026:154521.101 2b4d14374000-2b4d14375000 rw-p 0000a000 08:02 2606105 /lib64/libnss_files-2.5.so
 17466:20111026:154521.101 2b4d14375000-2b4d23e26000 rw-s 00000000 00:09 57376768 /SYSV6802233c (deleted)
 17466:20111026:154521.101 2b4d23e26000-2b4d33827000 rw-s 00000000 00:09 57409537 /SYSV7802233c (deleted)
 17466:20111026:154521.101 2b4d33827000-2b4d43228000 rw-s 00000000 00:09 57442306 /SYSV7402233c (deleted)
 17466:20111026:154521.101 2b4d43228000-2b4d5db28000 rw-s 00000000 00:09 57475075 /SYSV6702233c (deleted)
 17466:20111026:154521.101 2b4d5db28000-2b4d62628000 rw-s 00000000 00:09 57507844 /SYSV7302233c (deleted)
 17466:20111026:154521.101 2b4d62628000-2b4d6264a000 rw-s 00000000 00:09 57540613 /SYSV5302233c (deleted)
 17466:20111026:154521.101 2b4d6264a000-2b4d6266c000 rw-p 2b4d6264a000 00:00 0
 17466:20111026:154521.101 7fff3e960000-7fff3e985000 rw-p 7ffffffda000 00:00 0 [stack]
 17466:20111026:154521.101 ffffffffff600000-ffffffffffe00000 ---p 00000000 00:00 0 [vdso]
 17466:20111026:154521.101 ================================
 17466:20111026:154521.101 Please consider attaching a disassembly listing to your bug report.
 17466:20111026:154521.101 This listing can be produced with, e.g., objdump -DSswx zabbix_server.
 17466:20111026:154521.101 ================================
 16706:20111026:154521.102 One child process died (PID:17466,exitcode/signal:255). Exiting ...
 16706:20111026:154523.583 Zabbix Server stopped. Zabbix 1.8.7 (revision 21394).

[root@GZT-Zabbix-33 zabbix]# tail -f zabbix_server.log
 17466:20111026:154521.101 2b4d62628000-2b4d6264a000 rw-s 00000000 00:09 57540613 /SYSV5302233c (deleted)
 17466:20111026:154521.101 2b4d6264a000-2b4d6266c000 rw-p 2b4d6264a000 00:00 0
 17466:20111026:154521.101 7fff3e960000-7fff3e985000 rw-p 7ffffffda000 00:00 0 [stack]
 17466:20111026:154521.101 ffffffffff600000-ffffffffffe00000 ---p 00000000 00:00 0 [vdso]
 17466:20111026:154521.101 ================================
 17466:20111026:154521.101 Please consider attaching a disassembly listing to your bug report.
 17466:20111026:154521.101 This listing can be produced with, e.g., objdump -DSswx zabbix_server.
 17466:20111026:154521.101 ================================
 16706:20111026:154521.102 One child process died (PID:17466,exitcode/signal:255). Exiting ...
 16706:20111026:154523.583 Zabbix Server stopped. Zabbix 1.8.7 (revision 21394).
  1. objdump.zabbinx_server.tgz
    2011 Oct 26 11:12
    6.94 MB
    Wenchun Chen
  2. zabbix_server.conf
    2011 Oct 26 11:12
    9 kB
    Wenchun Chen
  3. zabbix.sql
    2011 Oct 26 11:02
    69 kB
    Wenchun Chen

Issue Links

Activity

Hide
Wenchun Chen added a comment -

DB structure

Show
Wenchun Chen added a comment - DB structure
Hide
Wenchun Chen added a comment -

attact objdump

Show
Wenchun Chen added a comment - attact objdump
Hide
Wenchun Chen added a comment -

Attach zabbix_server.conf

Show
Wenchun Chen added a comment - Attach zabbix_server.conf
Hide
Wenchun Chen added a comment -

Environment:
Redhat 5.4
Linux GZT-Zabbix-33 2.6.18-164.el5 #1 SMP Tue Aug 18 15:51:48 EDT 2009 x86_64 x86_64 x86_64 GNU/Linux
mysql> select version();
------------

version()

------------

5.5.15-log

------------

Show
Wenchun Chen added a comment - Environment: Redhat 5.4 Linux GZT-Zabbix-33 2.6.18-164.el5 #1 SMP Tue Aug 18 15:51:48 EDT 2009 x86_64 x86_64 x86_64 GNU/Linux mysql> select version(); ------------
version()
------------
5.5.15-log
------------
Hide
Aleksandrs Saveljevs added a comment -

Issue has already been fixed in Zabbix 1.8.8 under ZBX-4099. Please upgrade to a newer version.

Show
Aleksandrs Saveljevs added a comment - Issue has already been fixed in Zabbix 1.8.8 under ZBX-4099. Please upgrade to a newer version.

People

Vote (0)
Watch (0)

Dates

  • Created:
    Updated:
    Resolved: