[ZBX-7659] Got signal [signal:7(SIGBUS),reason:128,refaddr:(nil)]. Crashing - in "poller" Created: 2014 Jan 15  Updated: 2017 May 30  Resolved: 2014 Mar 03

Status: Closed
Project: ZABBIX BUGS AND ISSUES
Component/s: Server (S)
Affects Version/s: 2.2.1
Fix Version/s: 2.0.12rc1, 2.2.3rc1, 2.3.0

Type: Incident report Priority: Major
Reporter: Johannes Assignee: Unassigned
Resolution: Fixed Votes: 1
Labels: crash, linux
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Linux 3.0.58-0.6.6-default #1 SMP Tue Feb 19 11:07:00 UTC 2013 (1576ecd) x86_64 x86_64 x86_64 GNU/Linux


Attachments: File objdump_zabbix_server.dump.bz2    
Issue Links:
Duplicate
is duplicated by ZBX-7603 Zabbix_server frequent crashes - in "... Closed

 Description   

after updating from 2.0.6 to 2.2.1 everything was running fine in the beginning. Since yesterday evening suddently the zabbix_server process stops multiple times a day

4346:20140115:145032.347 Got signal [signal:7(SIGBUS),reason:128,refaddr:(nil)]. Crashing ...
4346:20140115:145032.347 ====== Fatal information: ======
4346:20140115:145032.347 Program counter: 0x498424
4346:20140115:145032.347 === Registers: ===
4346:20140115:145032.347 r8 = 0 = 0 = 0
4346:20140115:145032.347 r9 = a = 10 = 10
4346:20140115:145032.347 r10 = 0 = 0 = 0
4346:20140115:145032.347 r11 = 1999999999999999 = 1844674407370955161 = 1844674407370955161
4346:20140115:145032.347 r12 = 892748 = 8988488 = 8988488
4346:20140115:145032.347 r13 = 0 = 0 = 0
4346:20140115:145032.348 r14 = 88e740 = 8972096 = 8972096
4346:20140115:145032.348 r15 = 40 = 64 = 64
4346:20140115:145032.348 rdi = 3 = 3 = 3
4346:20140115:145032.348 rsi = 88e550 = 8971600 = 8971600
4346:20140115:145032.348 rbp = 7fffe5f14400 = 140737051182080 = 140737051182080
4346:20140115:145032.348 rbx = 896700 = 9004800 = 9004800
4346:20140115:145032.348 rdx = 4ca830 = 5023792 = 5023792
4346:20140115:145032.348 rax = 5f786279 = 1601725049 = 1601725049
4346:20140115:145032.348 rcx = 7f8f1da6fe80 = 140252654534272 = 140252654534272
4346:20140115:145032.348 rsp = 7fffe5f13670 = 140737051178608 = 140737051178608
4346:20140115:145032.348 rip = 498424 = 4817956 = 4817956
4346:20140115:145032.348 efl = 10202 = 66050 = 66050
4346:20140115:145032.348 csgsfs = 33 = 51 = 51
4346:20140115:145032.348 err = 0 = 0 = 0
4346:20140115:145032.349 trapno = c = 12 = 12
4346:20140115:145032.349 oldmask = 0 = 0 = 0
4346:20140115:145032.349 cr2 = 0 = 0 = 0
4346:20140115:145032.349 === Backtrace: ===
4346:20140115:145032.349 11: /usr/sbin/zabbix_server: poller #7 [got 0 values in 0.000005 sec, getting values](print_fatal_info+0x10f) [0x4666ff]
4346:20140115:145032.350 10: /usr/sbin/zabbix_server: poller #7 [got 0 values in 0.000005 sec, getting values]() [0x466fd2]
4346:20140115:145032.350 9: /lib64/libc.so.6(+0x32bd0) [0x7f8f1d72fbd0]
4346:20140115:145032.350 8: /usr/sbin/zabbix_server: poller #7 [got 0 values in 0.000005 sec, getting values](DBcopy_template_elements+0xbd4) [0x498424]
4346:20140115:145032.350 7: /usr/sbin/zabbix_server: poller #7 [got 0 values in 0.000005 sec, getting values](DBlld_update_hosts+0xe04) [0x4b16a4]
4346:20140115:145032.350 6: /usr/sbin/zabbix_server: poller #7 [got 0 values in 0.000005 sec, getting values](DBlld_process_discovery_rule+0x531) [0x4a43d1]
4346:20140115:145032.350 5: /usr/sbin/zabbix_server: poller #7 [got 0 values in 0.000005 sec, getting values]() [0x42480d]
4346:20140115:145032.350 4: /usr/sbin/zabbix_server: poller #7 [got 0 values in 0.000005 sec, getting values](main_poller_loop+0x92) [0x425032]
4346:20140115:145032.350 3: /usr/sbin/zabbix_server: poller #7 [got 0 values in 0.000005 sec, getting values](MAIN_ZABBIX_ENTRY+0x829) [0x419779]
4346:20140115:145032.350 2: /usr/sbin/zabbix_server: poller #7 [got 0 values in 0.000005 sec, getting values](daemon_start+0x1b2) [0x465c92]
4346:20140115:145032.350 1: /lib64/libc.so.6(__libc_start_main+0xe6) [0x7f8f1d71bc16]
4346:20140115:145032.350 0: /usr/sbin/zabbix_server: poller #7 [got 0 values in 0.000005 sec, getting values]() [0x414339]
4346:20140115:145032.350 === Memory map: ===
4346:20140115:145032.350 00400000-00500000 r-xp 00000000 fd:05 233435 /usr/sbin/zabbix_server
4346:20140115:145032.350 006ff000-00750000 r--p 000ff000 fd:05 233435 /usr/sbin/zabbix_server
4346:20140115:145032.351 00750000-00754000 rw-p 00150000 fd:05 233435 /usr/sbin/zabbix_server
4346:20140115:145032.351 00754000-0077b000 rw-p 00000000 00:00 0 [heap]
4346:20140115:145032.351 0077b000-007b5000 rw-p 00000000 00:00 0 [heap]
4346:20140115:145032.351 007b5000-008b9000 rw-p 00000000 00:00 0 [heap]
4346:20140115:145032.351 7f8ef5901000-7f8ef5906000 r-xp 00000000 fd:01 106520 /lib64/libnss_dns-2.11.3.so
4346:20140115:145032.351 7f8ef5906000-7f8ef5b05000 ---p 00005000 fd:01 106520 /lib64/libnss_dns-2.11.3.so
4346:20140115:145032.351 7f8ef5b05000-7f8ef5b06000 r--p 00004000 fd:01 106520 /lib64/libnss_dns-2.11.3.so
4346:20140115:145032.351 7f8ef5b06000-7f8ef5b07000 rw-p 00005000 fd:01 106520 /lib64/libnss_dns-2.11.3.so
4346:20140115:145032.351 7f8ef5b07000-7f8ef5b1c000 r-xp 00000000 fd:01 106625 /lib64/libgcc_s.so.1
4346:20140115:145032.351 7f8ef5b1c000-7f8ef5d1b000 ---p 00015000 fd:01 106625 /lib64/libgcc_s.so.1
4346:20140115:145032.351 7f8ef5d1b000-7f8ef5d1c000 r--p 00014000 fd:01 106625 /lib64/libgcc_s.so.1
4346:20140115:145032.351 7f8ef5d1c000-7f8ef5d1d000 rw-p 00015000 fd:01 106625 /lib64/libgcc_s.so.1
4346:20140115:145032.352 7f8ef5d4f000-7f8ef5d50000 ---p 00000000 00:00 0
4346:20140115:145032.352 7f8ef5d50000-7f8ef6550000 rw-p 00000000 00:00 0
4346:20140115:145032.352 7f8ef6550000-7f8ef6d50000 rw-s 00000000 00:04 8192008 /SYSV760140cf (deleted)
4346:20140115:145032.352 7f8ef6d50000-7f8ef8d50000 rw-s 00000000 00:04 8159238 /SYSV770140cf (deleted)
4346:20140115:145032.352 7f8ef8d50000-7f8ef8d5b000 rw-s 00000000 00:04 8126469 /SYSV530140cf (deleted)
4346:20140115:145032.352 7f8ef8d5b000-7f8efda28000 rw-s 00000000 00:04 8093700 /SYSV730140cf (deleted)
4346:20140115:145032.352 7f8efda28000-7f8f18d5c000 rw-s 00000000 00:04 8060931 /SYSV670140cf (deleted)
4346:20140115:145032.352 7f8f18d5c000-7f8f1915c000 rw-s 00000000 00:04 8028162 /SYSV740140cf (deleted)
4346:20140115:145032.352 7f8f1915c000-7f8f1a15c000 rw-s 00000000 00:04 7995393 /SYSV780140cf (deleted)
4346:20140115:145032.352 7f8f1a15c000-7f8f1a95c000 rw-s 00000000 00:04 7962624 /SYSV680140cf (deleted)
4346:20140115:145032.352 7f8f1a95c000-7f8f1a968000 r-xp 00000000 fd:01 106522 /lib64/libnss_files-2.11.3.so
4346:20140115:145032.352 7f8f1a968000-7f8f1ab67000 ---p 0000c000 fd:01 106522 /lib64/libnss_files-2.11.3.so
4346:20140115:145032.352 7f8f1ab67000-7f8f1ab68000 r--p 0000b000 fd:01 106522 /lib64/libnss_files-2.11.3.so
4346:20140115:145032.353 7f8f1ab68000-7f8f1ab69000 rw-p 0000c000 fd:01 106522 /lib64/libnss_files-2.11.3.so
4346:20140115:145032.353 7f8f1ab69000-7f8f1ab73000 r-xp 00000000 fd:01 106526 /lib64/libnss_nis-2.11.3.so
4346:20140115:145032.353 7f8f1ab73000-7f8f1ad72000 ---p 0000a000 fd:01 106526 /lib64/libnss_nis-2.11.3.so
4346:20140115:145032.353 7f8f1ad72000-7f8f1ad73000 r--p 00009000 fd:01 106526 /lib64/libnss_nis-2.11.3.so
4346:20140115:145032.353 7f8f1ad73000-7f8f1ad74000 rw-p 0000a000 fd:01 106526 /lib64/libnss_nis-2.11.3.so
4346:20140115:145032.353 7f8f1ad74000-7f8f1ad89000 r-xp 00000000 fd:01 106516 /lib64/libnsl-2.11.3.so
4346:20140115:145032.353 7f8f1ad89000-7f8f1af88000 ---p 00015000 fd:01 106516 /lib64/libnsl-2.11.3.so
4346:20140115:145032.353 7f8f1af88000-7f8f1af89000 r--p 00014000 fd:01 106516 /lib64/libnsl-2.11.3.so
4346:20140115:145032.353 7f8f1af89000-7f8f1af8a000 rw-p 00015000 fd:01 106516 /lib64/libnsl-2.11.3.so
4346:20140115:145032.353 7f8f1af8a000-7f8f1af8c000 rw-p 00000000 00:00 0
4346:20140115:145032.353 7f8f1af8c000-7f8f1af93000 r-xp 00000000 fd:01 106518 /lib64/libnss_compat-2.11.3.so
4346:20140115:145032.353 7f8f1af93000-7f8f1b192000 ---p 00007000 fd:01 106518 /lib64/libnss_compat-2.11.3.so
4346:20140115:145032.353 7f8f1b192000-7f8f1b193000 r--p 00006000 fd:01 106518 /lib64/libnss_compat-2.11.3.so
4346:20140115:145032.354 7f8f1b193000-7f8f1b194000 rw-p 00007000 fd:01 106518 /lib64/libnss_compat-2.11.3.so
4346:20140115:145032.354 7f8f1b194000-7f8f1b19b000 r-xp 00000000 fd:05 134626 /usr/lib64/libkrb5support.so.0.1
4346:20140115:145032.354 7f8f1b19b000-7f8f1b39b000 ---p 00007000 fd:05 134626 /usr/lib64/libkrb5support.so.0.1
4346:20140115:145032.354 7f8f1b39b000-7f8f1b39c000 r--p 00007000 fd:05 134626 /usr/lib64/libkrb5support.so.0.1
4346:20140115:145032.354 7f8f1b39c000-7f8f1b39d000 rw-p 00008000 fd:05 134626 /usr/lib64/libkrb5support.so.0.1
4346:20140115:145032.354 7f8f1b39d000-7f8f1b3b6000 r-xp 00000000 fd:05 134656 /usr/lib64/libsasl2.so.2.0.22
4346:20140115:145032.354 7f8f1b3b6000-7f8f1b5b6000 ---p 00019000 fd:05 134656 /usr/lib64/libsasl2.so.2.0.22
4346:20140115:145032.354 7f8f1b5b6000-7f8f1b5b7000 r--p 00019000 fd:05 134656 /usr/lib64/libsasl2.so.2.0.22
4346:20140115:145032.354 7f8f1b5b7000-7f8f1b5b8000 rw-p 0001a000 fd:05 134656 /usr/lib64/libsasl2.so.2.0.22
4346:20140115:145032.354 7f8f1b5b8000-7f8f1b5bd000 r-xp 00000000 fd:05 131238 /usr/lib64/libgdbm.so.3.0.0
4346:20140115:145032.354 7f8f1b5bd000-7f8f1b7bc000 ---p 00005000 fd:05 131238 /usr/lib64/libgdbm.so.3.0.0
4346:20140115:145032.354 7f8f1b7bc000-7f8f1b7bd000 r--p 00004000 fd:05 131238 /usr/lib64/libgdbm.so.3.0.0
4346:20140115:145032.354 7f8f1b7bd000-7f8f1b7be000 rw-p 00005000 fd:05 131238 /usr/lib64/libgdbm.so.3.0.0
4346:20140115:145032.354 7f8f1b7be000-7f8f1b7fc000 r-xp 00000000 fd:01 106574 /lib64/libncurses.so.5.6
4346:20140115:145032.354 7f8f1b7fc000-7f8f1b9fc000 ---p 0003e000 fd:01 106574 /lib64/libncurses.so.5.6
4346:20140115:145032.354 7f8f1b9fc000-7f8f1ba00000 r--p 0003e000 fd:01 106574 /lib64/libncurses.so.5.6
4346:20140115:145032.354 7f8f1ba00000-7f8f1ba06000 rw-p 00042000 fd:01 106574 /lib64/libncurses.so.5.6
4346:20140115:145032.355 7f8f1ba06000-7f8f1ba0d000 r-xp 00000000 fd:05 165893 /usr/lib64/libOpenIPMIutils.so.0.0.1
4346:20140115:145032.355 7f8f1ba0d000-7f8f1bc0c000 ---p 00007000 fd:05 165893 /usr/lib64/libOpenIPMIutils.so.0.0.1
4346:20140115:145032.355 7f8f1bc0c000-7f8f1bc0d000 r--p 00006000 fd:05 165893 /usr/lib64/libOpenIPMIutils.so.0.0.1
4346:20140115:145032.355 7f8f1bc0d000-7f8f1bc0e000 rw-p 00007000 fd:05 165893 /usr/lib64/libOpenIPMIutils.so.0.0.1
4346:20140115:145032.355 7f8f1bc0e000-7f8f1bc10000 r-xp 00000000 fd:01 106599 /lib64/libkeyutils-1.2.so
4346:20140115:145032.355 7f8f1bc10000-7f8f1be0f000 ---p 00002000 fd:01 106599 /lib64/libkeyutils-1.2.so
4346:20140115:145032.355 7f8f1be0f000-7f8f1be10000 r--p 00001000 fd:01 106599 /lib64/libkeyutils-1.2.so
4346:20140115:145032.355 7f8f1be10000-7f8f1be11000 rw-p 00002000 fd:01 106599 /lib64/libkeyutils-1.2.so
4346:20140115:145032.355 7f8f1be11000-7f8f1be14000 r-xp 00000000 fd:01 106630 /lib64/libcom_err.so.2.1
4346:20140115:145032.355 7f8f1be14000-7f8f1c013000 ---p 00003000 fd:01 106630 /lib64/libcom_err.so.2.1
4346:20140115:145032.355 7f8f1c013000-7f8f1c014000 r--p 00002000 fd:01 106630 /lib64/libcom_err.so.2.1
4346:20140115:145032.355 7f8f1c014000-7f8f1c015000 rw-p 00003000 fd:01 106630 /lib64/libcom_err.so.2.1
4346:20140115:145032.355 7f8f1c015000-7f8f1c039000 r-xp 00000000 fd:05 134614 /usr/lib64/libk5crypto.so.3.1
4346:20140115:145032.355 7f8f1c039000-7f8f1c238000 ---p 00024000 fd:05 134614 /usr/lib64/libk5crypto.so.3.1
4346:20140115:145032.355 7f8f1c238000-7f8f1c23a000 r--p 00023000 fd:05 134614 /usr/lib64/libk5crypto.so.3.1
4346:20140115:145032.355 7f8f1c23a000-7f8f1c23b000 rw-p 00025000 fd:05 134614 /usr/lib64/libk5crypto.so.3.1
4346:20140115:145032.355 7f8f1c23b000-7f8f1c2d9000 r-xp 00000000 fd:05 134624 /usr/lib64/libkrb5.so.3.3
4346:20140115:145032.356 7f8f1c2d9000-7f8f1c4d8000 ---p 0009e000 fd:05 134624 /usr/lib64/libkrb5.so.3.3
4346:20140115:145032.356 7f8f1c4d8000-7f8f1c4db000 r--p 0009d000 fd:05 134624 /usr/lib64/libkrb5.so.3.3
4346:20140115:145032.356 7f8f1c4db000-7f8f1c4dd000 rw-p 000a0000 fd:05 134624 /usr/lib64/libkrb5.so.3.3
4346:20140115:145032.356 7f8f1c4dd000-7f8f1c508000 r-xp 00000000 fd:05 134610 /usr/lib64/libgssapi_krb5.so.2.2
4346:20140115:145032.356 7f8f1c508000-7f8f1c708000 ---p 0002b000 fd:05 134610 /usr/lib64/libgssapi_krb5.so.2.2
4346:20140115:145032.356 7f8f1c708000-7f8f1c709000 r--p 0002b000 fd:05 134610 /usr/lib64/libgssapi_krb5.so.2.2
4346:20140115:145032.356 7f8f1c709000-7f8f1c70a000 rw-p 0002c000 fd:05 134610 /usr/lib64/libgssapi_krb5.so.2.2
4346:20140115:145032.356 7f8f1c70a000-7f8f1c757000 r-xp 00000000 fd:05 134987 /usr/lib64/libssl.so.0.9.8
4346:20140115:145032.356 7f8f1c757000-7f8f1c956000 ---p 0004d000 fd:05 134987 /usr/lib64/libssl.so.0.9.8
4346:20140115:145032.356 7f8f1c956000-7f8f1c958000 r--p 0004c000 fd:05 134987 /usr/lib64/libssl.so.0.9.8
4346:20140115:145032.356 7f8f1c958000-7f8f1c95e000 rw-p 0004e000 fd:05 134987 /usr/lib64/libssl.so.0.9.8
4346:20140115:145032.356 7f8f1c95e000-7f8f1c98f000 r-xp 00000000 fd:05 134518 /usr/lib64/libidn.so.11.5.39
4346:20140115:145032.356 7f8f1c98f000-7f8f1cb8f000 ---p 00031000 fd:05 134518 /usr/lib64/libidn.so.11.5.39
4346:20140115:145032.356 7f8f1cb8f000-7f8f1cb90000 r--p 00031000 fd:05 134518 /usr/lib64/libidn.so.11.5.39
4346:20140115:145032.356 7f8f1cb90000-7f8f1cb91000 rw-p 00032000 fd:05 134518 /usr/lib64/libidn.so.11.5.39
4346:20140115:145032.356 7f8f1cb91000-7f8f1cba6000 r-xp 00000000 fd:01 106543 /lib64/libz.so.1.2.3
4346:20140115:145032.356 7f8f1cba6000-7f8f1cda5000 ---p 00015000 fd:01 106543 /lib64/libz.so.1.2.3
4346:20140115:145032.357 7f8f1cda5000-7f8f1cda6000 r--p 00014000 fd:01 106543 /lib64/libz.so.1.2.3
4346:20140115:145032.357 7f8f1cda6000-7f8f1cda7000 rw-p 00015000 fd:01 106543 /lib64/libz.so.1.2.3
4346:20140115:145032.357 7f8f1cda7000-7f8f1cdaa000 r-xp 00000000 fd:01 106624 /lib64/libgpg-error.so.0.8.0
4346:20140115:145032.357 7f8f1cdaa000-7f8f1cfa9000 ---p 00003000 fd:01 106624 /lib64/libgpg-error.so.0.8.0
4346:20140115:145032.357 7f8f1cfa9000-7f8f1cfaa000 r--p 00002000 fd:01 106624 /lib64/libgpg-error.so.0.8.0
4346:20140115:145032.357 7f8f1cfaa000-7f8f1cfab000 rw-p 00003000 fd:01 106624 /lib64/libgpg-error.so.0.8.0
4346:20140115:145032.357 7f8f1cfab000-7f8f1d024000 r-xp 00000000 fd:01 106751 /lib64/libgcrypt.so.11.7.0
4346:20140115:145032.357 7f8f1d024000-7f8f1d223000 ---p 00079000 fd:01 106751 /lib64/libgcrypt.so.11.7.0
4346:20140115:145032.357 7f8f1d223000-7f8f1d224000 r--p 00078000 fd:01 106751 /lib64/libgcrypt.so.11.7.0
4346:20140115:145032.357 7f8f1d224000-7f8f1d227000 rw-p 00079000 fd:01 106751 /lib64/libgcrypt.so.11.7.0
4346:20140115:145032.357 7f8f1d227000-7f8f1d2d5000 r-xp 00000000 fd:05 134993 /usr/lib64/libgnutls.so.26.4.5
4346:20140115:145032.357 7f8f1d2d5000-7f8f1d4d4000 ---p 000ae000 fd:05 134993 /usr/lib64/libgnutls.so.26.4.5
4346:20140115:145032.357 7f8f1d4d4000-7f8f1d4de000 r--p 000ad000 fd:05 134993 /usr/lib64/libgnutls.so.26.4.5
4346:20140115:145032.357 7f8f1d4de000-7f8f1d4e0000 rw-p 000b7000 fd:05 134993 /usr/lib64/libgnutls.so.26.4.5
4346:20140115:145032.357 7f8f1d4e0000-7f8f1d4f7000 r-xp 00000000 fd:01 106531 /lib64/libpthread-2.11.3.so
4346:20140115:145032.357 7f8f1d4f7000-7f8f1d6f7000 ---p 00017000 fd:01 106531 /lib64/libpthread-2.11.3.so
4346:20140115:145032.357 7f8f1d6f7000-7f8f1d6f8000 r--p 00017000 fd:01 106531 /lib64/libpthread-2.11.3.so
4346:20140115:145032.357 7f8f1d6f8000-7f8f1d6f9000 rw-p 00018000 fd:01 106531 /lib64/libpthread-2.11.3.so
4346:20140115:145032.357 7f8f1d6f9000-7f8f1d6fd000 rw-p 00000000 00:00 0
4346:20140115:145032.357 7f8f1d6fd000-7f8f1d86a000 r-xp 00000000 fd:01 106505 /lib64/libc-2.11.3.so
4346:20140115:145032.357 7f8f1d86a000-7f8f1da6a000 ---p 0016d000 fd:01 106505 /lib64/libc-2.11.3.so
4346:20140115:145032.358 7f8f1da6a000-7f8f1da6e000 r--p 0016d000 fd:01 106505 /lib64/libc-2.11.3.so
4346:20140115:145032.358 7f8f1da6e000-7f8f1da6f000 rw-p 00171000 fd:01 106505 /lib64/libc-2.11.3.so
4346:20140115:145032.358 7f8f1da6f000-7f8f1da74000 rw-p 00000000 00:00 0
4346:20140115:145032.358 7f8f1da74000-7f8f1da87000 r-xp 00000000 fd:01 106533 /lib64/libresolv-2.11.3.so
4346:20140115:145032.358 7f8f1da87000-7f8f1dc87000 ---p 00013000 fd:01 106533 /lib64/libresolv-2.11.3.so
4346:20140115:145032.358 7f8f1dc87000-7f8f1dc88000 r--p 00013000 fd:01 106533 /lib64/libresolv-2.11.3.so
4346:20140115:145032.358 7f8f1dc88000-7f8f1dc89000 rw-p 00014000 fd:01 106533 /lib64/libresolv-2.11.3.so
4346:20140115:145032.358 7f8f1dc89000-7f8f1dc8b000 rw-p 00000000 00:00 0
4346:20140115:145032.358 7f8f1dc8b000-7f8f1dc93000 r-xp 00000000 fd:01 106535 /lib64/librt-2.11.3.so
4346:20140115:145032.358 7f8f1dc93000-7f8f1de92000 ---p 00008000 fd:01 106535 /lib64/librt-2.11.3.so
4346:20140115:145032.358 7f8f1de92000-7f8f1de93000 r--p 00007000 fd:01 106535 /lib64/librt-2.11.3.so
4346:20140115:145032.358 7f8f1de93000-7f8f1de94000 rw-p 00008000 fd:01 106535 /lib64/librt-2.11.3.so
4346:20140115:145032.358 7f8f1de94000-7f8f1de96000 r-xp 00000000 fd:01 106511 /lib64/libdl-2.11.3.so
4346:20140115:145032.358 7f8f1de96000-7f8f1e096000 ---p 00002000 fd:01 106511 /lib64/libdl-2.11.3.so
4346:20140115:145032.358 7f8f1e096000-7f8f1e097000 r--p 00002000 fd:01 106511 /lib64/libdl-2.11.3.so
4346:20140115:145032.358 7f8f1e097000-7f8f1e098000 rw-p 00003000 fd:01 106511 /lib64/libdl-2.11.3.so
4346:20140115:145032.358 7f8f1e098000-7f8f1e0f3000 r-xp 00000000 fd:01 106513 /lib64/libm-2.11.3.so
4346:20140115:145032.358 7f8f1e0f3000-7f8f1e2f2000 ---p 0005b000 fd:01 106513 /lib64/libm-2.11.3.so
4346:20140115:145032.358 7f8f1e2f2000-7f8f1e2f3000 r--p 0005a000 fd:01 106513 /lib64/libm-2.11.3.so
4346:20140115:145032.358 7f8f1e2f3000-7f8f1e311000 rw-p 0005b000 fd:01 106513 /lib64/libm-2.11.3.so
4346:20140115:145032.358 7f8f1e311000-7f8f1e31f000 r-xp 00000000 fd:05 134694 /usr/lib64/liblber-2.4.so.2.7.1
4346:20140115:145032.358 7f8f1e31f000-7f8f1e51e000 ---p 0000e000 fd:05 134694 /usr/lib64/liblber-2.4.so.2.7.1
4346:20140115:145032.358 7f8f1e51e000-7f8f1e51f000 r--p 0000d000 fd:05 134694 /usr/lib64/liblber-2.4.so.2.7.1
4346:20140115:145032.358 7f8f1e51f000-7f8f1e520000 rw-p 0000e000 fd:05 134694 /usr/lib64/liblber-2.4.so.2.7.1
4346:20140115:145032.359 7f8f1e520000-7f8f1e566000 r-xp 00000000 fd:05 134696 /usr/lib64/libldap-2.4.so.2.7.1
4346:20140115:145032.359 7f8f1e566000-7f8f1e766000 ---p 00046000 fd:05 134696 /usr/lib64/libldap-2.4.so.2.7.1
4346:20140115:145032.359 7f8f1e766000-7f8f1e767000 r--p 00046000 fd:05 134696 /usr/lib64/libldap-2.4.so.2.7.1
4346:20140115:145032.359 7f8f1e767000-7f8f1e769000 rw-p 00047000 fd:05 134696 /usr/lib64/libldap-2.4.so.2.7.1
4346:20140115:145032.359 7f8f1e769000-7f8f1e76e000 r-xp 00000000 fd:05 165887 /usr/lib64/libOpenIPMIposix.so.0.0.1
4346:20140115:145032.359 7f8f1e76e000-7f8f1e96d000 ---p 00005000 fd:05 165887 /usr/lib64/libOpenIPMIposix.so.0.0.1
4346:20140115:145032.359 7f8f1e96d000-7f8f1e96e000 r--p 00004000 fd:05 165887 /usr/lib64/libOpenIPMIposix.so.0.0.1
4346:20140115:145032.359 7f8f1e96e000-7f8f1e96f000 rw-p 00005000 fd:05 165887 /usr/lib64/libOpenIPMIposix.so.0.0.1
4346:20140115:145032.359 7f8f1e96f000-7f8f1ea56000 r-xp 00000000 fd:05 165881 /usr/lib64/libOpenIPMI.so.0.0.5
4346:20140115:145032.359 7f8f1ea56000-7f8f1ec55000 ---p 000e7000 fd:05 165881 /usr/lib64/libOpenIPMI.so.0.0.5
4346:20140115:145032.359 7f8f1ec55000-7f8f1ec6b000 r--p 000e6000 fd:05 165881 /usr/lib64/libOpenIPMI.so.0.0.5
4346:20140115:145032.359 7f8f1ec6b000-7f8f1ec71000 rw-p 000fc000 fd:05 165881 /usr/lib64/libOpenIPMI.so.0.0.5
4346:20140115:145032.359 7f8f1ec71000-7f8f1ec75000 rw-p 00000000 00:00 0
4346:20140115:145032.359 7f8f1ec75000-7f8f1ede8000 r-xp 00000000 fd:05 131423 /usr/lib64/libcrypto.so.0.9.8
4346:20140115:145032.359 7f8f1ede8000-7f8f1efe7000 ---p 00173000 fd:05 131423 /usr/lib64/libcrypto.so.0.9.8
4346:20140115:145032.359 7f8f1efe7000-7f8f1eff7000 r--p 00172000 fd:05 131423 /usr/lib64/libcrypto.so.0.9.8
4346:20140115:145032.359 7f8f1eff7000-7f8f1f010000 rw-p 00182000 fd:05 131423 /usr/lib64/libcrypto.so.0.9.8
4346:20140115:145032.359 7f8f1f010000-7f8f1f014000 rw-p 00000000 00:00 0
4346:20140115:145032.359 7f8f1f014000-7f8f1f0b7000 r-xp 00000000 fd:05 135122 /usr/lib64/libnetsnmp.so.15.1.2
4346:20140115:145032.359 7f8f1f0b7000-7f8f1f2b7000 ---p 000a3000 fd:05 135122 /usr/lib64/libnetsnmp.so.15.1.2
4346:20140115:145032.359 7f8f1f2b7000-7f8f1f2b8000 r--p 000a3000 fd:05 135122 /usr/lib64/libnetsnmp.so.15.1.2
4346:20140115:145032.359 7f8f1f2b8000-7f8f1f2bb000 rw-p 000a4000 fd:05 135122 /usr/lib64/libnetsnmp.so.15.1.2
4346:20140115:145032.359 7f8f1f2bb000-7f8f1f2f0000 rw-p 00000000 00:00 0
4346:20140115:145032.359 7f8f1f2f0000-7f8f1f336000 r-xp 00000000 fd:05 134738 /usr/lib64/libcurl.so.4.1.1
4346:20140115:145032.359 7f8f1f336000-7f8f1f535000 ---p 00046000 fd:05 134738 /usr/lib64/libcurl.so.4.1.1
4346:20140115:145032.359 7f8f1f535000-7f8f1f536000 r--p 00045000 fd:05 134738 /usr/lib64/libcurl.so.4.1.1
4346:20140115:145032.359 7f8f1f536000-7f8f1f537000 rw-p 00046000 fd:05 134738 /usr/lib64/libcurl.so.4.1.1
4346:20140115:145032.359 7f8f1f537000-7f8f1f689000 r-xp 00000000 fd:05 131365 /usr/lib64/libxml2.so.2.7.6
4346:20140115:145032.359 7f8f1f689000-7f8f1f888000 ---p 00152000 fd:05 131365 /usr/lib64/libxml2.so.2.7.6
4346:20140115:145032.359 7f8f1f888000-7f8f1f890000 r--p 00151000 fd:05 131365 /usr/lib64/libxml2.so.2.7.6
4346:20140115:145032.359 7f8f1f890000-7f8f1f892000 rw-p 00159000 fd:05 131365 /usr/lib64/libxml2.so.2.7.6
4346:20140115:145032.359 7f8f1f892000-7f8f1f893000 rw-p 00000000 00:00 0
4346:20140115:145032.359 7f8f1f893000-7f8f1f8a0000 r-xp 00000000 fd:05 134470 /usr/lib64/libiksemel.so.3.1.1
4346:20140115:145032.359 7f8f1f8a0000-7f8f1fa9f000 ---p 0000d000 fd:05 134470 /usr/lib64/libiksemel.so.3.1.1
4346:20140115:145032.360 7f8f1fa9f000-7f8f1faa0000 r--p 0000c000 fd:05 134470 /usr/lib64/libiksemel.so.3.1.1
4346:20140115:145032.360 7f8f1faa0000-7f8f1faa1000 rw-p 0000d000 fd:05 134470 /usr/lib64/libiksemel.so.3.1.1
4346:20140115:145032.360 7f8f1faa1000-7f8f1fd7c000 r-xp 00000000 fd:05 145587 /usr/lib64/libmysqlclient.so.18.0.0
4346:20140115:145032.360 7f8f1fd7c000-7f8f1ff7c000 ---p 002db000 fd:05 145587 /usr/lib64/libmysqlclient.so.18.0.0
4346:20140115:145032.360 7f8f1ff7c000-7f8f1ff81000 r--p 002db000 fd:05 145587 /usr/lib64/libmysqlclient.so.18.0.0
4346:20140115:145032.360 7f8f1ff81000-7f8f20000000 rw-p 002e0000 fd:05 145587 /usr/lib64/libmysqlclient.so.18.0.0
4346:20140115:145032.360 7f8f20000000-7f8f20005000 rw-p 00000000 00:00 0
4346:20140115:145032.360 7f8f20005000-7f8f20024000 r-xp 00000000 fd:01 106782 /lib64/ld-2.11.3.so
4346:20140115:145032.360 7f8f201e1000-7f8f201f0000 rw-p 00000000 00:00 0
4346:20140115:145032.360 7f8f20220000-7f8f20221000 rw-p 00000000 00:00 0
4346:20140115:145032.360 7f8f20221000-7f8f20222000 rw-p 00000000 00:00 0
4346:20140115:145032.360 7f8f20222000-7f8f20223000 rw-p 00000000 00:00 0
4346:20140115:145032.360 7f8f20223000-7f8f20224000 r--p 0001e000 fd:01 106782 /lib64/ld-2.11.3.so
4346:20140115:145032.360 7f8f20224000-7f8f20225000 rw-p 0001f000 fd:01 106782 /lib64/ld-2.11.3.so
4346:20140115:145032.360 7f8f20225000-7f8f20226000 rw-p 00000000 00:00 0
4346:20140115:145032.360 7fffe5f01000-7fffe5f32000 rw-p 00000000 00:00 0 [stack]
4346:20140115:145032.360 7fffe5fff000-7fffe6000000 r-xp 00000000 00:00 0 [vdso]
4346:20140115:145032.360 ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall]
4346:20140115:145032.360 ================================
4346:20140115:145032.360 Please consider attaching a disassembly listing to your bug report.
4346:20140115:145032.360 This listing can be produced with, e.g., objdump -DSswx zabbix_server.
4346:20140115:145032.360 ================================
4330:20140115:145032.362 One child process died (PID:4346,exitcode/signal:255). Exiting ...
4330:20140115:145034.366 syncing history data...
4330:20140115:145034.369 syncing history data done
4330:20140115:145034.369 syncing trends data...
4330:20140115:145034.969 syncing trends data done
4330:20140115:145034.969 Zabbix Server stopped. Zabbix 2.2.1 (revision 40808).



 Comments   
Comment by Johannes [ 2014 Jan 15 ]

I have just updated to Pre-2.2.2rc1 (stable) Rev. 41531 and will report any problems

related to https://support.zabbix.com/browse/ZBX-7521 ??

Comment by Oleksii Zagorskyi [ 2014 Jan 16 ]

it's not related to the ZBX-7521 (crash in "history syncer")

Comment by Oleksii Zagorskyi [ 2014 Jan 16 ]

Did you start to use vmware monitoring exactly yesterday (backtrace above have references to specific functions) ?

Could you share with any specific details if you have them ?

Comment by Johannes [ 2014 Jan 16 ]

yes we started to use VMware monitoring but this is working like a charm.
everything was discovered correctly.

what I saw in zabbix_server.log before the crash was this:

28844:20140115:165031.439 [Z3005] query failed: [1364] Field 'hardware_full' doesn't have a default value [insert into host_inventory (hostid,inventory_mode) values (100100000010978,1),(100100000010979,1),(100100000010980,1),(100100000010981,1),(100100000010982,1),(100100000010983,1),(100100000010984,1),(100100000010985,1),(100100000010986,1),(100100000010987,1),(100100000010988,1),(100100000010989,1),(100100000010990,1),(100100000010991,1),(100100000010992,1),(100100000010993,1),(100100000010994,1),(100100000010995,1),(100100000010996,1),(100100000010997,1),(100100000010998,1),(100100000010999,1),(100100000011000,1),(100100000011001,1),(100100000011002,1),(100100000011003,1),(100100000011004,1),(100100000011005,1),(100100000011006,1),(100100000011007,1),(100100000011008,1),(100100000011009,1),(100100000011010,1),(100100000011011,1),(100100000011012,1),(100100000011013,1),(100100000011014,1),(100100000011015,1),(100100000011016,1),(100100000011017,1),(100100000011018,1),(100100000011019,1),(100100000011020,1),(100100000011021,1),(100100000011022,1),(100100000011023,1),(100100000011024,1),(100100000011025,1),(100100000011026,1),(100100000011027,1),(100100000011028,1),(100100000011029,1),(100100000011030,1),(100100000011031,1),(100100000011032,1),(100100000011033,1),(100100000011034,1),(100100000011035,1),(100100000011036,1),(100100000011037,1),(100100000011038,1),(100100000011039,1),(100100000011040,1),(100100000011041,1),(100100000011042,1),(100100000011043,1),(100100000011044,1),(100100000011045,1),(100100000011046,1),(100100000011047,1),(100100000011048,1),(100100000011049,1),(100100000011050,1),(100100000011051,1),(100100000011052,1),(100100000011053,1),(100100000011054,1),(100100000011055,1),(100100000011056,1),(100100000011057,1),(100100000011058,1);]

28844:20140115:165031.457 Got signal [signal:7(SIGBUS),reason:128,refaddr:(nil)]. Crashing ...

What I did is setting the sql_mode of our MySQL server to "" which means that strict mode is disabled now.
Since this change no more crashes!

MySQL version: mysql Ver 14.14 Distrib 5.5.29, for Linux (x86_64)

mysql> SELECT @@GLOBAL.sql_mode;
-------------------

@@GLOBAL.sql_mode

-------------------

 

-------------------
1 row in set (0.02 sec)

Comment by Johannes [ 2014 Jan 16 ]

maybe related to: https://support.zabbix.com/browse/ZBX-7561

EDIT:

When I imported the templates the first time and let it discover I changed nothing and it was working, but I deleted and reimported the templates again and changed the following:

  • Template Virt Vmware
    -> Discovery Rules
    -> Discover VMware VMs
    -> Host prototypes
    -> {#VM.NAME}

    -> and changed the "Host Inventory" to "Automatic" (this is disabled after importing the default template)

Comment by Alexander Vladishev [ 2014 Jan 20 ]

Related issue: ZBX-7603

Comment by Andris Zeila [ 2014 Jan 21 ]

The crash happens when trying to link template to a host without any interfaces. While normally it can't happen, but apparently in this case host saving failed because of "28844:20140115:165031.439 [Z3005] query failed: [1364] Field 'hardware_full' doesn't have a default value [insert into host_inventory (hostid,inventory_mode) values..." error, leading to following crash when trying to link templates.

We have to decide how to handle it.

Comment by Juris Miščenko (Inactive) [ 2014 Feb 18 ]

Fixed in svn://svn.zabbix.com/branches/dev/ZBX-7659

Upon attempting to link a template to a host that has no interfaces a log message will be presented that will inform of the event. The crash has been averted.

Comment by Alexander Vladishev [ 2014 Feb 26 ]

"28844:20140115:165031.439 [Z3005] query failed: [1364] Field 'hardware_full' doesn't have a default value [insert into host_inventory (hostid,inventory_mode) values..." error was fixed under ZBX-7561.

Comment by Juris Miščenko (Inactive) [ 2014 Mar 06 ]

Fixed in 2.0.12rc1 r43259, 2.2.3rc1 r43240, 2.3.0 (trunk) r43258

Comment by Juris Miščenko (Inactive) [ 2014 Mar 06 ]

To the reporter: could you please explain to me how you managed to trigger this crash, because during reproducing/testing we had to introduce severe breakage in the code just to get this to happen. Was it a database corruption of some sort? It would be nice to know

Comment by Johannes [ 2014 Mar 31 ]

Hi, imported vmware templates, added the vcenter server and changed the inventory to automatic.
vmware discovery starts and after the while the zabbix server process stops with:

6901:20140331:135135.034 [Z3005] query failed: [1364] Field 'hardware_full' doesn't have a default value [insert into host_inventory (hostid,inventory_mode) values (200200000014422,1),(200200000014423,1),(200200000014424,1),(200200000014425,1),(200200000014426,1),(200200000014427,1),(200200000014428,1),(200200000014429,1),(200200000014430,1),(200200000014431,1),(200200000014432,1),(200200000014433,1),(200200000014434,1),(200200000014435,1),(200200000014436,1),(200200000014437,1),(200200000014438,1),(200200000014439,1),(200200000014440,1),(200200000014441,1),(200200000014442,1),(200200000014443,1),(200200000014444,1),(200200000014445,1),(200200000014446,1),(200200000014447,1),(200200000014448,1);]

Mysql version currently 5.5 but we had to set sql_mode to ' ' (which means no_strict mode) and it does not happen anymore with automatic inventory enabled.

Generated at Sat Apr 27 01:34:47 EEST 2024 using Jira 9.12.4#9120004-sha1:625303b708afdb767e17cb2838290c41888e9ff0.