Uploaded image for project: 'ZABBIX BUGS AND ISSUES'
  1. ZABBIX BUGS AND ISSUES
  2. ZBX-9218

agent crash, Got signal [signal:11(SIGSEGV),reason:1,refaddr:(nil)]

    Details

    • Type: Incident report
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.4.3
    • Fix Version/s: 2.4.4rc1, 2.5.0
    • Component/s: Agent (G)
    • Environment:
      CentOS 6.5 X64, 64G Memory, Dell R620

      Description

      i have write one module for collecting mysql information, named mysql.so, parse it's config file /etc/zabbix/module_mysql.conf.

      1. when i use zabbix-2.2.5, it works fine
      2. when i upgrade to 2.4.3, it always crash when the mysql server stop or the user don't have privileges. (when return SYSINFO_RET_FAIL)

      log:

       10723:20150115:163242.473 Starting Zabbix Agent [172.16.20.191]. Zabbix 2.4.3 (revision 51175).
       10723:20150115:163242.473 using configuration file: /etc/zabbix/zabbix_agentd.conf
       10723:20150115:163242.483 module [mysql], func [zbx_module_init], using configuration file: [/etc/zabbix/module_mysql.conf]
       10723:20150115:163242.483 loaded modules: mysql.so
       10723:20150115:163242.484 agent #0 started [main process]
       10725:20150115:163242.484 agent #1 started [collector]
       10727:20150115:163242.484 agent #3 started [listener #2]
       10728:20150115:163242.485 agent #4 started [listener #3]
       10729:20150115:163242.485 agent #5 started [listener #4]
       10730:20150115:163242.485 agent #6 started [listener #5]
       10726:20150115:163242.485 agent #2 started [listener #1]
       10731:20150115:163242.485 agent #7 started [active checks #1]
       10731:20150115:163242.804 module [mysql], func [zbx_module_mysql_discovery], CONFIG_MYSQL_INSTANCE_PORT value is: [3306,3307]
       10731:20150115:163242.804 module [mysql], func [zbx_module_mysql_discovery], add port [3306] to discovery list
       10731:20150115:163242.804 module [mysql], func [zbx_module_mysql_discovery], add port [3307] to discovery list
       10731:20150115:163242.804 module [mysql], func [zbx_module_mysql_conn], connect mysql server [127.0.0.1:3306] using user [zabbix_agent] fail: [Can't connect to MySQL server on '127.0.0.1' (111)]
       10731:20150115:163242.806 module [mysql], func [zbx_module_mysql_conn], connect mysql server [127.0.0.1:3307] using user [zabbix_agent] fail: [Access denied for user 'zabbix_agent'@'localhost' (using password: YES)]
       10731:20150115:163242.806 module [mysql], func [zbx_module_mysql_conn], connect mysql server [127.0.0.1:3306] using user [zabbix_agent] fail: [Can't connect to MySQL server on '127.0.0.1' (111)]
       10731:20150115:163242.806 active check "mysql.status[127.0.0.1,3306,Bytes_received]" is not supported
       10731:20150115:163242.806 Got signal [signal:11(SIGSEGV),reason:1,refaddr:(nil)]. Crashing ...
       10731:20150115:163242.806 ====== Fatal information: ======
       10731:20150115:163242.806 Program counter: 0x40bebc
       10731:20150115:163242.806 === Registers: ===
       10731:20150115:163242.806 r8      =     7f96477d5700 =      140283421218560 =      140283421218560
       10731:20150115:163242.806 r9      =                1 =                    1 =                    1
       10731:20150115:163242.806 r10     = 657479422c363033 =  7310601420245708851 =  7310601420245708851
       10731:20150115:163242.806 r11     =              206 =                  518 =                  518
       10731:20150115:163242.806 r12     =         54b77b2a =           1421310762 =           1421310762
       10731:20150115:163242.806 r13     =                0 =                    0 =                    0
       10731:20150115:163242.806 r14     =     7fff22d7ce70 =      140733777956464 =      140733777956464
       10731:20150115:163242.806 r15     =          2217900 =             35748096 =             35748096
       10731:20150115:163242.806 rdi     =           43a10f =              4432143 =              4432143
       10731:20150115:163242.806 rsi     =              172 =                  370 =                  370
       10731:20150115:163242.806 rbp     =              540 =                 1344 =                 1344
       10731:20150115:163242.806 rbx     =              580 =                 1408 =                 1408
       10731:20150115:163242.806 rdx     =           648920 =              6588704 =              6588704
       10731:20150115:163242.806 rax     =          2228e40 =             35819072 =             35819072
       10731:20150115:163242.806 rcx     =          2225770 =             35805040 =             35805040
       10731:20150115:163242.806 rsp     =     7fff22d7ade0 =      140733777948128 =      140733777948128
       10731:20150115:163242.806 rip     =           40bebc =              4243132 =              4243132
       10731:20150115:163242.806 efl     =            10206 =                66054 =                66054
       10731:20150115:163242.806 csgsfs  = 676f000000000033 =  7453175908321460275 =  7453175908321460275
       10731:20150115:163242.806 err     =                4 =                    4 =                    4
       10731:20150115:163242.806 trapno  =                e =                   14 =                   14
       10731:20150115:163242.806 oldmask =                0 =                    0 =                    0
       10731:20150115:163242.806 cr2     =                0 =                    0 =                    0
       10731:20150115:163242.806 === Backtrace: ===
       10731:20150115:163242.807 10: zabbix_agentd: active checks #1 [processing active checks](print_fatal_info+0xa6) [0x425796]
       10731:20150115:163242.807 9: zabbix_agentd: active checks #1 [processing active checks]() [0x425e46]
       10731:20150115:163242.807 8: /lib64/libc.so.6(+0x326a0) [0x7f96469b76a0]
       10731:20150115:163242.807 7: zabbix_agentd: active checks #1 [processing active checks]() [0x40bebc]
       10731:20150115:163242.807 6: zabbix_agentd: active checks #1 [processing active checks](active_checks_thread+0x1ed) [0x40c61d]
       10731:20150115:163242.807 5: zabbix_agentd: active checks #1 [processing active checks](zbx_thread_start+0x62) [0x4243f2]
       10731:20150115:163242.807 4: zabbix_agentd: active checks #1 [processing active checks](MAIN_ZABBIX_ENTRY+0x2d9) [0x4118c9]
       10731:20150115:163242.807 3: zabbix_agentd: active checks #1 [processing active checks](daemon_start+0x18c) [0x424a9c]
       10731:20150115:163242.807 2: zabbix_agentd: active checks #1 [processing active checks](main+0x197) [0x4113a7]
       10731:20150115:163242.807 1: /lib64/libc.so.6(__libc_start_main+0xfd) [0x7f96469a3d5d]
       10731:20150115:163242.807 0: zabbix_agentd: active checks #1 [processing active checks]() [0x40a319]
       10731:20150115:163242.807 === Memory map: ===
       10731:20150115:163242.807 00400000-00446000 r-xp 00000000 fd:00 132325                             /usr/sbin/zabbix_agentd
       10731:20150115:163242.807 00646000-00648000 rw-p 00046000 fd:00 132325                             /usr/sbin/zabbix_agentd
       10731:20150115:163242.807 00648000-0064d000 rw-p 00000000 00:00 0 
       10731:20150115:163242.807 02215000-02236000 rw-p 00000000 00:00 0 
       10731:20150115:163242.807 02236000-02257000 rw-p 00000000 00:00 0 
       10731:20150115:163242.807 7f9643f07000-7f9643f1d000 r-xp 00000000 fd:00 791467                     /lib64/libgcc_s-4.4.7-20120601.so.1
       10731:20150115:163242.807 7f9643f1d000-7f964411c000 ---p 00016000 fd:00 791467                     /lib64/libgcc_s-4.4.7-20120601.so.1
       10731:20150115:163242.807 7f964411c000-7f964411d000 rw-p 00015000 fd:00 791467                     /lib64/libgcc_s-4.4.7-20120601.so.1
       10731:20150115:163242.807 7f964411d000-7f964413a000 r-xp 00000000 fd:00 791469                     /lib64/libselinux.so.1
       10731:20150115:163242.807 7f964413a000-7f9644339000 ---p 0001d000 fd:00 791469                     /lib64/libselinux.so.1
       10731:20150115:163242.807 7f9644339000-7f964433a000 r--p 0001c000 fd:00 791469                     /lib64/libselinux.so.1
       10731:20150115:163242.807 7f964433a000-7f964433b000 rw-p 0001d000 fd:00 791469                     /lib64/libselinux.so.1
       10731:20150115:163242.807 7f964433b000-7f964433c000 rw-p 00000000 00:00 0 
       10731:20150115:163242.807 7f964433c000-7f964433e000 r-xp 00000000 fd:00 786895                     /lib64/libkeyutils.so.1.3
       10731:20150115:163242.807 7f964433e000-7f964453d000 ---p 00002000 fd:00 786895                     /lib64/libkeyutils.so.1.3
       10731:20150115:163242.807 7f964453d000-7f964453e000 r--p 00001000 fd:00 786895                     /lib64/libkeyutils.so.1.3
       10731:20150115:163242.807 7f964453e000-7f964453f000 rw-p 00002000 fd:00 786895                     /lib64/libkeyutils.so.1.3
       10731:20150115:163242.807 7f964453f000-7f9644549000 r-xp 00000000 fd:00 790311                     /lib64/libkrb5support.so.0.1
       10731:20150115:163242.807 7f9644549000-7f9644748000 ---p 0000a000 fd:00 790311                     /lib64/libkrb5support.so.0.1
       10731:20150115:163242.807 7f9644748000-7f9644749000 r--p 00009000 fd:00 790311                     /lib64/libkrb5support.so.0.1
       10731:20150115:163242.807 7f9644749000-7f964474a000 rw-p 0000a000 fd:00 790311                     /lib64/libkrb5support.so.0.1
       10731:20150115:163242.807 7f964474a000-7f9644773000 r-xp 00000000 fd:00 786900                     /lib64/libk5crypto.so.3.1
       10731:20150115:163242.807 7f9644773000-7f9644973000 ---p 00029000 fd:00 786900                     /lib64/libk5crypto.so.3.1
       10731:20150115:163242.807 7f9644973000-7f9644974000 r--p 00029000 fd:00 786900                     /lib64/libk5crypto.so.3.1
       10731:20150115:163242.807 7f9644974000-7f9644975000 rw-p 0002a000 fd:00 786900                     /lib64/libk5crypto.so.3.1
       10731:20150115:163242.807 7f9644975000-7f9644976000 rw-p 00000000 00:00 0 
       10731:20150115:163242.807 7f9644976000-7f9644979000 r-xp 00000000 fd:00 790305                     /lib64/libcom_err.so.2.1
       10731:20150115:163242.807 7f9644979000-7f9644b78000 ---p 00003000 fd:00 790305                     /lib64/libcom_err.so.2.1
       10731:20150115:163242.807 7f9644b78000-7f9644b79000 r--p 00002000 fd:00 790305                     /lib64/libcom_err.so.2.1
       10731:20150115:163242.807 7f9644b79000-7f9644b7a000 rw-p 00003000 fd:00 790305                     /lib64/libcom_err.so.2.1
       10731:20150115:163242.807 7f9644b7a000-7f9644c55000 r-xp 00000000 fd:00 790310                     /lib64/libkrb5.so.3.3
       10731:20150115:163242.808 7f9644c55000-7f9644e54000 ---p 000db000 fd:00 790310                     /lib64/libkrb5.so.3.3
       10731:20150115:163242.808 7f9644e54000-7f9644e5e000 r--p 000da000 fd:00 790310                     /lib64/libkrb5.so.3.3
       10731:20150115:163242.808 7f9644e5e000-7f9644e60000 rw-p 000e4000 fd:00 790310                     /lib64/libkrb5.so.3.3
       10731:20150115:163242.808 7f9644e60000-7f9644ea1000 r-xp 00000000 fd:00 790308                     /lib64/libgssapi_krb5.so.2.2
       10731:20150115:163242.808 7f9644ea1000-7f96450a1000 ---p 00041000 fd:00 790308                     /lib64/libgssapi_krb5.so.2.2
       10731:20150115:163242.808 7f96450a1000-7f96450a2000 r--p 00041000 fd:00 790308                     /lib64/libgssapi_krb5.so.2.2
       10731:20150115:163242.808 7f96450a2000-7f96450a4000 rw-p 00042000 fd:00 790308                     /lib64/libgssapi_krb5.so.2.2
       10731:20150115:163242.808 7f96450a4000-7f9645117000 r-xp 00000000 fd:00 786703                     /lib64/libfreebl3.so
       10731:20150115:163242.808 7f9645117000-7f9645316000 ---p 00073000 fd:00 786703                     /lib64/libfreebl3.so
       10731:20150115:163242.808 7f9645316000-7f9645318000 r--p 00072000 fd:00 786703                     /lib64/libfreebl3.so
       10731:20150115:163242.808 7f9645318000-7f9645319000 rw-p 00074000 fd:00 786703                     /lib64/libfreebl3.so
       10731:20150115:163242.808 7f9645319000-7f964531d000 rw-p 00000000 00:00 0 
       10731:20150115:163242.808 7f964531d000-7f9645332000 r-xp 00000000 fd:00 786762                     /lib64/libz.so.1.2.3
       10731:20150115:163242.808 7f9645332000-7f9645531000 ---p 00015000 fd:00 786762                     /lib64/libz.so.1.2.3
       10731:20150115:163242.808 7f9645531000-7f9645532000 r--p 00014000 fd:00 786762                     /lib64/libz.so.1.2.3
       10731:20150115:163242.808 7f9645532000-7f9645533000 rw-p 00015000 fd:00 786762                     /lib64/libz.so.1.2.3
       10731:20150115:163242.808 7f9645533000-7f96456eb000 r-xp 00000000 fd:00 141004                     /usr/lib64/libcrypto.so.1.0.1e
       10731:20150115:163242.808 7f96456eb000-7f96458eb000 ---p 001b8000 fd:00 141004                     /usr/lib64/libcrypto.so.1.0.1e
       10731:20150115:163242.808 7f96458eb000-7f9645906000 r--p 001b8000 fd:00 141004                     /usr/lib64/libcrypto.so.1.0.1e
       10731:20150115:163242.808 7f9645906000-7f9645912000 rw-p 001d3000 fd:00 141004                     /usr/lib64/libcrypto.so.1.0.1e
       10731:20150115:163242.808 7f9645912000-7f9645916000 rw-p 00000000 00:00 0 
       10731:20150115:163242.808 7f9645916000-7f9645978000 r-xp 00000000 fd:00 141005                     /usr/lib64/libssl.so.1.0.1e
       10731:20150115:163242.808 7f9645978000-7f9645b77000 ---p 00062000 fd:00 141005                     /usr/lib64/libssl.so.1.0.1e
       10731:20150115:163242.808 7f9645b77000-7f9645b7b000 r--p 00061000 fd:00 141005                     /usr/lib64/libssl.so.1.0.1e
       10731:20150115:163242.808 7f9645b7b000-7f9645b82000 rw-p 00065000 fd:00 141005                     /usr/lib64/libssl.so.1.0.1e
       10731:20150115:163242.808 7f9645b82000-7f9645b98000 r-xp 00000000 fd:00 790275                     /lib64/libnsl-2.12.so
       10731:20150115:163242.808 7f9645b98000-7f9645d97000 ---p 00016000 fd:00 790275                     /lib64/libnsl-2.12.so
       10731:20150115:163242.808 7f9645d97000-7f9645d98000 r--p 00015000 fd:00 790275                     /lib64/libnsl-2.12.so
       10731:20150115:163242.808 7f9645d98000-7f9645d99000 rw-p 00016000 fd:00 790275                     /lib64/libnsl-2.12.so
       10731:20150115:163242.808 7f9645d99000-7f9645d9b000 rw-p 00000000 00:00 0 
       10731:20150115:163242.808 7f9645d9b000-7f9645da2000 r-xp 00000000 fd:00 786714                     /lib64/libcrypt-2.12.so
       10731:20150115:163242.808 7f9645da2000-7f9645fa2000 ---p 00007000 fd:00 786714                     /lib64/libcrypt-2.12.so
       10731:20150115:163242.808 7f9645fa2000-7f9645fa3000 r--p 00007000 fd:00 786714                     /lib64/libcrypt-2.12.so
       10731:20150115:163242.808 7f9645fa3000-7f9645fa4000 rw-p 00008000 fd:00 786714                     /lib64/libcrypt-2.12.so
       10731:20150115:163242.808 7f9645fa4000-7f9645fd2000 rw-p 00000000 00:00 0 
       10731:20150115:163242.808 7f9645fd2000-7f9646108000 r-xp 00000000 fd:00 134180                     /usr/lib64/mysql/libmysqlclient.so.16.0.0
       10731:20150115:163242.808 7f9646108000-7f9646307000 ---p 00136000 fd:00 134180                     /usr/lib64/mysql/libmysqlclient.so.16.0.0
       10731:20150115:163242.808 7f9646307000-7f9646355000 rw-p 00135000 fd:00 134180                     /usr/lib64/mysql/libmysqlclient.so.16.0.0
       10731:20150115:163242.808 7f9646355000-7f9646356000 rw-p 00000000 00:00 0 
       10731:20150115:163242.808 7f9646356000-7f964635a000 r-xp 00000000 fd:00 918314                     /etc/zabbix/modules/mysql.so
       10731:20150115:163242.808 7f964635a000-7f9646559000 ---p 00004000 fd:00 918314                     /etc/zabbix/modules/mysql.so
       10731:20150115:163242.808 7f9646559000-7f964655a000 rw-p 00003000 fd:00 918314                     /etc/zabbix/modules/mysql.so
       10731:20150115:163242.808 7f964655a000-7f9646566000 r-xp 00000000 fd:00 790277                     /lib64/libnss_files-2.12.so
       10731:20150115:163242.808 7f9646566000-7f9646766000 ---p 0000c000 fd:00 790277                     /lib64/libnss_files-2.12.so
       10731:20150115:163242.808 7f9646766000-7f9646767000 r--p 0000c000 fd:00 790277                     /lib64/libnss_files-2.12.so
       10731:20150115:163242.808 7f9646767000-7f9646768000 rw-p 0000d000 fd:00 790277                     /lib64/libnss_files-2.12.so
       10731:20150115:163242.808 7f9646768000-7f964677f000 r-xp 00000000 fd:00 786734                     /lib64/libpthread-2.12.so
       10731:20150115:163242.808 7f964677f000-7f964697f000 ---p 00017000 fd:00 786734                     /lib64/libpthread-2.12.so
       10731:20150115:163242.809 7f964697f000-7f9646980000 r--p 00017000 fd:00 786734                     /lib64/libpthread-2.12.so
       10731:20150115:163242.809 7f9646980000-7f9646981000 rw-p 00018000 fd:00 786734                     /lib64/libpthread-2.12.so
       10731:20150115:163242.809 7f9646981000-7f9646985000 rw-p 00000000 00:00 0 
       10731:20150115:163242.809 7f9646985000-7f9646b0f000 r-xp 00000000 fd:00 786710                     /lib64/libc-2.12.so
       10731:20150115:163242.809 7f9646b0f000-7f9646d0f000 ---p 0018a000 fd:00 786710                     /lib64/libc-2.12.so
       10731:20150115:163242.809 7f9646d0f000-7f9646d13000 r--p 0018a000 fd:00 786710                     /lib64/libc-2.12.so
       10731:20150115:163242.809 7f9646d13000-7f9646d14000 rw-p 0018e000 fd:00 786710                     /lib64/libc-2.12.so
       10731:20150115:163242.809 7f9646d14000-7f9646d19000 rw-p 00000000 00:00 0 
       10731:20150115:163242.809 7f9646d19000-7f9646d2f000 r-xp 00000000 fd:00 790279                     /lib64/libresolv-2.12.so
       10731:20150115:163242.809 7f9646d2f000-7f9646f2f000 ---p 00016000 fd:00 790279                     /lib64/libresolv-2.12.so
       10731:20150115:163242.809 7f9646f2f000-7f9646f30000 r--p 00016000 fd:00 790279                     /lib64/libresolv-2.12.so
       10731:20150115:163242.809 7f9646f30000-7f9646f31000 rw-p 00017000 fd:00 790279                     /lib64/libresolv-2.12.so
       10731:20150115:163242.809 7f9646f31000-7f9646f33000 rw-p 00000000 00:00 0 
       10731:20150115:163242.809 7f9646f33000-7f9646f3a000 r-xp 00000000 fd:00 790280                     /lib64/librt-2.12.so
       10731:20150115:163242.809 7f9646f3a000-7f9647139000 ---p 00007000 fd:00 790280                     /lib64/librt-2.12.so
       10731:20150115:163242.809 7f9647139000-7f964713a000 r--p 00006000 fd:00 790280                     /lib64/librt-2.12.so
       10731:20150115:163242.809 7f964713a000-7f964713b000 rw-p 00007000 fd:00 790280                     /lib64/librt-2.12.so
       10731:20150115:163242.809 7f964713b000-7f964713d000 r-xp 00000000 fd:00 790273                     /lib64/libdl-2.12.so
       10731:20150115:163242.809 7f964713d000-7f964733d000 ---p 00002000 fd:00 790273                     /lib64/libdl-2.12.so
       10731:20150115:163242.809 7f964733d000-7f964733e000 r--p 00002000 fd:00 790273                     /lib64/libdl-2.12.so
       10731:20150115:163242.809 7f964733e000-7f964733f000 rw-p 00003000 fd:00 790273                     /lib64/libdl-2.12.so
       10731:20150115:163242.809 7f964733f000-7f96473c2000 r-xp 00000000 fd:00 790274                     /lib64/libm-2.12.so
       10731:20150115:163242.809 7f96473c2000-7f96475c1000 ---p 00083000 fd:00 790274                     /lib64/libm-2.12.so
       10731:20150115:163242.809 7f96475c1000-7f96475c2000 r--p 00082000 fd:00 790274                     /lib64/libm-2.12.so
       10731:20150115:163242.809 7f96475c2000-7f96475c3000 rw-p 00083000 fd:00 790274                     /lib64/libm-2.12.so
       10731:20150115:163242.809 7f96475c3000-7f96475e3000 r-xp 00000000 fd:00 791495                     /lib64/ld-2.12.so
       10731:20150115:163242.809 7f96477a9000-7f96477d4000 rw-s 00000000 00:04 360448                     /SYSV6c000b49 (deleted)
       10731:20150115:163242.809 7f96477d4000-7f96477d9000 rw-p 00000000 00:00 0 
       10731:20150115:163242.809 7f96477df000-7f96477e0000 rw-p 00000000 00:00 0 
       10731:20150115:163242.809 7f96477e0000-7f96477e1000 rw-p 00000000 00:00 0 
       10731:20150115:163242.809 7f96477e1000-7f96477e2000 rw-p 00000000 00:00 0 
       10731:20150115:163242.809 7f96477e2000-7f96477e3000 r--p 0001f000 fd:00 791495                     /lib64/ld-2.12.so
       10731:20150115:163242.809 7f96477e3000-7f96477e4000 rw-p 00020000 fd:00 791495                     /lib64/ld-2.12.so
       10731:20150115:163242.809 7f96477e4000-7f96477e5000 rw-p 00000000 00:00 0 
       10731:20150115:163242.809 7fff22d68000-7fff22d7f000 rw-p 00000000 00:00 0                          [stack]
       10731:20150115:163242.809 7fff22dff000-7fff22e00000 r-xp 00000000 00:00 0                          [vdso]
       10731:20150115:163242.809 ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]
       10731:20150115:163242.809 ================================
       10731:20150115:163242.809 Please consider attaching a disassembly listing to your bug report.
       10731:20150115:163242.809 This listing can be produced with, e.g., objdump -DSswx zabbix_agentd.
       10731:20150115:163242.809 ================================
       10723:20150115:163242.810 One child process died (PID:10731,exitcode/signal:1). Exiting ...
      zabbix_agentd [10723]: Error on thread waiting.
       10723:20150115:163242.811 Zabbix Agent stopped. Zabbix 2.4.3 (revision 51175).
      

      sorry for my bad english.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              shawncao ShawnCao
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: