Uploaded image for project: 'ZABBIX BUGS AND ISSUES'
  1. ZABBIX BUGS AND ISSUES
  2. ZBX-20145

Zabbix Agent killed with SIGSEV on aarch64 / apple silicon when using TLS encryption

XMLWordPrintable

    • Icon: Problem report Problem report
    • Resolution: Cannot Reproduce
    • Icon: Trivial Trivial
    • 6.2 (plan)
    • None
    • Agent (G)
    • None
    • Team A
    • Sprint 84 (Jan 2022), Sprint 85 (Feb 2022), Sprint 86 (Mar 2022), Sprint 87 (Apr 2022), Sprint 88 (May 2022)
    • 1

      Steps to reproduce:

      1. Get an apple silicon mac (MBP 16", M1 Max) and install official docker for m1 mac
      2. create docker container with ubuntu image ubuntu:focal
      3. Install zabbix agent 5.4.7 either via official repo or by compiling yourself (./configure --sysconfdir=/etc/zabbix --enable-agent --enable-ipv6 --with-openssl)
      4. Configure a tls connection to a zabbix server
        TLSConnect=psk
        TLSAccept=psk
        TLSPSKIdentity=tls_autoregistration
        TLSPSKFile=/etc/zabbix/autoregistration_key.psk
      5. Start Zabbix Agent (eg ./zabbix_agentd -c /etc/zabbix/zabbix_agentd.conf -f)

      Result:

      The agent immediately dies:

       

      Starting Zabbix Agent [REDACTED]. Zabbix 5.4.7 (revision 84dc2ec5dc).
Press Ctrl+C to exit.
52141:20211031:051351.668 Starting Zabbix Agent [REDACTED]. Zabbix 5.4.7 (revision 84dc2ec5dc).
 52141:20211031:051351.668 **** Enabled features ****
 52141:20211031:051351.668 IPv6 support: YES
 52141:20211031:051351.668 TLS support: YES
 52141:20211031:051351.668 **************************
 52141:20211031:051351.668 using configuration file: /etc/zabbix/zabbix_agentd.conf
 52141:20211031:051351.668 In zbx_load_modules()
 52141:20211031:051351.668 End of zbx_load_modules():SUCCEED
 52141:20211031:051351.668 In init_collector_data()
 52141:20211031:051351.669 In zbx_dshm_create() size:0
 52141:20211031:051351.669 End of zbx_dshm_create():SUCCEED shmid:-1
 52141:20211031:051351.669 End of init_collector_data()
 52141:20211031:051351.669 agent #0 started [main process]
 52142:20211031:051351.669 agent #1 started [collector]
 52142:20211031:051351.669 In init_cpu_collector()
 52142:20211031:051351.669 End of init_cpu_collector():SUCCEED
 52142:20211031:051351.670 zbx_setproctitle() title:'collector [processing data]'
 52142:20211031:051351.670 In update_cpustats()
 52142:20211031:051351.670 End of update_cpustats()
 52142:20211031:051351.670 zbx_setproctitle() title:'collector [idle 1 sec]'
 52144:20211031:051351.671 agent #3 started [listener #2]
 52144:20211031:051351.671 In zbx_tls_init_child()
 52145:20211031:051351.672 agent #4 started [listener #3]
 52145:20211031:051351.672 In zbx_tls_init_child()
 52145:20211031:051351.673 OpenSSL library (version OpenSSL 1.1.1f 31 Mar 2020) initialized
 52145:20211031:051351.673 zbx_tls_init_child() loaded PSK identity "tls_autoregistration"
 52145:20211031:051351.673 zbx_tls_init_child() loaded PSK from file "/etc/zabbix/autoregistration_key.psk"
 52145:20211031:051351.673 zbx_tls_init_child() PSK ciphersuites: TLS_CHACHA20_POLY1305_SHA256 TLS_AES_128_GCM_SHA256 ECDHE-PSK-AES128-CBC-SHA256 ECDHE-PSK-AES128-CBC-SHA PSK-AES128-GCM-SHA256 PSK-AES128-CCM8 PSK-AES128-CCM PSK-AES128-CBC-SHA256 PSK-AES128-CBC-SHA
 52145:20211031:051351.673 End of zbx_tls_init_child()
 52145:20211031:051351.673 zbx_setproctitle() title:'listener #3 [waiting for connection]'
 52143:20211031:051351.674 agent #2 started [listener #1]
 52143:20211031:051351.674 In zbx_tls_init_child()
 52144:20211031:051351.674 OpenSSL library (version OpenSSL 1.1.1f 31 Mar 2020) initialized
 52146:20211031:051351.674 agent #5 started [active checks #1]
 52144:20211031:051351.674 zbx_tls_init_child() loaded PSK identity "tls_autoregistration"
 52146:20211031:051351.675 In zbx_tls_init_child()
 52144:20211031:051351.675 zbx_tls_init_child() loaded PSK from file "/etc/zabbix/autoregistration_key.psk"
 52144:20211031:051351.676 zbx_tls_init_child() PSK ciphersuites: TLS_CHACHA20_POLY1305_SHA256 TLS_AES_128_GCM_SHA256 ECDHE-PSK-AES128-CBC-SHA256 ECDHE-PSK-AES128-CBC-SHA PSK-AES128-GCM-SHA256 PSK-AES128-CCM8 PSK-AES128-CCM PSK-AES128-CBC-SHA256 PSK-AES128-CBC-SHA
 52144:20211031:051351.676 End of zbx_tls_init_child()
 52144:20211031:051351.676 zbx_setproctitle() title:'listener #2 [waiting for connection]'
 52143:20211031:051351.676 OpenSSL library (version OpenSSL 1.1.1f 31 Mar 2020) initialized
 52143:20211031:051351.676 zbx_tls_init_child() loaded PSK identity "tls_autoregistration"
 52143:20211031:051351.676 zbx_tls_init_child() loaded PSK from file "/etc/zabbix/autoregistration_key.psk"
 52143:20211031:051351.676 zbx_tls_init_child() PSK ciphersuites: TLS_CHACHA20_POLY1305_SHA256 TLS_AES_128_GCM_SHA256 ECDHE-PSK-AES128-CBC-SHA256 ECDHE-PSK-AES128-CBC-SHA PSK-AES128-GCM-SHA256 PSK-AES128-CCM8 PSK-AES128-CCM PSK-AES128-CBC-SHA256 PSK-AES128-CBC-SHA
 52143:20211031:051351.677 End of zbx_tls_init_child()
 52143:20211031:051351.677 zbx_setproctitle() title:'listener #1 [waiting for connection]'
 52146:20211031:051351.679 OpenSSL library (version OpenSSL 1.1.1f 31 Mar 2020) initialized
 52146:20211031:051351.679 zbx_tls_init_child() loaded PSK identity "tls_autoregistration"
 52146:20211031:051351.679 zbx_tls_init_child() loaded PSK from file "/etc/zabbix/autoregistration_key.psk"
 52146:20211031:051351.679 zbx_tls_init_child() PSK ciphersuites: TLS_CHACHA20_POLY1305_SHA256 TLS_AES_128_GCM_SHA256 ECDHE-PSK-AES128-CBC-SHA256 ECDHE-PSK-AES128-CBC-SHA PSK-AES128-GCM-SHA256 PSK-AES128-CCM8 PSK-AES128-CCM PSK-AES128-CBC-SHA256 PSK-AES128-CBC-SHA
 52146:20211031:051351.679 End of zbx_tls_init_child()
 52146:20211031:051351.679 In init_active_metrics()
 52146:20211031:051351.679 buffer: first allocation for 100 elements
 52146:20211031:051351.679 End of init_active_metrics()
 52146:20211031:051351.679 In send_buffer() host:'10.8.0.1' port:10051 entries:0/100
 52146:20211031:051351.679 End of send_buffer():SUCCEED
 52146:20211031:051351.679 zbx_setproctitle() title:'active checks #1 [getting list of active checks]'
 52146:20211031:051351.679 In refresh_active_checks() host:'10.8.0.1' port:10051
 52146:20211031:051351.698 In zbx_tls_connect(): psk_identity:"tls_autoregistration"
 52146:20211031:051351.699 zbx_psk_client_cb() requested PSK identity "tls_autoregistration"
 52146:20211031:051351.735 End of zbx_tls_connect():SUCCEED (established TLSv1.3 TLS_CHACHA20_POLY1305_SHA256)
 52146:20211031:051351.736 sending [{"request":"active checks","host":"REDACTED","host_metadata":"tpl:oauth2-proxy|tpl:nginx|tpl:apache|tpl:linux","ip":"10.8.0.6"}]
 52146:20211031:051351.736 Got signal [signal:11(SIGSEGV),reason:1,refaddr:0x20ffff9a60e07c]. Crashing ...
 52146:20211031:051351.736 ====== Fatal information: ======
 52146:20211031:051351.736 program counter not available for this architecture
 52146:20211031:051351.736 === Registers: ===
 52146:20211031:051351.736 register dump not available for this architecture
 52146:20211031:051351.736 === Backtrace: ===
 52141:20211031:051351.737 One child process died (PID:52146,exitcode/signal:11). Exiting ...
 52141:20211031:051351.737 zbx_on_exit() called
zabbix_agentd [52141]: Error waiting for process with PID 52146: [10] No child processes
 52141:20211031:051351.739 In zbx_dshm_destroy() shmid:-1
 52141:20211031:051351.739 End of zbx_dshm_destroy():SUCCEED
 52141:20211031:051351.739 In zbx_unload_modules()
 52141:20211031:051351.739 End of zbx_unload_modules()
 52141:20211031:051351.739 Zabbix Agent stopped. Zabbix 5.4.7 (revision 84dc2ec5dc).

       

       

      --version

       

      zabbix_agentd (daemon) (Zabbix) 5.4.7Revision 84dc2ec5dc 28 October 2021, compilation time: Oct 20 2021 12:31:10

Copyright (C) 2021 Zabbix SIALicense GPLv2+: GNU GPL version 2 or later <http://gnu.org/licenses/gpl.html>.This is free software: you are free to change and redistribute it according tothe license. There is NO WARRANTY, to the extent permitted by law.

This product includes software developed by the OpenSSL Projectfor use in the OpenSSL Toolkit (http://www.openssl.org/).

Compiled with OpenSSL 1.1.1f  31 Mar 2020
Running with OpenSSL 1.1.1f  31 Mar 2020

       

      The same is happening with Zabbix Agent 2 (see log attached).
      When TLS is disabled, the agent works fine.

      Expected:
      I expected the same working zabbix agent as of other platforms like AMD64, Raspberry Pi.

      The zabbix agent package (4.0.17+dfsg-1) provided by ubuntu focal 20.04 is working fine on docker ubuntu @ apple silicon.

        1. zabbix_agent2.log
          11 kB
          Adrian Kirchner

            vso Vladislavs Sokurenko
            akirchner Adrian Kirchner
            Team A
            Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

              Created:
              Updated:
              Resolved: