Uploaded image for project: 'ZABBIX BUGS AND ISSUES'
  1. ZABBIX BUGS AND ISSUES
  2. ZBX-21927

Crash in trappers when having SQL connections issue with Oracle DB backend

XMLWordPrintable

    • Team A
    • Sprint 94 (Nov 2022), Sprint 95 (Dec 2022)
    • 1

      This happened during Oracle DB patching.
      The Oracle DB is running in cluster, with "Oracle Single Client Access Name (SCAN)" connection configured.

      There are SQL errors in the log, which is not good of course, but the crash should not happen anyway!

      SQL errors and Backtrace for the process:

       16902:20221112:021019.185 [Z3006] fetch failed: [3113] ORA-03113: end-of-file on communication channel
      Process ID: 35401
      Session ID: 34 Serial number: 9380
      
       16902:20221112:021019.185 [Z3005] query failed: [-1] ORA-03114: not connected to ORACLE [commit failed]
       16902:20221112:021019.185 [Z3005] query failed: [-1] ORA-03114: not connected to ORACLE [rollback failed]
       16902:20221112:021019.185 cannot perform transaction rollback, connection will be reset
       16902:20221112:021019.185 cannot process queries: database is closed
      .....
       16902:20221112:021022.588 Got signal [signal:11(SIGSEGV),reason:1,refaddr:(nil)]. Crashing ...
       16902:20221112:021022.588 ====== Fatal information: ======
      .....
       16902:20221112:021022.589 === Backtrace: ===
       16867:20221112:021022.591 [Z3007] query failed: [-1] ORA-01036: illegal variable name/number
       16867:20221112:021022.592 [Z3007] query failed: [-1] ORA-01036: illegal variable name/number
       16902:20221112:021022.595 17: /zbx/pzbx/a01/zabbix-server/sbin/zabbix_server: trapper #5 [processing data](zbx_backtrace+0x3d) [0x61f692]
       16902:20221112:021022.595 16: /zbx/pzbx/a01/zabbix-server/sbin/zabbix_server: trapper #5 [processing data](zbx_log_fatal_info+0x139) [0x61f8e9]
       16902:20221112:021022.595 15: /zbx/pzbx/a01/zabbix-server/sbin/zabbix_server: trapper #5 [processing data]() [0x620064]
       16902:20221112:021022.595 14: /lib64/libpthread.so.0(+0xf630) [0x7f7b8ab88630]
       16902:20221112:021022.595 13: /lib64/libc.so.6(+0x3afe7) [0x7f7b892dcfe7]
       16902:20221112:021022.595 12: /lib64/libc.so.6(atoi+0x10) [0x7f7b892d9900]
       16902:20221112:021022.595 11: /zbx/pzbx/a01/zabbix-server/sbin/zabbix_server: trapper #5 [processing data]() [0x489f30]
       16902:20221112:021022.595 10: /zbx/pzbx/a01/zabbix-server/sbin/zabbix_server: trapper #5 [processing data](send_list_of_active_checks_json+0x3aa) [0x48ac24]
       16902:20221112:021022.595 9: /zbx/pzbx/a01/zabbix-server/sbin/zabbix_server: trapper #5 [processing data]() [0x488bb3]
       16902:20221112:021022.595 8: /zbx/pzbx/a01/zabbix-server/sbin/zabbix_server: trapper #5 [processing data]() [0x48921e]
       16902:20221112:021022.595 7: /zbx/pzbx/a01/zabbix-server/sbin/zabbix_server: trapper #5 [processing data](trapper_thread+0x2d1) [0x4894f1]
       16902:20221112:021022.595 6: /zbx/pzbx/a01/zabbix-server/sbin/zabbix_server: trapper #5 [processing data](zbx_thread_start+0x37) [0x62f438]
       16902:20221112:021022.595 5: /zbx/pzbx/a01/zabbix-server/sbin/zabbix_server: trapper #5 [processing data]() [0x44e35e]
       16902:20221112:021022.595 4: /zbx/pzbx/a01/zabbix-server/sbin/zabbix_server: trapper #5 [processing data](MAIN_ZABBIX_ENTRY+0x765) [0x44f250]
       16902:20221112:021022.595 3: /zbx/pzbx/a01/zabbix-server/sbin/zabbix_server: trapper #5 [processing data](daemon_start+0x367) [0x61f366]
       16902:20221112:021022.595 2: /zbx/pzbx/a01/zabbix-server/sbin/zabbix_server: trapper #5 [processing data](main+0x30b) [0x44da95]
       16902:20221112:021022.595 1: /lib64/libc.so.6(__libc_start_main+0xf5) [0x7f7b892c4555]
       16902:20221112:021022.595 0: /zbx/pzbx/a01/zabbix-server/sbin/zabbix_server: trapper #5 [processing data]() [0x44c869]
      

      Full log with SQL errors for other processes is attached.

            vso Vladislavs Sokurenko
            zalex_ua Oleksii Zagorskyi
            Team A
            Votes:
            1 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated:
              Resolved: