Zabbix5.4.10 Crash occasionally

XMLWordPrintable

    • Type: Problem report
    • Resolution: Duplicate
    • Priority: Trivial
    • None
    • Affects Version/s: None
    • Component/s: Server (S)
    • None
    • Environment:
      5.4.10

      Zabbix5.4.10 Crash occasionally,  eg. a few days, one month, two months.

       

      zabbix server.conf

      ListenPort=10051 
      DebugLevel=3
      LogFileSize=0
      LogFile=/opt/soft/zabbix_server54/log/zabbix_server.log
      PidFile=/opt/soft/zabbix_server54/log/zabbix_server.pid
      DBHost=10.110.10.10
      DBName=zabbix
      DBUser=xxxxxxxx
      DBPassword=yyyyyyyyy
      DBPort=3306
      StartPollers=512
      StartPollersUnreachable=32
      StartTrappers=128
      StartPingers=32
      StartTimers=20
      StartDiscoverers=32
      HousekeepingFrequency=24
      MaxHousekeeperDelete=1000
      CacheSize=8192M
      CacheUpdateFrequency=60 
      StartDBSyncers=100
      HistoryCacheSize=2048M
      HistoryIndexCacheSize=2048M
      TrendCacheSize=2048M
      ValueCacheSize=48G 
      Timeout=30 
      TrapperTimeout=300 
      UnreachablePeriod=200 
      UnavailableDelay=200 
      UnreachableDelay=60 
      AlertScriptsPath=/opt/soft/zabbix_server54/alertscripts
      ExternalScripts=/opt/soft/zabbix_server54/externalscripts 
      FpingLocation=/usr/sbin/fping
      LogSlowQueries=1000
      TmpDir=/dev/shm/ 
      StartProxyPollers=2
      ProxyConfigFrequency=3600
      ProxyDataFrequency=1
      AllowRoot=1

       

      zabbix_server.log

      469006:20221121:105548.601 Got signal [signal:11(SIGSEGV),reason:1,refaddr:0x637ae8b0]. Crashing ...
      469006:20221121:105548.601 ====== Fatal information: ======
      469006:20221121:105548.601 Program counter: 0x7f58580a2b91
      469006:20221121:105548.601 === Registers: ===
      469006:20221121:105548.601 r8      =          1730ab8 =             24316600 =             24316600
      469006:20221121:105548.601 r9      =                0 =                    0 =                    0
      469006:20221121:105548.601 r10     =                c =                   12 =                   12
      469006:20221121:105548.601 r11     =     7f585819df10 =      140017411940112 =      140017411940112
      469006:20221121:105548.601 r12     =           5fe4cd =              6284493 =              6284493
      469006:20221121:105548.601 r13     =             1f9b =                 8091 =                 8091
      469006:20221121:105548.601 r14     =     7f54e6d10938 =      140002626439480 =      140002626439480
      469006:20221121:105548.601 r15     =                8 =                    8 =                    8
      469006:20221121:105548.601 rdi     =         637ae8b1 =           1668999345 =           1668999345
      469006:20221121:105548.601 rsi     =             1f9b =                 8091 =                 8091
      469006:20221121:105548.601 rbp     =         637ae8b1 =           1668999345 =           1668999345
      469006:20221121:105548.601 rbx     =         637ae8b1 =           1668999345 =           1668999345
      469006:20221121:105548.601 rdx     =                0 =                    0 =                    0
      469006:20221121:105548.601 rax     =         637ae8b0 =           1668999344 =           1668999344
      469006:20221121:105548.601 rcx     =               31 =                   49 =                   49
      469006:20221121:105548.601 rsp     =     7ffebf7b1428 =      140732110935080 =      140732110935080

      469006:20221121:105548.601 rip     =     7f58580a2b91 =      140017410911121 =      140017410911121
      469006:20221121:105548.601 efl     =            10202 =                66050 =                66050
      469006:20221121:105548.601 csgsfs  =   2b000000000033 =    12103423998558259 =    12103423998558259
      469006:20221121:105548.601 err     =                4 =                    4 =                    4
      469006:20221121:105548.601 trapno  =                e =                   14 =                   14
      469006:20221121:105548.601 oldmask =             4a06 =                18950 =                18950
      469006:20221121:105548.601 cr2     =         637ae8b0 =           1668999344 =           1668999344
      469006:20221121:105548.601 === Backtrace: ===
      469006:20221121:105548.602 16: /opt/soft/zabbix_server54/sbin/zabbix_server: history syncer #72 [processed 239 values, 263 triggers in 0.220227 sec, syncing history](zbx_backtrace+0x35) [0x549205]
      469006:20221121:105548.602 15: /opt/soft/zabbix_server54/sbin/zabbix_server: history syncer #72 [processed 239 values, 263 triggers in 0.220227 sec, syncing history](zbx_log_fatal_info+0x22d) [0x54951d]
      469006:20221121:105548.602 14: /opt/soft/zabbix_server54/sbin/zabbix_server: history syncer #72 [processed 239 values, 263 triggers in 0.220227 sec, syncing history]() [0x549ad6]
      469006:20221121:105548.602 13: /lib64/libpthread.so.0(+0xf370) [0x7f5859e18370]
      469006:20221121:105548.602 12: /lib64/libc.so.6(+0x86b91) [0x7f58580a2b91]
      469006:20221121:105548.602 11: /lib64/libc.so.6(__strdup+0xe) [0x7f58580a287e]
      469006:20221121:105548.602 10: /opt/soft/zabbix_server54/sbin/zabbix_server: history syncer #72 [processed 239 values, 263 triggers in 0.220227 sec, syncing history](zbx_strdup2+0x3f) [0x55a1bf]
      469006:20221121:105548.602 9: /opt/soft/zabbix_server54/sbin/zabbix_server: history syncer #72 [processed 239 values, 263 triggers in 0.220227 sec, syncing history]() [0x4f8839]
      469006:20221121:105548.602 8: /opt/soft/zabbix_server54/sbin/zabbix_server: history syncer #72 [processed 239 values, 263 triggers in 0.220227 sec, syncing history](DCconfig_get_triggers_by_itemids+0x140) [0x503cf0]
      469006:20221121:105548.602 7: /opt/soft/zabbix_server54/sbin/zabbix_server: history syncer #72 [processed 239 values, 263 triggers in 0.220227 sec, syncing history]() [0x4f1fdb]
      469006:20221121:105548.602 6: /opt/soft/zabbix_server54/sbin/zabbix_server: history syncer #72 [processed 239 values, 263 triggers in 0.220227 sec, syncing history](dbsyncer_thread+0x17c) [0x43ec5c]
      469006:20221121:105548.602 5: /opt/soft/zabbix_server54/sbin/zabbix_server: history syncer #72 [processed 239 values, 263 triggers in 0.220227 sec, syncing history](zbx_thread_start+0x3e) [0x55435e]
      469006:20221121:105548.602 4: /opt/soft/zabbix_server54/sbin/zabbix_server: history syncer #72 [processed 239 values, 263 triggers in 0.220227 sec, syncing history](MAIN_ZABBIX_ENTRY+0xa20) [0x432500]
      469006:20221121:105548.602 3: /opt/soft/zabbix_server54/sbin/zabbix_server: history syncer #72 [processed 239 values, 263 triggers in 0.220227 sec, syncing history](daemon_start+0x1ac) [0x54900c]
      469006:20221121:105548.602 2: /opt/soft/zabbix_server54/sbin/zabbix_server: history syncer #72 [processed 239 values, 263 triggers in 0.220227 sec, syncing history](main+0x5f9) [0x4311d9]
      469006:20221121:105548.602 1: /lib64/libc.so.6(__libc_start_main+0xf5) [0x7f585803db35]
      469006:20221121:105548.602 0: /opt/soft/zabbix_server54/sbin/zabbix_server: history syncer #72 [processed 239 values, 263 triggers in 0.220227 sec, syncing history]() [0x431515] 

      469006:20221121:105548.602 === Memory map: ===
      469006:20221121:105548.602 00400000-00724000 r-xp 00000000 08:05 5905607190                         /opt/soft/zabbix_server54/sbin/zabbix_server
      469006:20221121:105548.602 00923000-00924000 r--p 00323000 08:05 5905607190                         /opt/soft/zabbix_server54/sbin/zabbix_server
      469006:20221121:105548.602 00924000-00932000 rw-p 00324000 08:05 5905607190                         /opt/soft/zabbix_server54/sbin/zabbix_server
      469006:20221121:105548.602 00932000-0093e000 rw-p 00000000 00:00 0 
      469006:20221121:105548.602 01695000-016b6000 rw-p 00000000 00:00 0                                  [heap]
      469006:20221121:105548.602 016b6000-016d4000 rw-p 00000000 00:00 0                                  [heap]
      469006:20221121:105548.602 016d4000-04679000 rw-p 00000000 00:00 0                                  [heap]
      469006:20221121:105548.602 7f48cc000000-7f48cc021000 rw-p 00000000 00:00 0 
      469006:20221121:105548.602 7f48cc021000-7f48d0000000 ---p 00000000 00:00 0 
      469006:20221121:105548.602 7f48d3e13000-7f48d3e14000 ---p 00000000 00:00 0 
      469006:20221121:105548.602 7f48d3e14000-7f48d4614000 rw-p 00000000 00:00 0 
      469006:20221121:105548.602 7f48d4614000-7f48d4a14000 rw-s 00000000 00:05 5079047                    /SYSV00000000 (deleted)
      469006:20221121:105548.602 7f48d4a14000-7f54d4a14000 rw-s 00000000 00:05 5046278                    /SYSV00000000 (deleted)
      469006:20221121:105548.602 7f54d4a14000-7f56d4a14000 rw-s 00000000 00:05 4980740                    /SYSV00000000 (deleted)
      469006:20221121:105548.602 7f56d4a14000-7f5754a14000 rw-s 00000000 00:05 4947971                    /SYSV00000000 (deleted)
      469006:20221121:105548.602 7f5754a14000-7f57d4a14000 rw-s 00000000 00:05 4915202                    /SYSV00000000 (deleted)
      469006:20221121:105548.602 7f57d4a14000-7f5854a14000 rw-s 00000000 00:05 4882433                    /SYSV00000000 (deleted)
      469006:20221121:105548.602 7f5854a14000-7f5854a16000 r-xp 00000000 08:03 76379151                   /usr/lib64/libfreebl3.so
      469006:20221121:105548.602 7f5854a16000-7f5854c15000 ---p 00002000 08:03 76379151                   /usr/lib64/libfreebl3.so
      469006:20221121:105548.602 7f5854c15000-7f5854c16000 r--p 00001000 08:03 76379151                   /usr/lib64/libfreebl3.so
      469006:20221121:105548.602 7f5854c16000-7f5854c17000 rw-p 00002000 08:03 76379151                   /usr/lib64/libfreebl3.so
      469006:20221121:105548.602 7f5854c17000-7f5854c1f000 r-xp 00000000 08:03 75576861                   /usr/lib64/libcrypt-2.17.so
      469006:20221121:105548.602 7f5854c1f000-7f5854e1e000 ---p 00008000 08:03 75576861                   /usr/lib64/libcrypt-2.17.so
      469006:20221121:105548.602 7f5854e1e000-7f5854e1f000 r--p 00007000 08:03 75576861                   /usr/lib64/libcrypt-2.17.so
      469006:20221121:105548.602 7f5854e1f000-7f5854e20000 rw-p 00008000 08:03 75576861                   /usr/lib64/libcrypt-2.17.so
      469006:20221121:105548.602 7f5854e20000-7f5854e4e000 rw-p 00000000 00:00 0 
      469006:20221121:105548.602 7f5854e4e000-7f5854e72000 r-xp 00000000 08:03 75850256                   /usr/lib64/libselinux.so.1
      469006:20221121:105548.602 7f5854e72000-7f5855071000 ---p 00024000 08:03 75850256                   /usr/lib64/libselinux.so.1
      469006:20221121:105548.602 7f5855071000-7f5855072000 r--p 00023000 08:03 75850256                   /usr/lib64/libselinux.so.1
      469006:20221121:105548.602 7f5855072000-7f5855073000 rw-p 00024000 08:03 75850256                   /usr/lib64/libselinux.so.1
      469006:20221121:105548.602 7f5855073000-7f5855075000 rw-p 00000000 00:00 0 

      .................

      469006:20221121:105548.604 7f585a502000-7f585a50b000 r--p 002dd000 08:03 92406836                   /usr/lib64/mysql/libmysqlclient.so.18.0.0
      469006:20221121:105548.604 7f585a50b000-7f585a51f000 rw-p 002e6000 08:03 92406836                   /usr/lib64/mysql/libmysqlclient.so.18.0.0
      469006:20221121:105548.604 7f585a51f000-7f585a525000 rw-p 00000000 00:00 0 
      469006:20221121:105548.604 7f585a525000-7f585a545000 r-xp 00000000 08:03 75528196                   /usr/lib64/ld-2.17.so
      469006:20221121:105548.604 7f585a6ad000-7f585a6e2000 r--s 00000000 08:03 84069523                   /var/db/nscd/services
      469006:20221121:105548.604 7f585a6e2000-7f585a724000 rw-s 00000000 00:05 5013509                    /SYSV00000000 (deleted)
      469006:20221121:105548.604 7f585a724000-7f585a738000 rw-p 00000000 00:00 0 
      469006:20221121:105548.604 7f585a740000-7f585a741000 rw-p 00000000 00:00 0 
      469006:20221121:105548.604 7f585a741000-7f585a742000 rw-s 00000000 00:05 4849664                    /SYSV00000000 (deleted)
      469006:20221121:105548.604 7f585a742000-7f585a743000 rw-p 00000000 00:00 0 
      469006:20221121:105548.604 7f585a743000-7f585a744000 rw-p 00000000 00:00 0 
      469006:20221121:105548.604 7f585a744000-7f585a745000 r--p 0001f000 08:03 75528196                   /usr/lib64/ld-2.17.so
      469006:20221121:105548.604 7f585a745000-7f585a746000 rw-p 00020000 08:03 75528196                   /usr/lib64/ld-2.17.so
      469006:20221121:105548.604 7f585a746000-7f585a747000 rw-p 00000000 00:00 0 
      469006:20221121:105548.604 7ffebf79f000-7ffebf7d1000 rw-p 00000000 00:00 0                          [stack]
      469006:20221121:105548.604 7ffebf7d1000-7ffebf7d4000 r--p 00000000 00:00 0                          [vvar]
      469006:20221121:105548.604 7ffebf7d4000-7ffebf7d6000 r-xp 00000000 00:00 0                          [vdso]
      469006:20221121:105548.604 ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]
      469006:20221121:105548.604 ================================
      469006:20221121:105548.604 Please consider attaching a disassembly listing to your bug report.
      469006:20221121:105548.604 This listing can be produced with, e.g., objdump -DSswx zabbix_server.
      469006:20221121:105548.604 ================================
      468821:20221121:105548.626 One child process died (PID:469006,exitcode/signal:1). Exiting ...
      zabbix_server [468821]: Error waiting for process with PID 469006: [10] No child processes

      468821:20221121:105549.007 syncing history data...
      468821:20221121:105549.137 syncing history data... 94.019139%
      468821:20221121:105549.137 syncing history data... 94.019139%
      468821:20221121:105549.137 syncing history data... 94.019139%
      468821:20221121:105549.137 syncing history data... 94.019139%
      468821:20221121:105549.137 syncing history data... 94.019139%
      468821:20221121:105549.137 syncing history data... 94.019139%
      468821:20221121:105549.137 syncing history data... 94.019139%
      468821:20221121:105549.137 syncing history data... 94.019139%
      468821:20221121:105549.137 syncing history data... 94.019139%
      468821:20221121:105549.137 syncing history data... 94.019139%
      468821:20221121:105549.137 syncing history data... 94.019139%
      468821:20221121:105549.137 syncing history data... 94.019139%
      468821:20221121:105549.137 syncing history data... 94.019139%
      468821:20221121:105549.137 syncing history data... 94.019139%
      468821:20221121:105549.137 syncing history data... 94.019139%
      468821:20221121:105549.137 syncing history data... 94.019139%
      468821:20221121:105549.137 syncing history data... 94.019139%
      468821:20221121:105549.138 syncing history data... 94.019139%
      468821:20221121:105549.138 syncing history data... 94.019139%
      468821:20221121:105549.138 syncing history data... 94.019139%
      468821:20221121:105549.138 syncing history data... 94.019139%
      468821:20221121:105549.138 syncing history data... 94.019139%
      468821:20221121:105549.138 syncing history data... 94.019139%
      468821:20221121:105549.138 syncing history data... 94.019139%
      468821:20221121:105549.138 syncing history data... 94.019139%
      468821:20221121:105549.138 syncing history data... 94.019139%

      ...................full filled with syncing history data... 94.019139%, flushing all the time...............

       

      Description:

      zabbix server runs only with one main process, the other subprocess died.

      Why zabbix server crash occasionally? Is the configuration wrong? or somewhere wrong?

       

       

       

       

       

       

       

       

       

       

            Assignee:
            Zabbix Development Team
            Reporter:
            shenhua
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: