Uploaded image for project: 'ZABBIX BUGS AND ISSUES'
  1. ZABBIX BUGS AND ISSUES
  2. ZBX-5233

Zabbix Server Crash while Zabbix Node send data

    XMLWordPrintable

Details

    • Incident report
    • Resolution: Duplicate
    • Blocker
    • None
    • 2.0.0
    • Server (S)
    • Ubuntu Server 12.04
      Zabbix 2.0

    Description

      When Zabbix Node (node1) try to syncronize data the Zabbix Master Node crash.

      Log from Zabbix_Server of NODE 1 (node id 2):

      20084:20120622:095136.759 NODE 2: Sending alerts of node 2 to node 1 datalen 946
      20084:20120622:095136.869 NOT OK
      20084:20120622:095136.893 NODE 2: Sending history_sync of node 2 to node 1 datalen 5230
      20084:20120622:095138.120 NODE 2: Sending events of node 2 to node 1 datalen 1337
      20084:20120622:095138.420 NODE 2: Sending auditlog of node 2 to node 1 datalen 726
      20084:20120622:095138.553 NODE 2: Sending auditlog_details of node 2 to node 1 datalen 152
      20084:20120622:095146.798 NODE 2: Sending alerts of node 2 to node 1 datalen 946
      20084:20120622:095146.907 NODE 2: Sending history_sync of node 2 to node 1 datalen 159
      20084:20120622:095217.849 NODE 2: Sending configuration changes to master node 1 for node 2 datalen 3290
      20084:20120622:095225.225 NODE 2: Error while receiving answer from Node [1] error: ZBX_TCP_READ() failed: [104] Connection reset by peer
      20084:20120622:095225.233 NODE 2: Sending history_sync of node 2 to node 1 datalen 501
      20084:20120622:095225.239 NODE 2: Unable to connect to Node [1] error: cannot connect to [[10.11.210.241]:10051]: [111] Connection refused

      Log from Zabbix_Server of MASTER NODE (node id 1):

      root@zabbixone:~# tail /tmp/zabbix_server.log -n100
      1470:20120622:053054.113 ******************************
      1472:20120622:053054.632 server #1 started configuration syncer #1
      1480:20120622:053054.655 server #9 started trapper #1
      1481:20120622:053054.656 server #10 started trapper #2
      1473:20120622:053054.659 server #2 started db watchdog #1
      1482:20120622:053054.664 server #11 started trapper #3
      1474:20120622:053054.670 server #3 started poller #1
      1484:20120622:053054.691 server #13 started trapper #5
      1485:20120622:053054.692 server #14 started icmp pinger #1
      1486:20120622:053054.693 server #15 started alerter #1
      1487:20120622:053054.694 server #16 started housekeeper #1
      1487:20120622:053054.695 executing housekeeper
      1483:20120622:053054.697 server #12 started trapper #4
      1477:20120622:053054.700 server #6 started poller #4
      1476:20120622:053054.701 server #5 started poller #3
      1478:20120622:053054.703 server #7 started poller #5
      1475:20120622:053054.707 server #4 started poller #2
      1494:20120622:053054.716 server #18 started node watcher #1
      1493:20120622:053054.723 server #17 started timer #1
      1479:20120622:053054.725 server #8 started unreachable poller #1
      1499:20120622:053054.732 server #21 started history syncer #1
      1500:20120622:053054.739 server #22 started history syncer #2
      1496:20120622:053054.740 server #19 started http poller #1
      1501:20120622:053054.741 server #23 started history syncer #3
      1507:20120622:053054.742 server #26 started ipmi poller #1
      1508:20120622:053054.743 server #27 started ipmi poller #2
      1505:20120622:053054.747 server #24 started history syncer #4
      1506:20120622:053054.748 server #25 started escalator #1
      1470:20120622:053054.757 server #0 started [main process]
      1516:20120622:053054.757 server #28 started proxy poller #1
      1497:20120622:053054.758 server #20 started discoverer #1
      1517:20120622:053054.763 server #29 started self-monitoring #1
      1497:20120622:053054.788 fping failed: "10.11.210.0 :"
      1487:20120622:053104.017 housekeeper deleted: 38 records from history and trends, 0 records of deleted items, 0 events, 0 alerts, 0 sessions
      1482:20120622:053151.326 NODE 1: Received configuration changes from slave node 2 for node 2 datalen 5760

          • stack smashing detected ***: /usr/local/sbin/zabbix_server terminated
            1482:20120622:053156.997 Got signal [signal:11(SIGSEGV),reason:1,refaddr:0x65373234]. Crashing ...
            1482:20120622:053156.998 ====== Fatal information: ======
            1482:20120622:053156.998 Program counter: 0xb7034b19
            1482:20120622:053156.998 === Registers: ===
            1482:20120622:053156.998 gs = 33 = 51 = 51
            1482:20120622:053156.998 fs = 0 = 0 = 0
            1482:20120622:053156.998 es = 7b = 123 = 123
            1482:20120622:053156.998 ds = 7b = 123 = 123
            1482:20120622:053156.998 edi = bfb83190 = 3216519568 = -1078447728
            1482:20120622:053156.998 esi = bfb830d0 = 3216519376 = -1078447920
            1482:20120622:053156.999 ebp = bfb83218 = 3216519704 = -1078447592
            1482:20120622:053156.999 esp = bfb83050 = 3216519248 = -1078448048
            1482:20120622:053156.999 ebx = b703bff4 = 3070476276 = -1224491020
            1482:20120622:053157.000 edx = 65373234 = 1698116148 = 1698116148
            1482:20120622:053157.000 ecx = bfb84470 = 3216524400 = -1078442896
            1482:20120622:053157.000 eax = bfb83190 = 3216519568 = -1078447728
            1482:20120622:053157.000 trapno = e = 14 = 14
            1482:20120622:053157.001 err = 4 = 4 = 4
            1482:20120622:053157.001 eip = b7034b19 = 3070446361 = -1224520935
            1482:20120622:053157.001 cs = 73 = 115 = 115
            1482:20120622:053157.001 efl = 210246 = 2163270 = 2163270
            1482:20120622:053157.002 uesp = bfb83050 = 3216519248 = -1078448048
            1482:20120622:053157.002 ss = 7b = 123 = 123
            1482:20120622:053157.002 === Stack frame: ===
            1482:20120622:053157.003 +0x40(%ebp) = ebp + 64 = 00000000 = 0 = 0
            1482:20120622:053157.003 +0x3c(%ebp) = ebp + 60 = 00000040 = 64 = 64
            1482:20120622:053157.003 +0x38(%ebp) = ebp + 56 = 00000004 = 4 = 4
            1482:20120622:053157.003 +0x34(%ebp) = ebp + 52 = bfb837a0 = 3216521120 = -1078446176
            1482:20120622:053157.004 +0x30(%ebp) = ebp + 48 = 00000000 = 0 = 0
            1482:20120622:053157.004 +0x2c(%ebp) = ebp + 44 = 080cebce = 135064526 = 135064526
            1482:20120622:053157.004 +0x28(%ebp) = ebp + 40 = bfb83908 = 3216521480 = -1078445816
            1482:20120622:053157.004 +0x24(%ebp) = ebp + 36 = 00000000 = 0 = 0
            1482:20120622:053157.005 +0x20(%ebp) = ebp + 32 = 00000000 = 0 = 0
            1482:20120622:053157.005 +0x1c(%ebp) = ebp + 28 = 080cebcf = 135064527 = 135064527
            1482:20120622:053157.005 +0x18(%ebp) = ebp + 24 = 00000000 = 0 = 0
            1482:20120622:053157.005 +0x14(%ebp) = ebp + 20 = b70b590b = 3070974219 = -1223993077
            1482:20120622:053157.006 +0x10(%ebp) = ebp + 16 = 081225d4 = 135407060 = 135407060
            1482:20120622:053157.006 +0x0c(%ebp) = ebp + 12 = bfb8324c = 3216519756 = -1078447540
            1482:20120622:053157.006 +0x08(%ebp) = ebp + 8 = b7173f00 = 3071753984 = 1223213312 <-- call arguments
            1482:20120622:053157.007 +0x04(%ebp) = ebp + 4 = b7174007 <--- return address
            1482:20120622:053157.007 (%ebp) = ebp = bfb83278 <--- saved ebp value
            1482:20120622:053157.007 0x04(%ebp) = ebp - 4 = bfb837a0 = 3216521120 = -1078446176 <-- local variables
            1482:20120622:053157.008 -0x08(%ebp) = ebp - 8 = 00000040 = 64 = 64
            1482:20120622:053157.008 -0x0c(%ebp) = ebp - 12 = 00000000 = 0 = 0
            1482:20120622:053157.008 -0x10(%ebp) = ebp - 16 = 00000000 = 0 = 0
            1482:20120622:053157.008 -0x14(%ebp) = ebp - 20 = 00000000 = 0 = 0
            1482:20120622:053157.009 -0x18(%ebp) = ebp - 24 = 00000000 = 0 = 0
            1482:20120622:053157.009 -0x1c(%ebp) = ebp - 28 = 00000000 = 0 = 0
            1482:20120622:053157.009 -0x20(%ebp) = ebp - 32 = 00000000 = 0 = 0
            1482:20120622:053157.009 -0x24(%ebp) = ebp - 36 = 00000000 = 0 = 0
            1482:20120622:053157.010 -0x28(%ebp) = ebp - 40 = 40000000 = 1073741824 = 1073741824
            1482:20120622:053157.010 -0x2c(%ebp) = ebp - 44 = 0806fc60 = 134675552 = 134675552
            1482:20120622:053157.010 -0x30(%ebp) = ebp - 48 = 0812fff4 = 135462900 = 135462900
            1482:20120622:053157.011 -0x34(%ebp) = ebp - 52 = 00000000 = 0 = 0
            1482:20120622:053157.011 -0x38(%ebp) = ebp - 56 = 00000000 = 0 = 0
            1482:20120622:053157.011 -0x3c(%ebp) = ebp - 60 = 65373234 = 1698116148 = 1698116148
            1482:20120622:053157.012 -0x40(%ebp) = ebp - 64 = bfb84470 = 3216524400 = -1078442896
            1482:20120622:053157.012 === Backtrace: ===
            1470:20120622:053157.013 One child process died (PID:1482,exitcode/signal:11). Exiting ...
            1470:20120622:053159.015 syncing history data...
            1470:20120622:053159.122 syncing history data done
            1470:20120622:053159.122 syncing trends data...
            1470:20120622:053159.945 syncing trends data done
            1470:20120622:053159.946 Zabbix Server stopped. Zabbix 2.0.0 (revision 27675).

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              spaww Adail Horst
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: