Uploaded image for project: 'ZABBIX BUGS AND ISSUES'
  1. ZABBIX BUGS AND ISSUES
  2. ZBX-11025

3.0 server with MySQL exits after receives WSREP related error message

    XMLWordPrintable

    Details

    • Team:
      Team A
    • Sprint:
      Sprint 50 (Mar 2019), Sprint 51 (Apr 2019)
    • Story Points:
      1

      Description

      This log file is part from a production, when MySQL server got a WSREP (Write Set REPlication) error. Zabbix server termination not on start but on run time:

       11126:20160505:030001.537 [Z3005] query failed: [1213] Deadlock found when trying to get lock; try restarting transaction [commit;]
       11125:20160505:030001.572 [Z3005] query failed: [1213] Deadlock found when trying to get lock; try restarting transaction [insert into history_uint (itemid,clock,ns,value) values (28561,1462435201,190482764,1462435201),(114841,1462435201,207893842,0),(253081,1462435201,209905590,0), trimmed ..... );
      ]
       11126:20160505:030002.054 [Z3005] query failed: [2013] Lost connection to MySQL server during query [commit;]
       11125:20160505:030002.054 [Z3005] query failed: [2013] Lost connection to MySQL server during query [commit;]
       11126:20160505:030002.059 [Z3001] connection to database 'zabbix' failed: [2013] Lost connection to MySQL server at 'reading initial communication packet', system error: 104
       11125:20160505:030002.059 [Z3001] connection to database 'zabbix' failed: [2013] Lost connection to MySQL server at 'reading initial communication packet', system error: 104
       11126:20160505:030002.059 database is down: reconnecting in 10 seconds
       11125:20160505:030002.059 database is down: reconnecting in 10 seconds
      ...
       11126:20160505:030022.062 [Z3001] connection to database 'zabbix' failed: [2013] Lost connection to MySQL server at 'reading initial communication packet', system error: 104
       11126:20160505:030022.062 database is down: reconnecting in 10 seconds
       11128:20160505:030022.193 [Z3001] connection to database 'zabbix' failed: [1047] WSREP has not yet prepared node for application use
       11128:20160505:030022.193 Cannot connect to the database. Exiting...
       10964:20160505:030022.265 One child process died (PID:11128,exitcode/signal:1). Exiting ...
       10964:20160505:030024.270 [Z3001] connection to database 'zabbix' failed: [1047] WSREP has not yet prepared node for application use
       10964:20160505:030024.270 Cannot connect to the database. Exiting...
       30929:20160505:030034.378 Starting Zabbix Server. Zabbix 3.0.2 (revision 59540).
       30929:20160505:030034.378 ****** Enabled features ******
       30929:20160505:030034.379 SNMP monitoring:           YES
       30929:20160505:030034.379 IPMI monitoring:           YES
       30929:20160505:030034.379 Web monitoring:            YES
       30929:20160505:030034.379 VMware monitoring:         YES
       30929:20160505:030034.379 SMTP authentication:       YES
       30929:20160505:030034.379 Jabber notifications:      YES
       30929:20160505:030034.379 Ez Texting notifications:  YES
       30929:20160505:030034.379 ODBC:                      YES
       30929:20160505:030034.379 SSH2 support:              YES
       30929:20160505:030034.379 IPv6 support:              YES
       30929:20160505:030034.379 TLS support:               YES
       30929:20160505:030034.379 ******************************
       30929:20160505:030034.379 using configuration file: /etc/zabbix/zabbix_server.conf
       30929:20160505:030034.394 [Z3001] connection to database 'zabbix' failed: [1047] WSREP has not yet prepared node for application use
       30929:20160505:030034.394 Cannot connect to the database. Exiting...
       30936:20160505:030044.619 Starting Zabbix Server. Zabbix 3.0.2 (revision 59540).
       30936:20160505:030044.619 ****** Enabled features ******
       30936:20160505:030044.619 SNMP monitoring:           YES
       30936:20160505:030044.619 IPMI monitoring:           YES
       30936:20160505:030044.619 Web monitoring:            YES
       30936:20160505:030044.619 VMware monitoring:         YES
       30936:20160505:030044.619 SMTP authentication:       YES
       30936:20160505:030044.619 Jabber notifications:      YES
       30936:20160505:030044.619 Ez Texting notifications:  YES
       30936:20160505:030044.619 ODBC:                      YES
       30936:20160505:030044.619 SSH2 support:              YES
       30936:20160505:030044.619 IPv6 support:              YES
       30936:20160505:030044.619 TLS support:               YES
       30936:20160505:030044.619 ******************************
       30936:20160505:030044.619 using configuration file: /etc/zabbix/zabbix_server.conf
       30936:20160505:030044.625 [Z3001] connection to database 'zabbix' failed: [1047] WSREP has not yet prepared node for application use
       30936:20160505:030044.625 Cannot connect to the database. Exiting...
      
      ... restarts repeated here 
      

      note - server daemon started again automatically by systemd (by default).

      The solution for the issue might be very similar to the issue in ZBX-10753 but most likely it has to do with communication error between DB replications not stopped MySQL.

        Attachments

          Activity

            People

            Assignee:
            viktors.tjarve Viktors Tjarve
            Reporter:
            viktors.tjarve Viktors Tjarve
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved: