Uploaded image for project: 'ZABBIX BUGS AND ISSUES'
  1. ZABBIX BUGS AND ISSUES
  2. ZBX-11228

Zabbix server terminates on MySQL restart

    XMLWordPrintable

    Details

    • Type: Incident report
    • Status: Closed
    • Priority: Critical
    • Resolution: Duplicate
    • Affects Version/s: 3.0.3, 3.0.4
    • Fix Version/s: None
    • Component/s: Server (S)
    • Labels:
      None
    • Environment:
      Debian

      Description

      Once in a while MySQL needs restart when a security patch is installed but every time when MySQL is restarted Zabbix server terminates:

       16584:20160919:132424.526 [Z3005] query failed: [2002] Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) [select escalationid,actionid,triggerid,eventid,r_eventid,nextcheck,esc_step,status,itemid from escalations where itemid is not null order by actionid,triggerid,itemid,escalationid]
       16584:20160919:132424.540 [Z3001] connection to database 'zabbix' failed: [2002] Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)
       16584:20160919:132424.572 Cannot connect to the database. Exiting...
       16583:20160919:132424.592 [Z3005] query failed: [2006] MySQL server has gone away [insert into history (itemid,clock,ns,value) values (62544,1474255464,578564042,75.676449),(48084,1474255464,580184457,67.923016);
      ]
       16583:20160919:132424.622 [Z3001] connection to database 'zabbix' failed: [2002] Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)
       16583:20160919:132424.641 Cannot connect to the database. Exiting...
       16559:20160919:132424.674 [Z3005] query failed: [1053] Server shutdown in progress [select hostid,key_,state,evaltype,formula,error,lifetime from items where itemid=30264]
       16559:20160919:132424.712 [Z3001] connection to database 'zabbix' failed: [2002] Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)
       16559:20160919:132424.714 Cannot connect to the database. Exiting...
       16582:20160919:132424.802 item "Zabbix server:mysql.status[Bytes_sent]" became not supported: Received value [ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)] is not suitable for value type [Numeric (float)]
       16582:20160919:132424.802 [Z3005] query failed: [2006] MySQL server has gone away [update items set state=1,error='Received value [ERROR 2002 (HY000): Can\'t connect to local MySQL server through socket \'/var/run/mysqld/mysqld.sock\' (2)] is not suitable for value type [Numeric (float)]' where itemid=23784;
      ]
       16580:20160919:132424.815 [Z3005] query failed: [2006] MySQL server has gone away [insert into history (itemid,clock,ns,value) values (57444,1474255464,812126377,9.089814);
      ]
       16580:20160919:132424.828 [Z3001] connection to database 'zabbix' failed: [2002] Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)
       16582:20160919:132424.828 [Z3001] connection to database 'zabbix' failed: [2002] Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)
       16580:20160919:132424.828 Cannot connect to the database. Exiting...
       16582:20160919:132424.828 Cannot connect to the database. Exiting...
       16553:20160919:132424.935 One child process died (PID:16584,exitcode/signal:1). Exiting ...
       16553:20160919:132426.974 [Z3001] connection to database 'zabbix' failed: [2002] Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)
       16553:20160919:132426.975 Cannot connect to the database. Exiting...
      

      This problem was introduced with 3.0.x (previously 2.4 could survive restart of MySQL). Clearly Zabbix Server should keep trying to re-establish connection with database for some time, ideally with configurable timeout. At the moment Zabbix server gives up on DB connectivity too fast so any restart of database (even if it takes seconds) reliably knock down Zabbix server.

      This issue may be somewhat related to ZBX-10753 (which discusses startup failure) or similar to ZBX-11004...

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              Unassigned Unassigned
              Reporter:
              onlyjob Dmitry Smirnov
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: