[ZBX-23456] Zabbix-Server service was hanging unlimited because database service was down at first to perform the reboot Created: 2023 Sep 22  Updated: 2023 Dec 22  Resolved: 2023 Dec 22

Status: Closed
Project: ZABBIX BUGS AND ISSUES
Component/s: None
Affects Version/s: 6.4.6
Fix Version/s: None

Type: Problem report Priority: Trivial
Reporter: Jerome Bose Assignee: Unassigned
Resolution: Commercial support required Votes: 0
Labels: crash, server, zabbix-server-mysql
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: PNG File Screenshot 2023-09-21 170323.png    

 Description   

Steps to reproduce:

  1. sudo apt update
  2. sudo apt upgrade
  3. sudo reboot

Result:
See screenshot...

See log file...

See memory dump...
Expected:

service from zabbix-server is running unlimited if this condition came up.

WARNING: MYSQL_OPT_RECONNECT is deprecated and will be removed in a future version.
  1408:20230921:163926.345 Got signal [signal:15(SIGTERM),sender_pid:2480380,sender_uid:0,reason:0]. Exiting ...
  1420:20230921:163927.004 [Z3005] query failed: [1053] Server shutdown in progress [select changelogid,object,objectid,operation,clock from changelog]
  1420:20230921:163927.191 [Z3001] connection to database 'zabbix' failed: [2002] Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.s>
  1420:20230921:163927.192 database is down: reconnecting in 10 seconds
  1452:20230921:163927.316 [Z3005] query failed: [2013] Lost connection to MySQL server during query [insert into history (itemid,clock,ns,value) values (8174>
]
  1455:20230921:163927.347 [Z3005] query failed: [2013] Lost connection to MySQL server during query [commit;]
  1455:20230921:163927.347 [Z3005] query failed: [2013] Lost connection to MySQL server during query [rollback;]
  1455:20230921:163927.347 cannot perform transaction rollback, connection will be reset
  1452:20230921:163927.348 [Z3001] connection to database 'zabbix' failed: [2002] Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.s>
  1452:20230921:163927.348 database is down: reconnecting in 10 seconds
  1455:20230921:163927.379 [Z3001] connection to database 'zabbix' failed: [2002] Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.s>
  1455:20230921:163927.379 database is down: reconnecting in 10 seconds
  1438:20230921:163927.660 [Z3005] query failed: [1053] Server shutdown in progress [select h.hostid,h.host,h.name,t.httptestid,t.name,t.agent,t.authenticatio>
  1442:20230921:163927.660 [Z3005] query failed: [1053] Server shutdown in progress [select h.hostid,h.host,h.name,t.httptestid,t.name,t.agent,t.authenticatio>
  1438:20230921:163927.660 [Z3001] connection to database 'zabbix' failed: [2002] Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.s>
  1438:20230921:163927.660 database is down: reconnecting in 10 seconds
  1442:20230921:163927.660 [Z3001] connection to database 'zabbix' failed: [2002] Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.s>
  1442:20230921:163927.660 database is down: reconnecting in 10 seconds
  1516:20230921:163927.817 [Z3005] query failed: [2013] Lost connection to MySQL server during query [select a.alertid,a.mediatypeid,a.sendto,a.subject,a.mess>
  1516:20230921:163927.910 [Z3001] connection to database 'zabbix' failed: [2002] Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.s>
  1516:20230921:163927.910 database is down: reconnecting in 10 seconds
  1420:20230921:163928.037 [Z3001] connection to database 'zabbix' failed: [2002] Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.s>
  1420:20230921:163928.037 database is down: reconnecting in 10 seconds
  1451:20230921:163928.037 syncing history data in progress...
  1442:20230921:163928.037 [Z3001] connection to database 'zabbix' failed: [2002] Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.s>
  1442:20230921:163928.037 database is down: reconnecting in 10 seconds
 

Workaround
Connect to console to VM/server and manually reboot the server. In managed environments this issue have a big impact because one ticket must be opened or the technician must be contact first to solve the corruption.



 Comments   
Comment by Jerome Bose [ 2023 Sep 22 ]

It came up to apply updates from Sept. 2023 on Ubuntu Server: 22.04.

-----------

58 updates can be applied immediately.
5 of these updates are standard security updates.
To see these additional updates run: apt list --upgradable

1 additional security update can be applied with ESM Apps.
Learn more about enabling ESM Apps service at https://ubuntu.com/esm


*** System restart required ***
.............
The following packages will be upgraded:
  bind9-dnsutils bind9-host bind9-libs dnsutils libapache2-mod-php7.4 libapache2-mod-php8.1 libapache2-mod-php8.2 libc-bin libc6 libcups2
  libnss-systemd libpam-systemd libsmbclient libsystemd0 libudev1 libwbclient0 locales openssh-client openssh-server openssh-sftp-server php7.4
  php7.4-cli php7.4-common php7.4-json php7.4-opcache php7.4-readline php8.1 php8.1-bcmath php8.1-cli php8.1-common php8.1-gd php8.1-ldap
  php8.1-mbstring php8.1-mysql php8.1-opcache php8.1-readline php8.1-xml php8.2 php8.2-bcmath php8.2-cli php8.2-common php8.2-gd php8.2-ldap
  php8.2-mbstring php8.2-mysql php8.2-opcache php8.2-readline php8.2-xml python3-samba samba-common samba-common-bin samba-dsdb-modules samba-libs
  systemd systemd-sysv systemd-timesyncd thermald udev
58 upgraded, 0 newly installed, 0 to remove and 1 not upgraded. 

....

Restarting services...
 /etc/needrestart/restart.d/systemd-manager
 systemctl restart apache2.service cron.service irqbalance.service multipathd.service mysql.service oddjobd.service packagekit.service polkit.service rsyslog.service snapd.service snmpd.service udisks2.service upower.service zabbix-agent.service zabbix-java-gateway.service zabbix-server.service 

After that I have initiated the reboot:

sudo reboot

In result:

SSH terminal kick and the only way is to connect via console direct to the server to solve the issue.

Comment by Oleksii Zagorskyi [ 2023 Dec 22 ]

Please be advised that this section of the tracker is for bug reports only. The case you have submitted can not be qualified as one, so please reach out to [email protected] for commercial support (https://zabbix.com/support) or consultancy services. Alternatively, you can also use our community channels (https://www.zabbix.com/community) for assistance. With that said, we are closing this ticket. Thank you for understanding.

Generated at Wed May 07 07:21:16 EEST 2025 using Jira 9.12.4#9120004-sha1:625303b708afdb767e17cb2838290c41888e9ff0.