Summary

Zabbix Server occasionally hangs indefinitely while stopping/restarting via systemd.

The service remains in:

Active: deactivating (stop-sigterm)

for multiple days.

Because the packaged systemd unit contains:

TimeoutSec=infinity

systemd never terminates the stuck processes.

In this particular case the problem was triggered during an automatic package upgrade. needrestart executed a restart of several services including zabbix-server.service. Zabbix Server did not finish stopping, which consequently caused needrestart, unattended-upgrades, and the APT/dpkg lock to remain blocked for more than two days.

This appears similar to previously reported issues ZBX-22097 / ZBX-22122, although those issues were marked as fixed.

It may also be related to the more recent ZBX-27804, which reports another Zabbix Server shutdown hang on Zabbix 7.4.9.

Environment

OS: Ubuntu 24.04 LTS (Noble)

Database: MySQL 8.0

Zabix-Server: 7.0.30 LTS

Init system: systemd

Please see the attached diagnostic output for the exact Zabbix version and package versions.

What happened

During package upgrades, needrestart executed:

systemctl restart atop.service mysql.service nginx.service packagekit.service php8.3-fpm.service polkit.service rsyslog.service snmpd.service ssh.service udisks2.service upower.service zabbix-agent.service zabbix-server.service

Zabbix Server entered shutdown at:

Sep 02 06:47:15 systemd[1]: Stopping zabbix-server.service - Zabbix Server...

and remained in this state for more than two days:

{{Active: deactivating (stop-sigterm)
SubState=stop-sigterm}}

The Zabbix systemd unit reported:

{{TimeoutStartUSec=infinity
TimeoutStopUSec=infinity}}

MySQL was successfully stopped:

{{Sep 02 06:47:18 systemd[1]: mysql.service: Deactivated successfully.
Sep 02 06:47:18 systemd[1]: Stopped mysql.service - MySQL Community Server.}}

However, Zabbix Server never completed shutdown.

Most Zabbix subprocesses showed [terminated], but several processes remained alive, including:

{{zabbix_server: ha manager
zabbix_server: service manager #1
zabbix_server: history syncer #1
zabbix_server: history syncer #2
zabbix_server: history syncer #3
zabbix_server: history syncer #4
zabbix_server: configuration syncer worker}}

There were also several zombie processes:

[zabbix_server] <defunct>

The main Zabbix Server process also remained alive.

systemd state

{{ActiveState=deactivating
SubState=stop-sigterm
MainPID=53710
ExecMainPID=53710
TimeoutStartUSec=infinity
TimeoutStopUSec=infinity}}

The systemd job queue contained:

{{mysql.service start waiting
apt-daily-upgrade.service start running
zabbix-server.service restart running}}

As a result, the pending Zabbix restart also prevented MySQL from being started again.

Expected result

systemctl stop zabbix-server or systemctl restart zabbix-server should terminate Zabbix Server cleanly within a reasonable amount of time.

If a Zabbix subprocess cannot terminate correctly, the shutdown should not remain blocked indefinitely.

Actual result

Zabbix Server remains indefinitely in:

deactivating (stop-sigterm)

and systemd waits forever because the packaged unit specifies:

TimeoutSec=infinity

This can have additional consequences when the restart is initiated by package management:

{{Zabbix shutdown hangs
↓
systemctl restart does not return
↓
needrestart does not return
↓
unattended-upgrades does not finish
↓
APT/dpkg lock remains held indefinitely}}

In this case the condition persisted for more than two days and normally appeared to require a full OS reboot to recover.

Previous occurrences

The issue appears related to:

• ZBX-22097 — HA manager is not stopped if Zabbix Server is stopped with unavailable database

• ZBX-22122 — Systemctl restart zabbix-server hangs

• ZBX-15602 — SystemD TimeoutSec=infinity

• ZBX-27804 — Zabbix server shutdown blocked by awaiting SNMP poller

ZBX-22097 was marked as fixed in older Zabbix releases, but a similar shutdown deadlock still appears to be possible in current installations.

The current case also differs from ZBX-22097 because several Zabbix processes remain alive, not only the HA manager.

Questions

1. Is this a known shutdown deadlock in current Zabbix Server versions?

1. Which remaining Zabbix process should normally coordinate termination of history syncer, configuration syncer worker, service manager, and ha manager?

1. What additional debugging information would be useful when the server is caught in this state?

1. Is TimeoutSec=infinity still considered safe for the packaged systemd unit?

1. Would it be reasonable for the official package to use a finite TimeoutStopSec, while keeping a longer timeout for normal history/cache synchronization?

1. Could repeated service restarts initiated by needrestart expose a race condition during Zabbix Server shutdown?