-
Type:
Problem report
-
Resolution: Unresolved
-
Priority:
Trivial
-
None
-
Affects Version/s: 7.4.15
-
Component/s: Server (S)
-
None
Environment
-----------
- Zabbix server 7.4.15, native HA cluster (2 nodes, 1 active / 1 standby), Ubuntu
- Database: MySQL 8 (Azure Database for MySQL Flexible Server)
- Zabbix frontend 7.4.15
- Agents: zabbix_agent2 7.4.16 (Linux: Ubuntu, RHEL 9, SLES 15 SP4/SP6)
- Agent config: ServerActive=<node1-fqdn>;<node2-fqdn> (HA cluster syntax), HeartbeatFrequency default (60s)
- ~3500 enabled hosts, part monitored directly by the server, part through 7.4 proxies
Problem
-------
On hosts monitored directly by the server (no proxy), the active checks availability
(host_rtdata.active_available) does not follow the real agent state:
1) Hosts stuck in "Not available" (active_available=2) while the agent is running, sending
heartbeats and active check data normally. ~57 hosts affected.
2) Hosts stuck in "Unknown" (active_available=0) although they run agent >= 6.4 (7.4.16)
and send heartbeats. ~33 hosts affected, including 15 upgraded today from agent < 6.4
to 7.4.16 that never left "Unknown".
Active check data from the same hosts IS received and stored (e.g. hourly system.run item
values are present in history), so only the availability state is wrong.
Hosts monitored through proxies are not affected. Some hosts monitored directly by the
server do transition correctly (0 -> 1 and 2 -> 1), so the issue is not systematic.
Steps to reproduce (observed on one host)
-----------------------------------------
1. Host monitored directly by the server, agent2 7.4.x, active items present, state = available (1).
2. Stop the agent: after the heartbeat timeout the server sets active_available=2 (correct).
3. Start the agent again.
4. Agent log shows successful active check configuration refresh and data sending.
5. Server trapper (DebugLevel 4 on trapper) logs: trapper got request 'active check heartbeat'
for the host every 60s.
6. active_available stays 2 indefinitely; availability manager (DebugLevel 4) never writes
the update back to 1 for this hostid.
Note: host_rtdata already had active_available=2 before the stop (11:42), yet the availability
manager wrote "active_available=2" again at 11:48 when the agent was stopped. Its in-memory state
was therefore out of sync with host_rtdata, and it never writes 1 once heartbeats resume.
See attached log excerpt.
Already tried (no effect)
-------------------------
- zabbix_server -R config_cache_reload
- unlinking / relinking the template containing the active items
- verified Hostname / ServerActive, no "host not found" errors, no duplicate host names
Expected
--------
active_available returns to 1 (available) when heartbeats resume, and goes from 0 to 1
when a heartbeat-capable agent (>= 6.4) starts sending heartbeats.
Actual
------
State remains 2 (or 0) until (assumed) server restart. Effect: false "Active checks not
available" problems, and missing detection of real active-agent outages for hosts in "Unknown".
Additional info
---------------
Server and availability manager debug logs can be provided on request.
Planned: upgrade to 7.4.16 (server restart) to check whether the state realigns.