-
Type:
Problem report
-
Resolution: Unresolved
-
Priority:
Blocker
-
Affects Version/s: 7.0.28
-
Component/s: Agent2 (G)
-
None
-
Sprint candidates
Zabbix Agent 2 can permanently stop executing active checks after a single plain failed response to an active checks request.
The failure is silent, active check heartbeats keep working and the host stays Available.
Steps to reproduce
- Wait until Agent 2 has received its active checks and sends a non-zero config_revision.
- Reply to one active checks request with a plain failed response, for example:
{"response":"failed","info":"host [HOST] not found"} - On all later requests, reply success without configuration data. This is what proxy sends when the config_revision it receives is still current:
{"response":"success"}
Result:
Agent 2 reports that active checks are active again, but it does not execute or send any active checks. It stays in this state until the agent is restarted or the host configuration revision changes.
Expected:
A failed response should not permanently remove the active checks the agent already received.
Production occurrence
This happened during proxy group recovery. A restarted proxy had not received its configuration from the server yet:
20260801:103427.160 Starting Zabbix Proxy (active) [HOST_ZABBIX_PROXY_P1]. Zabbix 7.0.23 (revision f4175e9b3ce).
20260801:104014.485 received configuration data from server at "IP_0231", datalen 25720290
The host was assigned to P4, but it sent one active checks request to P1 while P1 was still without configuration. P1 replied with a plain failed response:
1202:20260801:103712.519 cannot send list of active checks to "IP_0395": host [HOST_0387] not found
2026/08/01 10:37:12.519128 [101] no active checks on server [IP_ZABBIX_PROXY_P1:10199]: host [HOST_0387] not found
2026/08/01 10:37:12.519768 [101] active checks on server started to fail
One refresh later, Agent 2 reported active checks as recovered:
2026/08/01 10:38:12.525778 [101] active checks on server are active again
After that the agent kept sending requests to P4 with the old config_revision, and P4 replied success without sending the active check list again:
1517:20260801:105957.482 trapper got '{"request":"active checks","host":"HOST_0387",...,"session":"26762de84f12bf9e9ae009d781db9d2c","config_revision":59091,...}' 1517:20260801:105957.485 End of get_hostid_by_host_or_autoregister():SUCCEED 1517:20260801:105957.485 send_list_of_active_checks_json() sending [{"response":"success"}]
get_hostid_by_host_or_autoregister() returns SUCCEED, so the proxy finds the host without any problem at this point.
No active check data was collected after that. The state lasted about 45 hours and the host stayed Available the whole time. Collection started again as soon as an active item was modified, which changed the host configuration revision. Agent 2 was not restarted.
Suspected cause
On a non-success response, Agent 2 calls UpdateTasks() with empty task lists but keeps configRevision, so the proxy sees an unchanged revision on the next request and replies success without data:
if response.Response != "success" { ... c.taskManager.UpdateTasks(c.clientID, c.resultCache.(resultcache.Writer), c.firstActiveChecksRefreshed, []*glexpr.Expression{}, []*scheduler.Request{}, now) c.firstActiveChecksRefreshed = true return false }
Any non-success response clears the task list, but the agent only gets stuck if configRevision is non-zero. An agent that still sends 0 gets the full list and recovers normally.
One note from my analysis, zabbix_agentd is not affected. In parse_list_of_checks() a non-success response is logged and returns FAIL without touching the active metrics list, so the agent keeps collecting the checks it already has.
if (0 != strcmp(tmp, ZBX_PROTO_VALUE_SUCCESS)) { ... goto out; }
- caused by
-
ZBX-27495 Agent can stuck in 'paused history upload'
-
- Closed
-