[ZBX-26560] Proxy cannot return to work after long network issue Created: 2025 Jun 12  Updated: 2025 Aug 18  Resolved: 2025 Aug 18

Status: Closed
Project: ZABBIX BUGS AND ISSUES
Component/s: Proxy (P), Server (S)
Affects Version/s: None
Fix Version/s: None

Type: Problem report Priority: Trivial
Reporter: Yurii Polenok Assignee: Unassigned
Resolution: Commercial support required Votes: 0
Labels: history, proxy, server, sync
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

7.2.5


Attachments: PNG File image-2025-06-12-17-49-11-767.png     PNG File image-2025-06-12-17-50-28-728.png    

 Description   

Steps to reproduce:

Break the network between the Zabbix Server and the Zabbix Proxy for several hours, then restore network connection.

Proxy conf:

Server=10.2.1.81;10.2.1.82
..
DataSenderFrequency=1
..
ProxyBufferMode=hybrid
ProxyConfigFrequency=5
ProxyMemoryBufferSize=10M
ProxyOfflineBuffer=12
..
StatsAllowedIP=127.0.0.1,192.168.1.76
..

Server conf:

SourceIP=10.2.1.81
..
CacheUpdateFrequency=5
..
StartProxyPollers=2
ProxyConfigFrequency=5
..
HANodeName=zabbix-01.exa.com
NodeAddress=10.2.1.81

Result:

Zabbix Server log:

  6890:20250612:043231.570 sending configuration data to proxy "zabbix-proxy-01.ex.com" at "192.168.1.76", datalen 17564, bytes 3579 with compression ratio 4.9
  6989:20250612:073320.906 Proxy "zabbix-proxy-01.ex.com" changed state from online to offline
  6989:20250612:121427.929 Proxy "zabbix-proxy-01.ex.com" changed state from offline to online
  6817:20250612:141630.944 reloading configuration on proxy "zabbix-proxy-01.ex.com"
Many lines like this:
  6864:20250612:144441.302 received id:25897580 is less than last id:26002109

Zabbix Proxy log:

903148:20250612:043231.600 received configuration data from server at "10.2.1.81", datalen 17564
...
903156:20250612:121409.962 Still unable to connect...
903156:20250612:121426.547 Connection restored.
903148:20250612:121430.967 Connection restored.
...
903156:20250612:145534.907 zbx_setproctitle() title:'data sender [sent 6 values in 0.225718 sec, sending data]'
...
903148:20250612:145658.159 received configuration data from server at "10.217.197.81", datalen 36
...903168:20250612:145936.271 zbx_setproctitle() title:'history syncer #1 [processed 22 values in 0.000674 sec, idle 1 sec]'
...
903177:20250612:150041.931 zbx_setproctitle() title:'self-monitoring [processed data in 0.000810 sec, idle 1 sec]'

zabbix[proxy,\{HOST.HOST},delay] item and trigger:
but at the same time:

That is, somehow the proxy and the server exchange information, but it seems that the server does not accept this data.
As a result, there is no new data from hosts behind this proxy.

Expected:
The connection between Zabbix Server and Zabbix Proxy should be restored, old data from the proxy cache and new data should arrive again.



 Comments   
Comment by Jan Prusinowski (Inactive) [ 2025 Jun 16 ]

Hello Yurii,

Would it be possible for you to provide logss (preferably with LogLevel=4) from server and proxy? Or the logs from the time that the DC occured?

Regards,

Jan

Comment by Yurii Polenok [ 2025 Jun 19 ]

The logs related to that issue during that time have been collected and have already been added to this task. Everything else in the logs was related to other running proxy servers. I didn't see anything else interesting at that time.

What is the reason of this log and how it can be fixed?

  6864:20250612:144441.302 received id:25897580 is less than last id:26002109

It seems that until the proxy writes all the old data to the database, it cannot transmit new data.

Comment by Edgar Akhmetshin [ 2025 Jun 20 ]

What is the reason of this log and how it can be fixed?

https://support.zabbix.com/browse/ZBX-26297
https://git.zabbix.com/projects/ZBX/repos/zabbix/commits/417281e6b9a#src/libs/zbxdbwrap/proxy.c

		/* last_valueid uniquely identifies each historical value (itemid/timestamp) generated by Zabbix      */
		/* agent for active items. It allows Zabbix server to detect and discard duplicates by comparing it   */
		/* to the last known last_valueid. It is always being incremented, so the situation when last_valueid */
		/* is smaller than the last known last_valueid represents a network anomaly and is logged.            */
Generated at Wed Apr 15 23:09:06 EEST 2026 using Jira 10.3.18#10030018-sha1:5642e4ad348b6c2a83ebdba689d04763a2393cab.