[ZBX-23541] Issues with queue and timestamp when Zabbix server and proxy are in different timezones Created: 2023 Oct 12  Updated: 2024 Apr 25  Resolved: 2024 Jan 23

Status: Closed
Project: ZABBIX BUGS AND ISSUES
Component/s: Documentation (D)
Affects Version/s: 6.0.22, 6.4.7, 7.0.0alpha6
Fix Version/s: 7.0 (plan)

Type: Documentation task Priority: Trivial
Reporter: Elvijs Bokanovs Assignee: Dmitrijs Goloscapovs
Resolution: Fixed Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: PNG File Screenshot 2023-10-13 at 13.59.40.png     PNG File Screenshot 2023-10-13 at 14.00.05.png     PNG File Screenshot 2023-10-13 at 14.00.34.png     PNG File Screenshot 2023-10-13 at 14.00.57.png     File ZBX-23541.yaml     PNG File image-2023-10-12-17-19-51-092.png     PNG File image-2023-10-12-17-20-58-827.png     PNG File image-2023-10-12-17-21-41-769.png     PNG File image-2023-10-12-17-22-08-516.png     PNG File image-2023-10-12-17-22-41-264.png     PNG File image-2023-10-12-17-23-01-403.png     PNG File image-2023-10-12-17-23-25-787.png     PNG File image-2023-10-12-17-23-49-848.png     PNG File image-2024-04-25-15-37-54-119.png    
Issue Links:
Duplicate
Team: Team A
Sprint: Sprint 105 (Oct 2023), Sprint 106 (Nov 2023), S2401-1, S2401-2
Story Points: 0.5

 Description   

Bug report guide is not available - https://zabbix.org/wiki/Docs/bug_reporting_guidelines, it redirects to Zabbix Community Area.

If Zabbix server and proxy servers are on different time zones (at least 1h interval between them) then there is issue with queue and data processing. Affected are items who has set  "Update interval" to 0 and "Custom intervals" is set to "Scheduling" with interval greater than 1h between checks, for example "wd1-7h0-23/2". Item type is "Zabbix agent", haven't tried with other item types.

Steps to reproduce:

  1. Set up server and proxy in different time zones - at least 1h difference.
    1. 1xZabbix Server - "Europe/Oslo", PHP-FPM timezone "Europe/Riga".
      1xZabbix Proxy - "Europe/Riga". 
      Databases on both servers are PostgreSQL 14 and their time zones are "Europe/Riga".
  2. Add one host, that gets monitored through server and other through proxy.
  3. Add few items to both hosts with different custom scheduling intervals "wd1-7h0-23m0-59/5","wd1-7h0-23/2","wd1-7h0-23/2","wd1-7h0-23/3","wd1-7h0-23/4" and with update interval set to 0.
    1. With the first to scheduling intervals there are no issues. With others however it get's interesting.
  4. Wait till data get's gathered.
  5. Check "Latest data" and "Queue details" for inconsistencies.

Result:
Latest data:

Queue:

Item configuration:

Expected:
Item execution time get's synchronized between Proxy and Server.

Queue uses same time (Zabbix server or Epoch) for item processing - not important if item get's monitored by Zabbix server or proxy servers in different time zones.

If neither of previous is possible and this behavior is by design, then please mention workaround in Zabbix official documentation. 

 Comments   
Comment by Elvijs Bokanovs [ 2023 Oct 12 ]

In user settings in GUI for timezone also are "Europe/Riga".

Comment by Edgar Akhmetshin [ 2023 Oct 13 ]

Hello Elvijs,

Thank you for a such detailed issue description.

OS:

head -n 2 /etc/os-release 
NAME="Rocky Linux"
VERSION="9.2 (Blue Onyx)"

Timezone:

[root@R9VM2]# timedatectl 
               Local time: Fri 2023-10-13 12:58:43 CEST
           Universal time: Fri 2023-10-13 10:58:43 UTC
                 RTC time: Fri 2023-10-13 10:58:44
                Time zone: Europe/Oslo (CEST, +0200)

[root@rl9vm3]# timedatectl 
               Local time: Fri 2023-10-13 13:59:04 EEST
           Universal time: Fri 2023-10-13 10:59:04 UTC
                 RTC time: Fri 2023-10-13 10:59:05
                Time zone: Europe/Riga (EEST, +0300)

Database (Server/Proxy):

mysqld -V
/usr/libexec/mysqld  Ver 8.0.32 for Linux on aarch64 (Source distribution)

MySQL settings:

default-time-zone='Europe/Riga'
default-authentication-plugin=mysql_native_password

# upload TZ data first:
# mysql_tzinfo_to_sql /usr/share/zoneinfo | mysql -u root -p mysql

Queue:

Latest data:

Proxy:

Hosts:

Template:
ZBX-23541.yaml

Confirmed.

Regards,
Edgar

Comment by dimir [ 2023 Oct 18 ]

FYI, coming from https://www.zabbix.com/forum/zabbix-help/472002-does-timezone-differences-between-zabbix-server-and-proxy-matter

Comment by Alex Kalimulin [ 2023 Oct 25 ]

We need to document that proxies and agent2 use their local time zones when processing scheduling intervals. Thus, to avoid incorrect queue reporting, we recommend running such proxies and agents with TZ env variable set to the server's time zone. The docs should provide an example systemd configuration.

It's worth considering passing server time zone to proxies and agents and/or custom time zones per scheduling intervals in the future, see ZBXNEXT-8789.

Comment by Marianna Zvaigzne [ 2024 Jan 18 ]

Documentation updated:

  • Configuration >
    • Items > Custom intervals > Aligning time zones for proxies and agent 2 (section added) (6.0, 6.4, 7.0)
    • Queue (See also added) (6.0, 6.4, 7.0)
  • Web interface > Time zones (See also added) (6.0, 6.4, 7.0)
Comment by Tomas Pavlovic [ 2024 Apr 25 ]

Hi,

I have similar problem...

On OS Rocky9.3 installed active proxy and zabbix_agent2 - I have problem with hourly executing items by scheduled time defined from 8:00AM till 3PM  (Upd.interval=0 Cust.int.Scheduling=h8-15) - no data came after 8:00 and Preprocessing is emty. 
Have I problem with configuration of agent2 startup environment in file /usr/lib/systemd/system/zabbix-agent2.service ?

(which now contains) » 
[Unit]
Description=Zabbix Agent 2
After=syslog.target
After=network.target

[Service]
Environment="CONFFILE=/etc/zabbix/zabbix_agent2.conf"
Environment="TZ=Europe/Bratislava"
EnvironmentFile=-/etc/sysconfig/zabbix-agent2
Type=simple
Restart=on-failure
PIDFile=/run/zabbix/zabbix_agent2.pid
KillMode=control-group
ExecStart=/usr/sbin/zabbix_agent2 -c $CONFFILE
ExecStop=/bin/kill -SIGTERM $MAINPID
RestartSec=10s
User=zabbix
Group=zabbix
LimitNOFILE=8192

[Install]
WantedBy=multi-user.target

 

On ZBX frontend, command line ZBX server and ZBX proxy is actual time an timezone, but problem is persisting - data for item with custom scheduled time doesn't collect. 

PostgreSQL 15 has time zone "Europe/Bratislava" too.

 

Time zone on ZBX frontend:

 

 

Timedatectl on ZBX server:

[root@ZBXserver~]$ timedatectl
Local time: Thu 2024-04-25 15:38:36 CEST
Universal time: Thu 2024-04-25 13:38:36 UTC
RTC time: Thu 2024-04-25 13:38:36
Time zone: Europe/Bratislava (CEST, +0200)
System clock synchronized: yes
NTP service: active
RTC in local TZ: no

Timedatectl on ZBX proxy:

[root@ZBXproxy~]$ timedatectl
Local time: Thu 2024-04-25 15:38:36 CEST
Universal time: Thu 2024-04-25 13:38:36 UTC
RTC time: Thu 2024-04-25 13:38:36
Time zone: Europe/Bratislava (CEST, +0200)
System clock synchronized: yes
NTP service: active
RTC in local TZ: no

Thanks.

Generated at Mon Jan 13 07:34:22 EET 2025 using Jira 9.12.4#9120004-sha1:625303b708afdb767e17cb2838290c41888e9ff0.