[ZBX-10695] Zabbix Agent on Windows won't start after server reboot Created: 2016 Apr 21  Updated: 2025 Apr 26

Status: Open
Project: ZABBIX BUGS AND ISSUES
Component/s: Agent (G)
Affects Version/s: 3.0.1, 3.0.2
Fix Version/s: None

Type: Incident report Priority: Trivial
Reporter: Jérôme Queneuder Assignee: Unassigned
Resolution: Unresolved Votes: 7
Labels: windows
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

OS : Windows 2008 R2 SP1


Attachments: File netlogon_dependency.reg     HTML File zabbix-log     File zabbix_agentd.conf    
Issue Links:
Duplicate

 Description   

The zabbix agent won't start after a normal start of the server on Windows 2008 R2. The issue is global of all Windows servers. The log of Zabbix agent is not written by executable (tested with debug level 5), the delayed windows service functionnality not help. When the Zabbix agent service want to start, Windows log in event log an EventID 7000 :
The Zabbix Agent service failed to start due to the following error:
The service did not respond to the start or control request in a timely fashion.

and an EventID 7009 :
A timeout was reached (60000 milliseconds) while waiting for the Zabbix Agent service to connect.

But the timeout was up to 60000ms from 30000ms and the issue still here.

In addition, the service start without problem if you start it manualy

I attach my zabbix agent conf file.

Thank you for support !



 Comments   
Comment by Aleksandrs Saveljevs [ 2016 Apr 25 ]

Did it work well with a previous version of Zabbix? If so, how did you perform the upgrade? Did you simply overwrite the binaries or uninstalled and installed the services, too?

Comment by Jérôme Queneuder [ 2016 Apr 25 ]

Dear Aleksandrs Saveljevs,

After your comment, I installed the version Zabbix 2.4.7 (revision 56682) and the server will be rebooted this night. I keep you informed tomorrow. In addition, it's a fresh Zabbix installation, no upgrade and a topic is opened by another person in the forum : https://www.zabbix.com/forum/showthread.php?t=52868

It's the same issue.

Thank you Aleksandrs for your help,

Jérôme

Comment by Jérôme Queneuder [ 2016 Apr 26 ]

Hello Aleksandrs Saveljevs,

I have the same issue with the version Zabbix 2.4.7 (revision 56682). Do you know if it's possible to have a specific version with debug mode at startup ? But the zabbix_agentd executable can't write log file because the issue start before than the agent can access to the log file (tested with debug level 5).

Thank you for your support !

Comment by Aleksandrs Saveljevs [ 2016 Apr 27 ]

Could you please show how you installed the agent and, maybe, a screenshot of the service configuration?

When you successfully start it manually, do you start it from the command line or through "Services" window?

There is the following option in the configuration file:

LogFile=C:\Program Files\Zabbix Agent\zabbix_agentd.log

Does the agent have permissions to write there when started at boot time?

There is also the following option:

HostnameItem=system.run[powershell -NonInteractive "[System.Net.Dns]::GetHostByName(($env:computerName)).Hostname.ToLower()"]

Does the agent start with this line removed?

Could it be that some third-party software like an antivirus prevents the agent from starting?

(Also, note that you can use the Windows-specific configuration file template "conf/zabbix_agentd.win.conf" instead of the Unix version "conf/zabbix_agentd.conf".)

Comment by Jérôme Queneuder [ 2016 Apr 28 ]

Hello Aleksandrs Saveljevs,

You can find a screenshot of my configuration here : http://www.hostingpics.net/viewer.php?id=252460Terminals20160428165435.png

I have installed the agent with this command line zabbix_agentd.exe --config C:\Program Files\Zabbix Agent\zabbix_agentd.conf --install with success. I followed the documentation here https://www.zabbix.com/documentation/3.0/manual/appendix/install/windows_agent

I start manually the service with service panel on windows and not manually by CLI.

Yes, the agent can write the log file without problem because the security right are not modified.

I will try to start the agent without system.run but I need it to retrieve FQDN because I use discovery process on Zabbix Server. Do you have another solution to fill the hostname parameter with fqdn without to edit each config file ?

I looked the antivirus logs but nothing find about Zabbix agent.

Thank you for your support !

Comment by Jérôme Queneuder [ 2016 May 01 ]

Hello Aleksandrs Saveljevs,

After 3 reboots without option HostnameItem=system.run it works good at startup . But, do you know a method to fill automatically the option Hostname= with FQDN ? Because the discovery process on zabbix server needs the fqdn of server agent to create the host.

Thank you again for your support !

Comment by Aleksandrs Saveljevs [ 2016 May 02 ]

Jérôme, thanks for info! Personally, I am not aware of a different method to fill Hostname with FQDN, but you may wish to ask on https://www.zabbix.org/wiki/Getting_help - there are many experienced community members.

As pointed out by zalex_ua, if HostnameItem fails for some reason, we may wish to log this somehow (e.g., if we do not have the log file open yet and we are not running on the console, can we log errors to Event Log by default on Windows?). Therefore, setting this issue to "Open".

Comment by Jérôme Queneuder [ 2016 May 02 ]

OK Aleksandrs. I have activated logging in the windows event log with debug=5 option. I keep you informed.

Thank you !

Jérôme

Comment by Oleksii Zagorskyi [ 2016 May 02 ]

It reminds me about ZBX-8893, although I'm not sure it's relevant in current context.

Comment by Jérôme Queneuder [ 2016 May 04 ]

Hello Aleksandrs and Oleksiy,

I have tested with my previous parameter

HostnameItem=system.run[powershell -NonInteractive "[System.Net.Dns]::GetHostByName(($env:computerName)).Hostname.ToLower()"]

And the system event log activated and debug level 5 in zabbix config file but the zabbix agent don't log an error after host reboot. The error message is the same

EventID 7009 :
A timeout was reached (60000 milliseconds) while waiting for the Zabbix Agent service to connect.

Could think if it possible to add some options at installation of the zabbix service like "-netbiosname 1" (activate netbios name for hostname parameter) and "-fqdn 1" (activate fqdn for hostname parameter) ?

Thank you,

Jérôme

Comment by Jérôme Queneuder [ 2016 May 18 ]

Dear all,

Do you have an update for me ?

Thank you,

Jérôme Queneuder

Comment by Aleksandrs Saveljevs [ 2016 May 23 ]

Jérôme, regarding what can be done on Zabbix side, we are leaving this issue open to provide some logging in case service startup fails or maybe provide a more permissive timeout for HostnameItem.

Regarding your question about obtaining FQDN in the meanwhile and assuming "system.hostname[host]" item does not work for you, it might be better to direct this question to experienced Zabbix users on IRC or forum, as mentioned at https://www.zabbix.org/wiki/Getting_help .

Comment by Reece Webb [ 2016 May 23 ]

I have this identical problem, same timeout error, though it's not across the board and is inconsistent... one reboot will have the error, the next won't. Having it with Windows 2008R2 and 2012R2.

Agent 3.0.0 (revision 58455). Installed with "C:\Zabbix\bin\zabbix_agentd.exe --config C:\Zabbix\conf\zabbix_agentd.win.conf --install", the service is started automatically at boot. We tried putting in a delayed start, but that hasn't helped. Permissions are correct, nothing is written to the Zabbix log when this happens. I have poured through the Eventlog, with debugging on, and nothing is jumping out at me.

I'm going to try adding a dependency to a late-starting service, just in case some other service is taking too long to start and is clogging things up.

Comment by Jérôme Queneuder [ 2016 May 26 ]

Hello Reece,

Could you try to add a dependency with Remote desktop service ? Because it's a last service who start on a Windows server .

Thank you for your test, we will find a solution, I hope ...

Jerome

Comment by Daniel V [ 2016 Sep 21 ]

I had this issue too on Windows Server 2008 R2. Just someday zabbix agent stopped working with the service control telling me this:

[SC] StartService FEHLER 1053:

Der Dienst antwortete nicht rechtzeitig auf die Start- oder Steuerungsanforderung.

By removing parts of my zabbix_agentd.conf I was able to find that the following caused the error:

HostnameItem=system.run['echo %COMPUTERNAME%.%USERDNSDOMAIN%']

By the way: Starting the zabbix_agentd.exe by using the cmd console worked fine!

Why can't there be a config key to tell the zabbix agent to use his hostname and domain as agent.hostname?

Comment by Jérôme Queneuder [ 2016 Sep 21 ]

Hello Daniel,

At our side, we have solved this situation by using the zabbix agent from suiviperf.com because we have requested a modification to include the possibility use FQDN or NETBIOS name

HOSTNAMEFQDN=1 : Hostname in FQDN in low case
HOSTNAMEFQDN=0 : Hostname in NETBIOS in low case

Thank you !

Comment by Oleksii Zagorskyi [ 2016 Sep 21 ]

Windows installation packages on suiviperf.com cannot do anything better regarding discussed issue.
Zabbix agent there is the same, but maybe hostname value is "hardcoded" in the config file, instead of getting it dynamically on agent start using HostnameItem.

Comment by Daniel V [ 2016 Sep 21 ]

I don't see why I should use a modified zabbix agent. Rather than forking the original code should be fixed.

I assume the zabbix agent crashes in my case, since nothing is written into the log file. Is it possible to find out why the zabbix agent crashes and prevent it from crashing, to be able to write into the log file what is wrong? Even with debug level of 4 or 5 nothing is written into the log file.

Comment by Oleksii Zagorskyi [ 2016 Sep 21 ]

Agents form suiviperf.com - should be original, not modified. But they are packaged to msi/exe installation files.

I don't think zabbix agent crashes, it just refuses to start or say, exits on start.
In such case some error message is logged to console (usually), not to log file.
Manual starting agent in CMD window may show that error message.

Comment by Daniel V [ 2016 Sep 21 ]

Ok but how do I get that error message? When running zabbix in a console (cmd.exe) then it just warns about running in a console. The error seems only to occur, when the zabbix agent is started by the windows service management.

I tried to use this to log any zabbix output when started by the service management. Sadly the file wasn't created.

sc config "Zabbix Agent" binpath= "C:\zabbix\zabbix_agentd.exe --config C:\zabbix\zabbix_agentd.conf >> C:\zabbix\zabbix_agentd-startup.log"

Can I force zabbix to spawn a console window?

Comment by Gian [ 2016 Oct 20 ]

Hi everyone

Same issue over here...
We're using Zabbix Agent on Windows Server 2012R2, some hosted on ESXi 6.0 and some running on physical machines.
The Zabbix Agent works perfectly on all virtual servers and starts up automatically after reboot.

But we have the same problem as described on all physical servers.
We use exactly the the same setup and zabbix_agentd.conf, containing "HostnameItem=system.run[powershell $env:COMPUTERNAME.ToLower()]".
If I try to start the Zabbix Agent Service manually, I get "Error 1503: <...>" and if I try it instantly again the service starts as expected.

I just cannot imagine why it ain't working on the physical servers too......

Excuse my english, I know it's not perfect at all but I guess it will do it for now

Gian

Comment by Reece Webb [ 2016 Oct 25 ]

We're still having this issue, but have found a workaround by adding a dependency on Netlogon to the "Zabbix Agent" service. Haven't had a timeout since. Registry key is attached.

Comment by Oleksii Zagorskyi [ 2016 Oct 26 ]

I personally consider Reece's workaround as a correct approach for those who experience such issues.
But I'd probably not agree to implement such dependency to zabbix agent service installation by default.
Maybe additional command line option for service installation could be developed.

Comment by Janne Korkkula [ 2016 Oct 27 ]

We're experiencing this issue too, after configuring a system.run powershell hack to get the server FQDN, which is of course required for active monitoring. Thank you Reece, we'll adopt that fix if it turns out to work.

The real fix would be for the Windows Zabbix agent to provide an agent-internal item with the host FQDN (in lower case). Adopting all these powershell hacks is just silly, but we're practically forced to do it...

PS. here's our hack:

HostnameItem=system.run[powershell -NonInteractive "[System.Net.Dns]::GetHostByName(($env:computerName)).Hostname.ToLower()"]
Comment by Janne Korkkula [ 2016 Nov 11 ]

Our Windows guys found an even more horrific hack, which just may work, from an older issue: ZBXNEXT-2106

I think that issue should be reconsidered and reopened.

Comment by Christophe Chapuis [ 2017 Nov 06 ]

Hello, we're also experiencing the same behavior by using :

HostnameItem=system.run[powershell -NonInteractive "[System.Net.Dns]::GetHostByName(($env:computerName)).Hostname.ToLower()"]

Environment :

  • Zabbix Server 3.4.2
  • Zabbix agent 3.4.2 and 3.4.3 on windows 2008 R2 and 2012 R2

+1 to get a solution that will provide FQDN in lowercase as Hostname

Cheers

Christophe

Comment by Jérôme Queneuder [ 2017 Nov 06 ]

Hi Christophe,

I have asked the developer of SuiviPerf.com to add an option to set fqdn as hostname in the installer
http://www.suiviperf.com/zabbix/index.php

The option is :
HOSTNAMEFQDN=1 : Hostname in FQDN in low case

Many thanks to suiviperf.com dev

Jerome

Comment by Max N [ 2018 Oct 22 ]

I came across this very same problem... its a real PITA to use zabbix with autoregistration/discovery with windows agents. There isliterally no way to get an all lowercase hostnameitem on windows, because every time one calls a powershell, the agent will fail with

HostnameItem=system.run[powershell -NonInteractive [System.Net.Dns]::GetHostByName(($env:computerName)).Hostname.ToLower()]

zabbix_agentd.exe [1960]: Warning: failed to get system hostname from [system.run[powershell -NonInteractive [System.Net.Dns]::GetHostByName(($env:computerName)).Hostname.ToLower()

 

What indeed works is using bat commands only:

# HostnameItem=system.run[echo %COMPUTERNAME%.%USERDNSDOMAIN%]

but this returns an FQDN without the guarantee for an all lowercase string, which is basically impossible to do in bat scripting.

 

IMHO it is currently not possible to do a large scale autoregister process where active checks on Windows hosts are a requirement and you wont have some hacked workaround to hardwire hostnames into the deployed zabbix_agentd.conf on windows hosts.

 

Possible way to fix this in zabbix might be:

Provide an option in zabbix_agentd.conf to use systems FQDN for the hostname (or even use it per default)

Make autodiscovery use the hostname instead of the FQDN to create hosts

A new, working powershell key to generate an FQDN in PS for hostnameitem - ZBXNEXT-1101

Make Hostnames case-insesitive in Zabbix-Server. I know, technically those names are not DNS, but RFC 4303 declares DNS names explicitly as case insensitive.

 

Comment by Reece Webb [ 2018 Oct 22 ]

To get lowercase hostnames I do a two-step process. Not perfect, but with our automated agent installs it has been working well for us.

zabbix_agentd.win.conf:

HostnameItem=system.run[powershell -NonInteractive "C:\Zabbix\scripts\hostname.ps1"]

hostname.ps1:

"$env:computerName".toLower()

Comment by HoaiDuyen [ 2018 Oct 31 ]

thank you so much !!!

Generated at Tue Apr 29 08:35:01 EEST 2025 using Jira 9.12.4#9120004-sha1:625303b708afdb767e17cb2838290c41888e9ff0.