[ZBX-10695] Zabbix Agent on Windows won't start after server reboot Created: 2016 Apr 21 Updated: 2025 Apr 26 |
|
Status: | Open |
Project: | ZABBIX BUGS AND ISSUES |
Component/s: | Agent (G) |
Affects Version/s: | 3.0.1, 3.0.2 |
Fix Version/s: | None |
Type: | Incident report | Priority: | Trivial |
Reporter: | Jérôme Queneuder | Assignee: | Unassigned |
Resolution: | Unresolved | Votes: | 7 |
Labels: | windows | ||
Remaining Estimate: | Not Specified | ||
Time Spent: | Not Specified | ||
Original Estimate: | Not Specified | ||
Environment: |
OS : Windows 2008 R2 SP1 |
Attachments: |
![]() ![]() ![]() |
||||
Issue Links: |
|
Description |
The zabbix agent won't start after a normal start of the server on Windows 2008 R2. The issue is global of all Windows servers. The log of Zabbix agent is not written by executable (tested with debug level 5), the delayed windows service functionnality not help. When the Zabbix agent service want to start, Windows log in event log an EventID 7000 : and an EventID 7009 : But the timeout was up to 60000ms from 30000ms and the issue still here. In addition, the service start without problem if you start it manualy I attach my zabbix agent conf file. Thank you for support ! |
Comments |
Comment by Aleksandrs Saveljevs [ 2016 Apr 25 ] |
Did it work well with a previous version of Zabbix? If so, how did you perform the upgrade? Did you simply overwrite the binaries or uninstalled and installed the services, too? |
Comment by Jérôme Queneuder [ 2016 Apr 25 ] |
Dear Aleksandrs Saveljevs, After your comment, I installed the version Zabbix 2.4.7 (revision 56682) and the server will be rebooted this night. I keep you informed tomorrow. In addition, it's a fresh Zabbix installation, no upgrade It's the same issue. Thank you Aleksandrs for your help, Jérôme |
Comment by Jérôme Queneuder [ 2016 Apr 26 ] |
Hello Aleksandrs Saveljevs, I have the same issue with the version Zabbix 2.4.7 (revision 56682). Do you know if it's possible to have a specific version with debug mode at startup ? But the zabbix_agentd executable can't write log file because the issue start before than the agent can access to the log file (tested with debug level 5). Thank you for your support ! |
Comment by Aleksandrs Saveljevs [ 2016 Apr 27 ] |
Could you please show how you installed the agent and, maybe, a screenshot of the service configuration? When you successfully start it manually, do you start it from the command line or through "Services" window? There is the following option in the configuration file: LogFile=C:\Program Files\Zabbix Agent\zabbix_agentd.log Does the agent have permissions to write there when started at boot time? There is also the following option: HostnameItem=system.run[powershell -NonInteractive "[System.Net.Dns]::GetHostByName(($env:computerName)).Hostname.ToLower()"] Does the agent start with this line removed? Could it be that some third-party software like an antivirus prevents the agent from starting? (Also, note that you can use the Windows-specific configuration file template "conf/zabbix_agentd.win.conf" instead of the Unix version "conf/zabbix_agentd.conf".) |
Comment by Jérôme Queneuder [ 2016 Apr 28 ] |
Hello Aleksandrs Saveljevs, You can find a screenshot of my configuration here : http://www.hostingpics.net/viewer.php?id=252460Terminals20160428165435.png I have installed the agent with this command line zabbix_agentd.exe --config C:\Program Files\Zabbix Agent\zabbix_agentd.conf --install with success. I followed the documentation here https://www.zabbix.com/documentation/3.0/manual/appendix/install/windows_agent I start manually the service with service panel on windows and not manually by CLI. Yes, the agent can write the log file without problem because the security right are not modified. I will try to start the agent without system.run but I need it to retrieve FQDN because I use discovery process on Zabbix Server. Do you have another solution to fill the hostname parameter with fqdn without to edit each config file ? I looked the antivirus logs but nothing find about Zabbix agent. Thank you for your support ! |
Comment by Jérôme Queneuder [ 2016 May 01 ] |
Hello Aleksandrs Saveljevs, After 3 reboots without option HostnameItem=system.run it works good at startup Thank you again for your support ! |
Comment by Aleksandrs Saveljevs [ 2016 May 02 ] |
Jérôme, thanks for info! Personally, I am not aware of a different method to fill Hostname with FQDN, but you may wish to ask on https://www.zabbix.org/wiki/Getting_help - there are many experienced community members. As pointed out by zalex_ua, if HostnameItem fails for some reason, we may wish to log this somehow (e.g., if we do not have the log file open yet and we are not running on the console, can we log errors to Event Log by default on Windows?). Therefore, setting this issue to "Open". |
Comment by Jérôme Queneuder [ 2016 May 02 ] |
OK Aleksandrs. I have activated logging in the windows event log with debug=5 option. I keep you informed. Thank you ! Jérôme |
Comment by Oleksii Zagorskyi [ 2016 May 02 ] |
It reminds me about ZBX-8893, although I'm not sure it's relevant in current context. |
Comment by Jérôme Queneuder [ 2016 May 04 ] |
Hello Aleksandrs and Oleksiy, I have tested with my previous parameter HostnameItem=system.run[powershell -NonInteractive "[System.Net.Dns]::GetHostByName(($env:computerName)).Hostname.ToLower()"] And the system event log activated and debug level 5 in zabbix config file but the zabbix agent don't log an error after host reboot. The error message is the same EventID 7009 : Could think if it possible to add some options at installation of the zabbix service like "-netbiosname 1" (activate netbios name for hostname parameter) and "-fqdn 1" (activate fqdn for hostname parameter) ? Thank you, Jérôme |
Comment by Jérôme Queneuder [ 2016 May 18 ] |
Dear all, Do you have an update for me ? Thank you, Jérôme Queneuder |
Comment by Aleksandrs Saveljevs [ 2016 May 23 ] |
Jérôme, regarding what can be done on Zabbix side, we are leaving this issue open to provide some logging in case service startup fails or maybe provide a more permissive timeout for HostnameItem. Regarding your question about obtaining FQDN in the meanwhile and assuming "system.hostname[host]" item does not work for you, it might be better to direct this question to experienced Zabbix users on IRC or forum, as mentioned at https://www.zabbix.org/wiki/Getting_help . |
Comment by Reece Webb [ 2016 May 23 ] |
I have this identical problem, same timeout error, though it's not across the board and is inconsistent... one reboot will have the error, the next won't. Having it with Windows 2008R2 and 2012R2. Agent 3.0.0 (revision 58455). Installed with "C:\Zabbix\bin\zabbix_agentd.exe --config C:\Zabbix\conf\zabbix_agentd.win.conf --install", the service is started automatically at boot. We tried putting in a delayed start, but that hasn't helped. Permissions are correct, nothing is written to the Zabbix log when this happens. I have poured through the Eventlog, with debugging on, and nothing is jumping out at me. I'm going to try adding a dependency to a late-starting service, just in case some other service is taking too long to start and is clogging things up. |
Comment by Jérôme Queneuder [ 2016 May 26 ] |
Hello Reece, Could you try to add a dependency with Remote desktop service ? Because it's a last service who start on a Windows server Thank you for your test, we will find a solution, I hope Jerome |
Comment by Daniel V [ 2016 Sep 21 ] |
I had this issue too on Windows Server 2008 R2. Just someday zabbix agent stopped working with the service control telling me this:
By removing parts of my zabbix_agentd.conf I was able to find that the following caused the error:
By the way: Starting the zabbix_agentd.exe by using the cmd console worked fine! Why can't there be a config key to tell the zabbix agent to use his hostname and domain as agent.hostname? |
Comment by Jérôme Queneuder [ 2016 Sep 21 ] |
Hello Daniel, At our side, we have solved this situation by using the zabbix agent from suiviperf.com because we have requested a modification to include the possibility use FQDN or NETBIOS name HOSTNAMEFQDN=1 : Hostname in FQDN in low case Thank you ! |
Comment by Oleksii Zagorskyi [ 2016 Sep 21 ] |
Windows installation packages on suiviperf.com cannot do anything better regarding discussed issue. |
Comment by Daniel V [ 2016 Sep 21 ] |
I don't see why I should use a modified zabbix agent. Rather than forking the original code should be fixed. I assume the zabbix agent crashes in my case, since nothing is written into the log file. Is it possible to find out why the zabbix agent crashes and prevent it from crashing, to be able to write into the log file what is wrong? Even with debug level of 4 or 5 nothing is written into the log file. |
Comment by Oleksii Zagorskyi [ 2016 Sep 21 ] |
Agents form suiviperf.com - should be original, not modified. But they are packaged to msi/exe installation files. I don't think zabbix agent crashes, it just refuses to start or say, exits on start. |
Comment by Daniel V [ 2016 Sep 21 ] |
Ok but how do I get that error message? When running zabbix in a console (cmd.exe) then it just warns about running in a console. The error seems only to occur, when the zabbix agent is started by the windows service management. I tried to use this to log any zabbix output when started by the service management. Sadly the file wasn't created.
Can I force zabbix to spawn a console window? |
Comment by Gian [ 2016 Oct 20 ] |
Hi everyone Same issue over here... But we have the same problem as described on all physical servers. I just cannot imagine why it ain't working on the physical servers too...... Excuse my english, I know it's not perfect at all but I guess it will do it for now Gian |
Comment by Reece Webb [ 2016 Oct 25 ] |
We're still having this issue, but have found a workaround by adding a dependency on Netlogon to the "Zabbix Agent" service. Haven't had a timeout since. Registry key is attached. |
Comment by Oleksii Zagorskyi [ 2016 Oct 26 ] |
I personally consider Reece's workaround as a correct approach for those who experience such issues. |
Comment by Janne Korkkula [ 2016 Oct 27 ] |
We're experiencing this issue too, after configuring a system.run powershell hack to get the server FQDN, which is of course required for active monitoring. Thank you Reece, we'll adopt that fix if it turns out to work. The real fix would be for the Windows Zabbix agent to provide an agent-internal item with the host FQDN (in lower case). Adopting all these powershell hacks is just silly, but we're practically forced to do it... PS. here's our hack:
HostnameItem=system.run[powershell -NonInteractive "[System.Net.Dns]::GetHostByName(($env:computerName)).Hostname.ToLower()"]
|
Comment by Janne Korkkula [ 2016 Nov 11 ] |
Our Windows guys found an even more horrific hack, which just may work, from an older issue: ZBXNEXT-2106 I think that issue should be reconsidered and reopened. |
Comment by Christophe Chapuis [ 2017 Nov 06 ] |
Hello, we're also experiencing the same behavior by using :
HostnameItem=system.run[powershell -NonInteractive "[System.Net.Dns]::GetHostByName(($env:computerName)).Hostname.ToLower()"]
Environment :
+1 to get a solution that will provide FQDN in lowercase as Hostname Cheers Christophe |
Comment by Jérôme Queneuder [ 2017 Nov 06 ] |
Hi Christophe, I have asked the developer of SuiviPerf.com to add an option to set fqdn as hostname in the installer The option is : Many thanks to suiviperf.com dev Jerome |
Comment by Max N [ 2018 Oct 22 ] |
I came across this very same problem... its a real PITA to use zabbix with autoregistration/discovery with windows agents. There isliterally no way to get an all lowercase hostnameitem on windows, because every time one calls a powershell, the agent will fail with
HostnameItem=system.run[powershell -NonInteractive [System.Net.Dns]::GetHostByName(($env:computerName)).Hostname.ToLower()]
zabbix_agentd.exe [1960]: Warning: failed to get system hostname from [system.run[powershell -NonInteractive [System.Net.Dns]::GetHostByName(($env:computerName)).Hostname.ToLower()
What indeed works is using bat commands only: # HostnameItem=system.run[echo %COMPUTERNAME%.%USERDNSDOMAIN%] but this returns an FQDN without the guarantee for an all lowercase string, which is basically impossible to do in bat scripting.
IMHO it is currently not possible to do a large scale autoregister process where active checks on Windows hosts are a requirement and you wont have some hacked workaround to hardwire hostnames into the deployed zabbix_agentd.conf on windows hosts.
Possible way to fix this in zabbix might be: Provide an option in zabbix_agentd.conf to use systems FQDN for the hostname (or even use it per default) Make autodiscovery use the hostname instead of the FQDN to create hosts A new, working powershell key to generate an FQDN in PS for hostnameitem - ZBXNEXT-1101 Make Hostnames case-insesitive in Zabbix-Server. I know, technically those names are not DNS, but RFC 4303 declares DNS names explicitly as case insensitive.
|
Comment by Reece Webb [ 2018 Oct 22 ] |
To get lowercase hostnames I do a two-step process. Not perfect, but with our automated agent installs it has been working well for us. zabbix_agentd.win.conf: HostnameItem=system.run[powershell -NonInteractive "C:\Zabbix\scripts\hostname.ps1"] hostname.ps1: "$env:computerName".toLower() |
Comment by HoaiDuyen [ 2018 Oct 31 ] |
thank you so much !!! |