Windows 10

Hello Victor,

Could you please specify:

• exact Windows 10/11 build used at least for 1 system;
• intel/amd (cpu name/model), which sleep states are configured?
• sleep or hibernation or hybrid mode is used?

Also if it is possible - enable debug log for Zabbix Agent and go through sleep/wake up cycle.

Regards,
Edgar

1. Win10 1709
2. Intel i3-6100, 3,7g Sleep is not configured
Sequence of events

Диспетчер питания ядра инициировал завершение работы.
Диспетчер питания ядра инициировал завершение работы.
Уведомление о выходе пользователя для программы улучшения качества ПО
Система переходит в спящий режим. -21.06.2022 8:36:11
Система возобновила работу после режима сна. - 21.06.2022 8:36:15
Системное время изменено с ‎2022‎-‎06‎-‎21T05:36:15.696737400Z на ‎2022‎-‎06‎-
Встроенное ПО сообщило метрики загрузки.
Число вариантов загрузки данной системы: 0x1.
Диспетчер загрузки затратил 0 мс на ожидание ввода данных пользователем.
Использовалась следующая политика меню загрузки: 0x1.
Использовался следующий тип загрузки: 0x1.
Журнал доступа в кусте \Device\HarddiskVolume2\EFI\Microsoft\Boot\BCD очищен; обновлено разделов: 127; создано измененных страниц: 11. --21.06.2022 8:39:36
-+*-At this point, the agent falls 15420:20220621:083936.769 Zabbix Agent stopped. Zabbix 6.0.4*+
Система вышла из состояния пониженного энергопотребления. -21.06.2022 8:39:38

The Kernel Power Manager Triggered Shutdown event occurs infrequently, once every few days, and without any apparent period. Getting extended debug is difficult

Victor,

We can not reproduce using Windows for 10 and 11 on several laptops used everyday.

Please provide debug log.

Regards,
Edgar

OK. What level of debug is needed?

Please try to use 4 debug level.

I am attaching the log. In the Windows log, the same sequence of entries. After recording the time change, there is a message about the network adapter being disconnected.
Intel(R) Ethernet Connection I217-LM Network link is disconnected.

• This message is always in the log sequence at the time the agent crashed.

[^zabbix_agentd_50433.log]

WinVer 10.0.14393
The fall of agents occurs with a frequency of 50 out of 300 in 2 weeks. Almost all HP computers. Windows versions are different. Version updates do not occur because the nodes do not have access to the Internet. The statistics are taken only for one of the groups, and the total number of nodes is about 10,000. Such a frequent drop in agents makes monitoring impossible.

Hello Victor,

Re-uploaded file with password for our DEV team, also removed clear text log with sensitive information.

Confirmed.

Regards,
Edgar

It should be controlled by function ServiceCtrlHandler for example ZBX-21227.diff could help, it looks like shutdown event is received by Zabbix agent.

Zabbix Agent should write info about received event into the log file, need to improve logs, otherwise looks like non Zabbix issue.

Perhaps this does not look like a zabbix problem to developers. But the fact is that the agent constantly crashes. Solving the problem - every day I run the program for detecting fallen nodes and the program for starting agents. Especially often agents fall on WINDOWS XP. For 10,000 knots, dropping 100 per day turns monitorin into a circus. What do you need to get started with this? Am I the only one with such problems?

You could try if patch ZBX-21227.diff helps, currently it looks like Zabbix agent receive command to stop. It's also possible for us to build agent for you to test with additional debug to determine root cause.

In Windows 10, when the computer is idle for a long time, a sleep signal occurs. The procedure for switching to this mode begins and after 5-10 seconds the restoration of normal operation. In this period of time, the agent falls down. Why the sleep mode signal appears is not clear to me. Power management never turns off either the processor or the drive.
I'm ready to try a specially compiled agent to find a solution to this problem.

Please help to assembly the agent with your changes. Errors occur when attempting to build the agent.
Command - nmake /K -f Makefile_agent PCREINCDIR=D:\pcre-8.45-install\include PCRELIBDIR=D:\pcre-8.45-install\lib
......
resource.rc(75) : error RC2127 : version WORDs separated by commas expected
resource.rc(75) : error RC2104 : undefined keyword or key name: ZABBIX_RC_NUM
NMAKE : warning U4010: zabbix_agent.res: сбой сборки; задан параметр /K, продолжение...
NMAKE : warning U4011: ..\..\..\bin\win64\zabbix_agentd.exe : не все зависимые компоненты; целевой объект не построен
NMAKE : warning U4011: all : не все зависимые компоненты; целевой объект не построен

If with a key nmake /A ...
resource.rc(75) : error RC2127 : version WORDs separated by commas expected
resource.rc(75) : error RC2104 : undefined keyword or key name: ZABBIX_RC_NUM
NMAKE : fatal error U1077: "C:\Program Files (x86)\Windows Kits\10\bin\10.0.19041.0\\x64\rc.exe" : возвращенный код "0x2"

Is there an updated assembly instruction for the agent?

Something similar to this below should work:

nmake -f Makefile CFLAGS="/D ZABBIX_VERSION_REVISION=ababab" RFLAGS="/d ZABBIX_VERSION_REVISION=ababab /d ZABBIX_VERSION_RC_NUM=12345" PCREINCDIR="C:\pcre-8.41-Win64\output\include" PCRELIBDIR="C:\pcre-8.41-Win64\output\lib"

But here is full description:
https://www.zabbix.com/documentation/current/en/manual/installation/install/win_agent

D:\zabbix-release-6.0\build\win32\project>nmake /K -f Makefile_agent CFLAGS="/D ZABBIX_VERSION_REVISION=6.0.8" RFLAGS="/d ZABBIX_VERSION_REVISION=6.0.8 /d ZABBIX_VERSION_RC_NUM=1" PCREINCDIR=D:\pcre-8.45-install\include PCRELIBDIR=D:\pcre-8.45-install\lib

Служебная программа обслуживания программ Microsoft (R), версия 14.33.31629.0
(C) Корпорация Майкрософт (Microsoft Corporation). Все права защищены.

rc.exe /d ZABBIX_VERSION_REVISION=6.0.8 /d ZABBIX_VERSION_RC_NUM=1 /d "NDEBUG" /d "_VC80_UPGRADE=0x0600" /l 0x419 /fo"zabbix_agent.res" resource.rc
Microsoft (R) Windows (R) Resource Compiler Version 10.0.10011.16384
Copyright (C) Microsoft Corporation. All rights reserved.

resource.rc(86) : error RC2104 : undefined keyword or key name: VER_FILEDESCRIPTION_STR
NMAKE : warning U4010: zabbix_agent.res: сбой сборки; задан параметр /K, продолжение...
NMAKE : warning U4011: ..\..\..\bin\win64\zabbix_agentd.exe : не все зависимые компоненты; целевой объект не построен
NMAKE : warning U4011: all : не все зависимые компоненты; целевой объект не построен

I use VS 2022. What compiler should be used? FROM visual studio or from MingW?

It's much better if you follow documentation and configure on linux, attached configured source

It's visual studio, configured sources can also be downloaded here:
https://www.zabbix.com/download_sources#60LTS

Is it possible to compile the agent for windows on linux?

I am afraid that no, have you managed to compile on windows, it seemed that you were close.

Failed to compile agent
D:\zabbix-6.0.8rc2\build\win32\project>nmake -f Makefile_agent CFLAGS="/D ZABBIX_VERSION_REVISION=6.0.8" RFLAGS="/d ZABBIX_VERSION_REVISION=6.0.8 /d ZABBIX_VERSION_RC_NUM=2" PCREINCDIR="D:\pcre-8.45-install\include" PCRELIBDIR="D:\pcre-8.45-install\lib"

Служебная программа обслуживания программ Microsoft (R), версия 14.28.29913.0
(C) Корпорация Майкрософт (Microsoft Corporation). Все права защищены.

mc.exe -U -h ".
" -r ".
" messages.mc
MC: Compiling messages.mc
cl.exe ..\..\..\src\libs\zbxalgo\algodefs.c /Fo"..\..\..\src\libs\zbxalgo\algodefs.o" /D ZABBIX_VERSION_REVISION=6.0.8
Оптимизирующий компилятор Microsoft (R) C/C++ версии 19.28.29913 для x64
(C) Корпорация Майкрософт (Microsoft Corporation). Все права защищены.

algodefs.c
..\..\..\src\libs\zbxalgo\algodefs.c(20): fatal error C1083: Не удается открыть файл включение: common.h: No such file or directory,
NMAKE : fatal error U1077: "C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.28.29910\bin\HostX64\x64\cl.exe" : возвращенный код "0x2"
Stop.

Please try zabbix_agent-6.0.10rc1-windows-amd64-openssl.zip

Fixed in:

• pre-5.0.29rc1 fcfab45db09
• pre-6.0.10rc1 ca1ef01b85a
• pre-6.2.4rc1 d115377a5e4
• pre-6.4.0beta2 (master) 301782e7d94

Agent 6.0.10rc1 crashes continue. The agent writes two identical messages to the log every second "Zabbix Agent recived request:4. Agent crash is no mark in the log. The agent stopped responding and sending a request: 4". Agent 6.0.8 and 6.0.10rc1 crashes stably on Windows 10.0.18363 and 6.3

The event is:
SERVICE_CONTROL_INTERROGATE 0x00000004

Can you please be so kind and provide last entries in the log

In what log, windows or zabbix? In zabbix log only every second "Zabbix Agent recived request:4" There are no reports of an agent crash. After the fall disappears "...request:4"

Your agent is ABSOLUTELY inoperable. It crashes on all versions of windows. Here is a list of versions where the fall was noticed
05.01.2600
05.02.3790
06.01.7600
06.01.7601
06.02.9200
06.03.9600
10.0.10240
10.0.10586
10.0.14393
10.0.16299
10.0.17134
10.0.17763
10.0.18362
10.0.18363
10.0.19041
10.0.19042
10.0.19043
10.0.19044
Agent 6.0.10rc1 also crashes steadily. Almost everything is disabled on workstations, only inventory data is collected by simple queries wmiget. A total of 10,000 nodes are monitored. Our correspondence has been going on for 4 months. So far to no avail.

Agent 6.0.X is ABSOLUTELY inoperable!

Please provide Zabbix agent log, having event log from windows would also be nice to have to see why Zabbix agent was stopped.

All Windows logs contain an event when an agent crashes:
Ошибка,13.10.2022 9:22:55,Service Control Manager,7000,Отсутствует,"Сбой при запуске службы ""Zabbix Agent"" из-за ошибки Служба не ответила на запрос своевременно."
Ошибка,13.10.2022 9:22:55,Service Control Manager,7009,Отсутствует,"Превышение времени ожидания (30000 мс) при ожидании подключения службы ""Zabbix Agent""."
Сведения,13.10.2022 9:22:33,Microsoft-Windows-Kernel-Boot,20,Отсутствует,Состояние успешного выполнения последнего завершения работы: true. Состояние успешного выполнения последней загрузки: true.
Сведения,13.10.2022 9:22:33,Microsoft-Windows-Kernel-General,12,Отсутствует,Системное время запуска операционной системы: ‎2022‎‎10‎‎13T06:22:33.490589300Z.
Сведения,13.10.2022 9:22:16,Microsoft-Windows-Kernel-General,13,Отсутствует,Системное время завершения работы операционной системы: ‎2022‎‎10‎‎13T06:22:16.705011900Z.
Сведения,13.10.2022 9:22:11,Microsoft-Windows-Kernel-Power,109,(103),Диспетчер питания ядра инициировал завершение работы.
Сведения,13.10.2022 9:22:08,Microsoft-Windows-Dhcp-Client,50037,Событие состояния службы, Остановка клиентской службы DHCPv4. Значение флага завершения работы: 1
Сведения,13.10.2022 9:21:55,Microsoft-Windows-Winlogon,7002,(1102),Уведомление о выходе пользователя для программы улучшения качества ПО
The agent stops at the same time as the service DHCP stops
5492:20221012:142211.050 agent #5 started active checks #1
6120:20221013:092208.102 Zabbix Agent stopped. Zabbix 6.0.8 (revision c7c3044a4a2).
The agent configuration file contains the parameter:
HostnameItem=system.run["c:\zabbix\script\ip.bat"]
Content of ip.bat
@echo off
set path=%SystemRoot%\system32;%SystemRoot%;%SystemRoot%\System32\Wbem;%SystemRoot%\System32\WindowsPowerShell\v1.0\
SetLocal EnableDelayedExpansion
set findip=0
FOR /F "usebackq tokens=2* delims=(:" %%a IN (`ipconfig.exe ^| FINDSTR.exe /I "IP"`) DO (
set ip=%%a
set ip=Unable to render embedded object: File (~1,3) not found.
IF !ip! equ 10. (
if Unable to render embedded object: File (findip) not found. equ 1 goto :endip
set ip=%%a
set ip=Unable to render embedded object: File ( =) not found.
set findip=1
echo !ip!
)
IF !ip! equ 172 (
if Unable to render embedded object: File (findip) not found. equ 1 goto :endip
set ip=%%a
set ip=Unable to render embedded object: File ( =) not found.
set findip=1
echo !ip!
)
)
:endip
exit

The script ip.bat defines the host address, finds addresses only from networks 10.Х.Х.Х and 172.Х.Х.Х, selects only one address. If an interface contains two or more addresses, or if there are multiple working interfaces, all but the first one in the command's ipconfig output are discarded. All host addresses are set manually. However, the downtime of the agent equals with the stop of the service DHCP

Unfortunately, the text ip.bat is displayed incorrectly.ip.rar

Thanks, unfortunately missing log with debug log level, could you please be so kind and provide, it looks like something is stopping Zabbix agent, it is not stopping by itself 

Nothing but an infinite number of performance counter reporting messages. I've already done that. There is only one message: zabbix agent stopped

It seems to me that the agent crash is related to the HostNameItem parameter. The fall of the agent occurs simultaneously with the fall of the dhсp service. After 30 seconds, an entry appears in the Windows log - the service did not respond to the request timely. When the dhcp service is disabled, the ipconfig request is not performed. I decided to test this hypothesis. Deleted HostNameItem and registered hostname. I'll report the results tomorrow.

HostNameItem has been removed by 1000 nodes. hostname=ip has been added. For 1.5 days, I do not observe falls. I change the configuration on another 9000 nodes. The result will be 31/10. What to do with ZBX-21227? Open a new issue? The HostNameItem configuration parameter not working correctly.

What was the value of HostNameItem ? Does it crash if it is changed from script to something else ? We can try script locally and get back to you

HostnameItem=system.run["c:\zabbix\script\ip.bat"]

Ther'is ip.bat in attachments

The script ip.bat defines the host address, finds addresses only from networks 10.Х.Х.Х and 172.Х.Х.Х, selects only one address. 

Hi SVDouble ,

It seems that in your case Zabbix agent is stopped by some external request, probably caused by the sleep mode. This is out of our control, so the agent just stops as it should. However some time later Zabbix agent service is attempted to be started again, but fails. This is proven by your logs:

Ошибка,13.10.2022 9:22:55,Service Control Manager,7000,Отсутствует,"Сбой при запуске службы ""Zabbix Agent"" из-за ошибки Служба не ответила на запрос своевременно."
Ошибка,13.10.2022 9:22:55,Service Control Manager,7009,Отсутствует,"Превышение времени ожидания (30000 мс) при ожидании подключения службы ""Zabbix Agent""."

Zabbix agent fails to start because your batch file cannot retrieve hostname (some network services are down at this moment). Now, when you configured 'Hostname' instead of 'HostnameItem' the agent starts without errors.

Unfortunately there are no logs for the configuration errors during initialization if agent is started as service. We will try to improve logging in this aspect.

It looks like the truth. You need to add in the documentation to the description of the agent configuration parameter HosnameItem - you can not use any requests to the operating system in the parameter.

Available in:

• 5.0.31rc1 c278ca75c93
• 6.0.13rc1 6181c559926
• 6.2.7rc1 e46ac3d280e
• 6.4.0beta6(master) d7c1edd04bb, fd716082cd1