-
Type:
Problem report
-
Resolution: Unresolved
-
Priority:
Major
-
None
-
Affects Version/s: 7.0.25
-
Component/s: Agent (G), Agent2 (G)
-
None
-
Support backlog
All details of the real scenario are unknown, but imagine a Windows host which being monitored with a dedicated template with custom perf counters.
When an app (BizTalk), which registers the custom counters, being restarted (supposedly), those counters disappear for a while and then return back. In such case zabbix agent catches the state that counters are missing and is not able to recover after the counters are back.
I.e. zabbix agent permanently stuck in a state that the perf counters are missing. Only agent restart allows to restore operations.
Scenario to reproduce:
We will be adding and removing Windows features - FTP server. This can be done using GUI (search "turn on" in Start menu) "Turn Windows features on or off" or using these commands (cmd elevated to Administrator):
dism /online /enable-feature /featurename:IIS-FTPServer dism /online /disable-feature /featurename:IIS-FTPServer
note - removal then requires a reboot to complete it.
We will be testing a counter path:
"\Microsoft FTP Service(_Total)\Microsoft FTP Service Uptime"
Before installing, making sure those counters are missing, output is empty:
> typeperf -qx | findstr -i ftp
Starting both agents - Agent2 and Classic one, on different ports - 10050 and 10060 correspondingly.
Both version 7.0.25, x64.
Testing the key, each twice:
$ zabbix_get -s 10.33.0.42 -k 'perf_counter["\Microsoft FTP Service(_Total)\Microsoft FTP Service Uptime"]' zabbix_get [269370]: Check access restrictions in Zabbix agent configuration $ zabbix_get -s 10.33.0.42 -k 'perf_counter["\Microsoft FTP Service(_Total)\Microsoft FTP Service Uptime"]' ZBX_NOTSUPPORTED: Failed to get counter for path "\\Microsoft FTP Service(_Total)\\Microsoft FTP Service Uptime" and lang 0: Cannot add counter: The specified object was not found onthe computer. .
$ zabbix_get -s 10.33.0.42 -k 'perf_counter["\Microsoft FTP Service(_Total)\Microsoft FTP Service Uptime"]' -p 10060 ZBX_NOTSUPPORTED: Invalid performance counter format. $ zabbix_get -s 10.33.0.42 -k 'perf_counter["\Microsoft FTP Service(_Total)\Microsoft FTP Service Uptime"]' -p 10060 ZBX_NOTSUPPORTED: Cannot obtain performance information from collector.
just a note - 1st error always different from following ones.
Installing the FTP server and making sure the counters have appeared:
>typeperf -qx | findstr -i ftp \Microsoft FTP Service(_Total)\Microsoft FTP Service Uptime \Microsoft FTP Service(_Total)\Total Logon Attempts .....
Testing agents again:
$ zabbix_get -s 10.33.0.42 -k 'perf_counter["\Microsoft FTP Service(_Total)\Microsoft FTP Service Uptime"]' ZBX_NOTSUPPORTED: Failed to get counter for path "\\Microsoft FTP Service(_Total)\\Microsoft FTP Service Uptime" and lang 0: Cannot add counter: The specified object was not found onthe computer. . $ zabbix_get -s 10.33.0.42 -k 'perf_counter["\Microsoft FTP Service(_Total)\Microsoft FTP Service Uptime"]' ZBX_NOTSUPPORTED: Failed to get counter for path "\\Microsoft FTP Service(_Total)\\Microsoft FTP Service Uptime" and lang 0: Cannot add counter: The specified object was not found onthe computer. .
$ zabbix_get -s 10.33.0.42 -k 'perf_counter["\Microsoft FTP Service(_Total)\Microsoft FTP Service Uptime"]' -p 10060 ZBX_NOTSUPPORTED: Cannot obtain performance information from collector. $ zabbix_get -s 10.33.0.42 -k 'perf_counter["\Microsoft FTP Service(_Total)\Microsoft FTP Service Uptime"]' -p 10060 ZBX_NOTSUPPORTED: Cannot obtain performance information from collector.
Both are failing, both attempts.
Both agents have been restarted here ...
$ zabbix_get -s 10.33.0.42 -k 'perf_counter["\Microsoft FTP Service(_Total)\Microsoft FTP Service Uptime"]' zabbix_get [269705]: Check access restrictions in Zabbix agent configuration $ zabbix_get -s 10.33.0.42 -k 'perf_counter["\Microsoft FTP Service(_Total)\Microsoft FTP Service Uptime"]' 65.000000
note - success from 2nd attempt.
$ zabbix_get -s 10.33.0.42 -k 'perf_counter["\Microsoft FTP Service(_Total)\Microsoft FTP Service Uptime"]' -p 10060
69.000000
note - success from 1st attempt
I could not test FTP server removal here, as it forces Windows reboot to complete it and remove the counters. That's why the reproduce scenarios is for Installing the FTP.
Note - this is not possible to reproduce if use for example perf path "\Process(......)\...." and just starting/stopping some process. In such case error is different and is not permanent.
I.e. it's important to add/remove a new path, which starts from 1st word after 1st back slash.