[ZBX-18180] Incorrect swap utilization on certain Win2016 servers reported by Agent1 5.0.2 Created: 2020 Aug 01 Updated: 2020 Aug 18 |
|
| Status: | Need info |
| Project: | ZABBIX BUGS AND ISSUES |
| Component/s: | Agent (G) |
| Affects Version/s: | 5.0.2 |
| Fix Version/s: | None |
| Type: | Problem report | Priority: | Trivial |
| Reporter: | Erik Carlseen | Assignee: | Zabbix Support Team |
| Resolution: | Unresolved | Votes: | 1 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Environment: |
Windows Server 2016, 32GiB RAM. |
||
| Attachments: |
|
||||||||
| Issue Links: |
|
||||||||
| Description |
|
I just added two Windows 2016 servers to Zabbix monitoring (5.0.2 server and 5.02 Agent1 / not using Agent2), and both show 0% swap space free: This is unsurprising, as the server in question is provisioned with 32GiB of RAM and is only using about 11GiB, as reported by TaskMangler: If I kick up the debug level on the agent to '5', this is what I get in the logs there: 12144:20200801:092603.661 End of send_buffer():SUCCEED *12144:20200801:092603.661 for key [system.swap.size[,free]] received value [0]* *12144:20200801:092603.662 In process_value() key:'<<<MY MONITORED HOST>>>:system.swap.size[,free]' lastlogsize:null value:'0'* 12144:20200801:092603.663 In send_buffer() host:'<<<MY ZABBIX SERVER>>>' port:10051 entries:3/100 12144:20200801:092603.664 send_buffer() now:1596299163 lastsent:1596299163 now-lastsent:0 BufferSend:5; will not send now 12144:20200801:092603.664 End of send_buffer():SUCCEED 12144:20200801:092603.665 buffer: new element 3 12144:20200801:092603.666 End of process_value():SUCCEED 12144:20200801:092603.667 In need_meta_update() key:system.swap.size[,free] 12144:20200801:092603.667 End of need_meta_update():FAIL 12144:20200801:092603.668 In send_buffer() host:'<<<MY ZABBIX SERVER>>>' port:10051 entries:4/100 12144:20200801:092603.669 send_buffer() now:1596299163 lastsent:1596299163 now-lastsent:0 BufferSend:5; will not send now 12144:20200801:092603.669 End of send_buffer():SUCCEED *12144:20200801:092603.670 for key [system.swap.size[,pfree]] received value [0.000000]* *12144:20200801:092603.671 In process_value() key:'<<<MY MONITORED HOST>>>:system.swap.size[,pfree]' lastlogsize:null value:'0.000000'* 12144:20200801:092603.672 In send_buffer() host:'<<<MY ZABBIX SERVER>>>' port:10051 entries:4/100 12144:20200801:092603.672 send_buffer() now:1596299163 lastsent:1596299163 now-lastsent:0 BufferSend:5; will not send now 12144:20200801:092603.673 End of send_buffer():SUCCEED 12144:20200801:092603.674 buffer: new element 4 12144:20200801:092603.675 End of process_value():SUCCEED 12144:20200801:092603.675 In need_meta_update() key:system.swap.size[,pfree] 12144:20200801:092603.676 End of need_meta_update():FAIL 12144:20200801:092603.677 In send_buffer() host:'<<<MY ZABBIX SERVER>>>' port:10051 entries:5/100 12144:20200801:092603.677 send_buffer() now:1596299163 lastsent:1596299163 now-lastsent:0 BufferSend:5; will not send now 12144:20200801:092603.678 End of send_buffer():SUCCEED *12144:20200801:092603.679 for key [system.swap.size[,total]] received value [5100273664]* *12144:20200801:092603.680 In process_value() key:'<<<MY MONITORED HOST>>>:system.swap.size[,total]' lastlogsize:null value:'5100273664'* 12144:20200801:092603.680 In send_buffer() host:'<<<MY ZABBIX SERVER>>>' port:10051 entries:5/100 12144:20200801:092603.681 send_buffer() now:1596299163 lastsent:1596299163 now-lastsent:0 BufferSend:5; will not send now 12144:20200801:092603.682 End of send_buffer():SUCCEED 12144:20200801:092603.683 buffer: new element 5 12144:20200801:092603.683 End of process_value():SUCCEED 12144:20200801:092603.684 In need_meta_update() key:system.swap.size[,total] 12144:20200801:092603.685 End of need_meta_update():FAIL 12144:20200801:092603.685 In send_buffer() host:'<<<MY ZABBIX SERVER>>>' port:10051 entries:6/100 12144:20200801:092603.686 send_buffer() now:1596299163 lastsent:1596299163 now-lastsent:0 BufferSend:5; will not send now 12144:20200801:092603.687 End of send_buffer():SUCCEED 12144:20200801:092603.740 Empty WMI search result. Looks like there is some problem with the agent getting the correct swap utilization info via WMI. This is only occurring on two of my Windows 2016 servers. The differences between these servers and others in my environment are the RAM provisioning (exactly 32GiB) and they're running Microsoft Exchange 2016 (probably unrelated). I have servers with more RAM and less RAM being monitored that do not have this issue.
|
| Comments |
| Comment by Edgar Akhmetshin [ 2020 Aug 03 ] |
|
Hello Erik, Could you please show output from: systeminfo | find "Virtual Memory" Also could you please confirm permissions granted to the agent on this servers? Templates are from 5.0 or this is an upgraded installation? Regards, |
| Comment by Erik Carlseen [ 2020 Aug 03 ] |
|
Edgar, Thank you for your very prompt response. The requested output is: Server 1: Virtual Memory: Max Size: 37,632 MB Server 2: Virtual Memory: Max Size: 37,631 MB In both cases, the agent is running as LocalSystem. Regards, |
| Comment by Erik Carlseen [ 2020 Aug 04 ] |
|
Oh, sorry, we did a fresh 5.0.1 installation, since upgraded to 5.0.2.
-Erik
|
| Comment by Edgar Akhmetshin [ 2020 Aug 05 ] |
|
Please export used templates and show output from zabbix_get for the system.swap.size[,pfree] and system.swap.size[,bytes] key on this systems. |
| Comment by Erik Carlseen [ 2020 Aug 08 ] |
|
We had one more system exhibit this behavior when added - not running Exchange, but Microsoft SystemCenter. Different amount of RAM as well.
Sample output from one system: system.swap.size[,pfree] = 0.000000 system.swap.size[,free] = 0 system.swap.size[,bytes] = ZBX_NOTSUPPORTED: Invalid second parameter. I think you meant: system.swap.size[,total] = 5100273664
Template is stock / unmodified (Template Module Windows memory by Zabbix agent active), but will upload anyway. |
| Comment by Edgar Akhmetshin [ 2020 Aug 13 ] |
|
Could you please confirm, this is a virtual machine? According to the screenshots and memory slots information, looks so. Please try to get value from perf_counter[\700(_Total)\702], will it show correct result? |
| Comment by Erik Carlseen [ 2020 Aug 13 ] |
|
Yes, these are VMs. We barely do bare metal anymore. It shows this result: 22.593287. I'm not certain what result it is supposed to show.
|
| Comment by Edgar Akhmetshin [ 2020 Aug 14 ] |
|
Hello Erik, Looks like duplicate of an issue: The correct way for now - use performance counter as source of the information. Regards, |
| Comment by Erik Carlseen [ 2020 Aug 18 ] |
|
Unfortunately, I do not have any relevant MS support agreements covering this right now. I can tell you this: We are experiencing the issue under both HyperV (Server 2016) and VMware ESXi 6.7, and on both Intel Xeon and AMD EPYC architectures. It works fine on some servers and not on others, even in different AD forests. I really wish I had some way to better nail this down for you, but I don't. I may try updating one of the servers to 2019 and see if it makes a difference. |