[ZBX-18800] Zabbix agent WMI returns incorrect values Created: 2020 Dec 22  Updated: 2021 Jun 28

Status: Confirmed
Project: ZABBIX BUGS AND ISSUES
Component/s: Agent (G), Agent2 plugin (G)
Affects Version/s: 5.0.7, 5.2.2
Fix Version/s: None

Type: Incident report Priority: Minor
Reporter: Daniel Assignee: Zabbix Development Team
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Windows Server 2016
Windows Server 2019



 Description   

Steps to reproduce:

  1. No special configuration required
  2. Query zabbix agent with a WMI value like 
wmi.get[root\cimv2,select AvgDisksecPerRead from Win32_PerfRawData_PerfDisk_Logicaldisk WHERE Name='_Total']

Result:

Values are unreliable:

2020-12-22 01:09:00 15584045
2020-12-22 01:08:00 14744837
2020-12-22 01:07:00 13824107
2020-12-22 01:06:00 12601199
2020-12-22 01:05:00 18446744073694678941
2020-12-22 01:04:00 18446744073693237047
2020-12-22 01:03:00 18446744073671703488
2020-12-22 01:02:10 18446744073671267304
2020-12-22 01:01:00 18446744073667840523
2020-12-22 01:00:00 18446744073648979239

...

2020-12-21 18:14:00 18446744071565508350
2020-12-21 18:13:00 18446744071563709666
2020-12-21 18:12:00 18446744071563225384
2020-12-21 18:11:00 2147280336
2020-12-21 18:10:00 2147275532
2020-12-21 18:09:00 2146821900
2020-12-21 18:08:00 2146369424
2020-12-21 18:07:00 2145678572

Error state can be reproduced on a system:

.\zabbix_agentd.exe -t "wmi.get[root\cimv2,select AvgDisksecPerRead from Win32_PerfRawData_PerfDisk_Logicaldisk WHERE Name LIKE '%VD%']"
wmi.get[root\cimv2,select AvgDisksecPerRead from Win32_PerfRawData_PerfDisk_Logicaldisk WHERE Name='C:\\mount\\Tank-VD-0'] [u|18446744073578909317]

Running wmi query directly (shortly after):

gwmi -Query "select AvgDisksecPerRead from Win32_PerfRawData_PerfDisk_Logicaldisk WHERE Name LIKE '%VD%'"
AvgDisksecPerRead : 4001443676

Same system - 30min later:

.\zabbix_agentd.exe -t "wmi.get[root\cimv2,select AvgDisksecPerRead from Win32_PerfRawData_PerfDisk_Logicaldisk WHERE Name LIKE '%VD%']"
wmi.get[root\cimv2,select AvgDisksecPerRead from Win32_PerfRawData_PerfDisk_Logicaldisk WHERE Name='C:\\mount\\Tank-VD-0'] [u|82629443]
Running wmi query directly (shortly after):
gwmi -Query "select AvgDisksecPerRead from Win32_PerfRawData_PerfDisk_Logicaldisk WHERE Name LIKE '%VD%'"
AvgDisksecPerRead : 82668265

There's no real starting point here - it just occurs "sometimes".
The only thing i've noticed so far is that only values reaching the limit of (signed) int32 range (2147483647) seem to disappear (values are increasing due the nature of the measured value) and then the error occurs. There's just no explanation why the value sometimes decreases again.

Expectation would be that the values are returned correctly (as the native WMI query does).



 Comments   
Comment by Edgar Akhmetshin [ 2020 Dec 24 ]

Hello Daniel,

Could you please clarify - which exact patch level of the 2016/2019 used, so we can reproduce such behaviour. On Windows 10 Pro 20H2 i'm not able to reproduce with 5.2.3. Also this metrics are from storage pools/spaces or from direct attached storage?

Regards,
Edgar

Comment by Daniel [ 2020 Dec 24 ]

Hello Edgar,

we're using both with latest patches installed. This issue occurs on many systems, hardware (with storage spaces) and virtual machines. It's hard to find a common component of all systems. It doesn't matter if we're using wmi.get or wmi.getall. 

The only interesting thing is that when values are approaching int32 signed max (2147483647), the system produces incorrect values (always 20 digits) until some point when it starts over with small values and it repeats. Seems just some sort of overflow.

I checked the source code (agent, wmi). May there be an issue with detection of vector type in zabbix agent that results in an incorrect value conversion? 

Regards,

Daniel

Comment by Aigars Kadikis [ 2020 Dec 28 ]

While agent reports 20 digit number, try listing same counters through Windows PowerShell:

Get-WmiObject -Query "select AvgDisksecPerRead from Win32_PerfRawData_PerfDisk_Logicaldisk" 

Please make sure you are using a 64-bit agent on the 64-bit windows system. Show the output:

zabbix_agentd -V
Comment by Daniel [ 2020 Dec 28 ]

I've already noted the result of WMI query in the issue itself. There's an example shown when it's correct and incorrect (agent & wmi result).

We're already using x64 Version of the agent.

 

zabbix_agentd Win64 (service) (Zabbix) 5.0.2
Revision 352ca05870 13 July 2020, compilation time: Jul 13 2020 13:49:01
Copyright (C) 2020 Zabbix SIA
License GPLv2+: GNU GPL version 2 or later <http://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it according to
the license. There is NO WARRANTY, to the extent permitted by law.

 

Issue exists with any agent (testet vom 4.4 to latest build). When it occurs, all versions on the same system show the same behavior at the same time - so the issue does not occur related to agent runtime or anything else, it's propably reproducible depending on processed values.

Comment by Aigars Kadikis [ 2020 Dec 29 ]

Can reproduce the issue.

Windows version:

C:\>ver
Microsoft Windows [Version 10.0.18363.1256] 

Powershell output from a command prompt:

powershell -nologo -command "Get-WmiObject -Query \"select AvgDisksecPerRead from Win32_PerfRawData_PerfDisk_Logicaldisk\" | foreach { $_.AvgDisksecPerRead }"
2596463301
8261
2596471562

Agent1 version:

c:\zabbix>zabbix_agentd.exe -V
zabbix_agentd Win64 (service) (Zabbix) 5.0.7
Revision 1b3db7f980 21 December 2020, compilation time: Dec 21 2020 16:00:09

Copyright (C) 2020 Zabbix SIA
License GPLv2+: GNU GPL version 2 or later <http://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it according to
the license. There is NO WARRANTY, to the extent permitted by law.

This product includes software developed by the OpenSSL Project
for use in the OpenSSL Toolkit (http://www.openssl.org/).

Compiled with OpenSSL 1.1.1g  21 Apr 2020
Running with OpenSSL 1.1.1g  21 Apr 2020 

Output:

c:\zabbix>zabbix_get -s 127.0.0.1 -p 10050 -k "wmi.getall[root\cimv2,select AvgDisksecPerRead from Win32_PerfRawData_PerfDisk_Logicaldisk]"
[{"AvgDisksecPerRead":18446744072012421710},{"AvgDisksecPerRead":8261},{"AvgDisksecPerRead":18446744072012429971}]

Agent2 version:

c:\zabbix>"c:\Program Files\Zabbix Agent 2\zabbix_agent2.exe" -V
zabbix_agent2 Win64 (Zabbix) 5.0.7
Revision 1b3db7f980 21 December 2020, compilation time: 2020-12-21 16:04:36

Copyright (C) 2020 Zabbix SIA
License GPLv2+: GNU GPL version 2 or later <http://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it according to
the license. There is NO WARRANTY, to the extent permitted by law.

This product includes software developed by the OpenSSL Project
for use in the OpenSSL Toolkit (http://www.openssl.org/).

Compiled with OpenSSL 1.1.1d  10 Sep 2019
Running with OpenSSL 1.1.1d  10 Sep 2019
 

Has negative values in the output:

[{"AvgDisksecPerRead":-1697079031,"Name":"C:"},{"AvgDisksecPerRead":8261,"Name":"HarddiskVolume2"},{"AvgDisksecPerRead":-1697070770,"Name":"_Total"}]
Comment by Daniel [ 2021 Jun 28 ]

We're still hanging on this issue because building metrics on these values is not possible.

On investiation we noticed that we're indeed talking about an overflow - these errors occur seemingly always when the value increases above int32 max value (2147483647). The formula to "fix" these values is: uint32Max - (int64Max - value) or in numbers: 4294967295-(18446744073709500000-value). This works at least for values in range int32max - uint32max but has to be implemented using a calculation on item processing (like javascript) which is a lot of effort.

Maybe this helps to fix this issue.

Generated at Sun May 18 07:49:58 EEST 2025 using Jira 9.12.4#9120004-sha1:625303b708afdb767e17cb2838290c41888e9ff0.