[ZBX-20260] system.cpu.util unable to get usage data of cores on second NUMA node Created: 2021 Nov 23 Updated: 2024 Dec 14 Resolved: 2022 Aug 16 |
|
Status: | Closed |
Project: | ZABBIX BUGS AND ISSUES |
Component/s: | Agent (G) |
Affects Version/s: | 5.4.7 |
Fix Version/s: | 6.0.8rc1, 6.2.2rc1, 6.4.0alpha1, 6.4 (plan) |
Type: | Problem report | Priority: | Trivial |
Reporter: | Aleksejs Cankovs | Assignee: | Mihails Prihodko |
Resolution: | Fixed | Votes: | 2 |
Labels: | agent, items, lld | ||
Remaining Estimate: | Not Specified | ||
Time Spent: | Not Specified | ||
Original Estimate: | Not Specified | ||
Environment: |
Microsoft Windows Server 2012 R2 |
Attachments: |
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
Team: | |
Sprint: | Sprint 84 (Jan 2022), Sprint 85 (Feb 2022), Sprint 86 (Mar 2022), Sprint 87 (Apr 2022), Sprint 88 (May 2022), Sprint 89 (Jun 2022), Sprint 90 (Jul 2022), Sprint 91 (Aug 2022) |
Story Points: | 1 |
Description |
Steps to reproduce:
Result: CPU cores from the second NUMA node produce an error: Performance counter is not ready. Expected: |
Comments |
Comment by Mihails Prihodko [ 2022 Jun 03 ] |
[email protected], could you please provide the following information? This bug seems to be a rare one and I hope your input might help us to find the root cause. 1) What is the service pack version of the Microsoft Windows Server 2012 R2? 2) Please provide us the output of the following commands:
3) If possible, please also send the output for CoreInfo command, available here: https://docs.microsoft.com/en-us/sysinternals/downloads/coreinfo 4) Please run the debug version of the agent with additional debug outputs in it, which is available zabbix_agent-6.0.5-windows-amd64-2c64a901cf8_v2.zip 5) Have you tried using Zabbix agent 2? Is the reported issue observed with agent 2?
UPD 1: The same issue on forum: |
Comment by Aleksejs Cankovs [ 2022 Jun 06 ] |
Hello, The server on which we initially observed the problem was already decommissioned. But I found one, where the problem persists and it is windows 2016. Indeed, not all servers are affected, I collected some info, I will attach output of typeperf commands and coreinfo, but for debugging I need client with TLS support. And no, we did not try agent2 on Windows systems yet. <mprihodko>Please find the version with TLS (OpenSSL) support zabbix_agent-6.0.5-windows-amd64-openssl-2c64a901cf8_v2.zip |
Comment by Aleksejs Cankovs [ 2022 Jun 06 ] |
Interesting, server has counters only for 16 logical processors under "\Processor\". However, under "Processor Information" there are counters for both sockets. <mprihodko> Yes, indeed. This is probably the direction for the further investigation. When the number of logical processors is less or equal to 64, Zabbix agent 1 relies on the "old" "Processor" counter, and not on the "new" "Processor Information" counter. It looks to be the cause of this bug. The counters are initialized in init_cpu_collector() function around here https://git.zabbix.com/projects/ZBX/repos/zabbix/browse/src/zabbix_agent/cpustat.c?at=refs%2Fheads%2Ffeature%2FZBX-20985-6.0#125 |
Comment by Mihails Prihodko [ 2022 Jun 07 ] |
This problem seems to be similar to the one we have: https://stackoverflow.com/questions/28098082/unable-to-use-more-than-one-processor-group-for-my-threads-in-a-c-sharp-app |
Comment by Mihails Prihodko [ 2022 Jun 08 ] |
A possible fix is available for testing zabbix_agent-6.0.6rc1-windows-amd64-openssl_2945440bba8_fix.zip [email protected], could you please test it when you have time? Everybody with the same problem is also welcome to test it. |
Comment by Aleksejs Cankovs [ 2022 Jun 08 ] |
I just tested it on two identical servers, it seems to work. At least I'm getting some values from the agent. |
Comment by Mihails Prihodko [ 2022 Aug 16 ] |
Available in:
|