-
Incident report
-
Resolution: Duplicate
-
Trivial
-
None
-
5.4.0
-
None
Steps to reproduce:
Upgrade from 5.2 to 5.4.0
vmware collector eats 100% cpu for hours
Result:
--------
ps -Af|fgrep vmware
zabbix 103077 102704 99 11:18 ? 00:43:54 /usr/sbin/zabbix_server: vmware collector #1 [updated 0, removed 0 VMware services in 0.000079 sec, querying VMware services] zabbix 103078 102704 0 11:18 ? 00:00:00 /usr/sbin/zabbix_server: vmware collector #2 [updated 0, removed 0 VMware services in 0.000073 sec, idle 5 sec] zabbix 103079 102704 0 11:18 ? 00:00:00 /usr/sbin/zabbix_server: vmware collector #3 [updated 0, removed 0 VMware services in 0.000094 sec, idle 5 sec]
--------
logs with zabbix-server zabbix_server -R log_level_increase="vmware collector" on maximum
May 28 12:04:17 monitoring docker-zabbix-server: 233:20210528:090417.339 zbx_setproctitle() title:'vmware collector #2 [updated 0, removed 0 VMware services in 0.000059 sec, querying VMware services]' May 28 12:04:17 monitoring docker-zabbix-server: 233:20210528:090417.339 zbx_setproctitle() title:'vmware collector #2 [updated 0, removed 0 VMware services in 0.000083 sec, idle 5 sec]' May 28 12:04:17 monitoring docker-zabbix-server: 234:20210528:090417.348 zbx_setproctitle() title:'vmware collector #3 [updated 0, removed 0 VMware services in 0.000030 sec, querying VMware services]' May 28 12:04:17 monitoring docker-zabbix-server: 234:20210528:090417.348 zbx_setproctitle() title:'vmware collector #3 [updated 0, removed 0 VMware services in 0.000067 sec, idle 5 sec]' May 28 12:04:19 monitoring docker-zabbix-server: 232:20210528:090419.970 In vmware_hv_get_ds_access() for DS:datastore-10777 May 28 12:04:19 monitoring docker-zabbix-server: 232:20210528:090419.971 End of vmware_hv_get_ds_access() mountinfo:15 May 28 12:04:22 monitoring docker-zabbix-server: 233:20210528:090422.339 zbx_setproctitle() title:'vmware collector #2 [updated 0, removed 0 VMware services in 0.000083 sec, querying VMware services]' May 28 12:04:22 monitoring docker-zabbix-server: 233:20210528:090422.340 zbx_setproctitle() title:'vmware collector #2 [updated 0, removed 0 VMware services in 0.000080 sec, idle 5 sec]' May 28 12:04:22 monitoring docker-zabbix-server: 234:20210528:090422.350 zbx_setproctitle() title:'vmware collector #3 [updated 0, removed 0 VMware services in 0.000067 sec, querying VMware services]' May 28 12:04:22 monitoring docker-zabbix-server: 234:20210528:090422.350 zbx_setproctitle() title:'vmware collector #3 [updated 0, removed 0 VMware services in 0.000046 sec, idle 5 sec]'
--------
strace on a hung PID
[root@monitoring zbx-instance]# strace -p 103077 strace: Process 103077 attached munmap(0x7fbb7eef9000, 4096) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fbb7eef9000 munmap(0x7fbb7eef9000, 4096) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fbb7eef9000 munmap(0x7fbb7eef9000, 4096) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fbb7eef9000 munmap(0x7fbb7eef9000, 4096) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fbb7eef9000 munmap(0x7fbb7eef9000, 4096) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fbb7eef9000 munmap(0x7fbb7eef9000, 4096) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fbb7eef9000 munmap(0x7fbb7eef9000, 4096) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fbb7eef9000 munmap(0x7fbb7eef9000, 4096) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fbb7eef9000 munmap(0x7fbb7eef9000, 4096) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fbb7eef9000 munmap(0x7fbb7eef9000, 4096) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fbb7eef9000 munmap(0x7fbb7eef9000, 4096) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fbb7eef9000 munmap(0x7fbb7eef9000, 4096) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fbb7eef9000 munmap(0x7fbb7eef9000, 4096) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fbb7eef9000 munmap(0x7fbb7eef9000, 4096) = 0
- duplicates
-
ZBX-19114 VMWare Collectors hangling. No data for all VMWare metrics
- Closed