Uploaded image for project: 'ZABBIX BUGS AND ISSUES'
  1. ZABBIX BUGS AND ISSUES
  2. ZBX-20164

Zabbix agent 2 crashes on Windows server 2022 - possibly wmi relatated crash

XMLWordPrintable

    • Team B
    • Sprint 82 (Nov 2021), Sprint 83 (Dec 2021)
    • 1

      Steps to reproduce:

      1. Create a host that represents a working Windows server 2022 machine ( i my case - Standard Evaluation - version 21H2)
      2. Add template "Template OS Windows by Zabbix agent" to this host
      3. Unlink the template
      4. Navigate to the list of items of this host and set update interval to 1 second for all items via mass update
      5. Start monitoring this host using a windows version of Zabbix agent2
      6. wait for some time (can take approximately 30 min) 

      Result: Agent crashes with the following output (but nothing is written to log):

      C:\Users\Administrator\Desktop\zabbix_agent2_old\bin>zabbix_agent2.exe
      Starting Zabbix Agent 2 [Windows_host]. (5.0.18rc1)
      Press Ctrl+C to exit.
      Exception 0xc0000005 0x0 0xffffffffffffffff 0x7ffb3b7cdf04
      PC=0x7ffb3b7cdf04
      signal arrived during external code execution
      
      syscall.Syscall(0x7ffb3b7cdd70, 0x1, 0xc0001cb040, 0x0, 0x0, 0x0, 0x0, 0x0)
              c:/go/src/runtime/syscall_windows.go:188 +0xe9
      syscall.(*Proc).Call(0xc0001d8d40, 0xc0002af8b8, 0x1, 0x1, 0xe339d0, 0xc00027d5c0, 0xa81389, 0x7ffb3b74e8f0)
              c:/go/src/syscall/dll_windows.go:171 +0x136
      github.com/go-ole/go-ole.VariantClear(0xc0001cb040, 0x0, 0x0)
              C:/Users/ZABBIX/go/pkg/mod/github.com/go-ole/[email protected]/com.go:248 +0x76
      github.com/go-ole/go-ole.(*VARIANT).Clear(...)
              C:/Users/ZABBIX/go/pkg/mod/github.com/go-ole/[email protected]/variant.go:47
      zabbix.com/pkg/wmi.clearOle(0xc0001cb040)
              C:/Users/ZABBIX/build-agents-dev/zabbix-5.0.18rc1/src/go/pkg/wmi/wmi.go:54 +0x36
      zabbix.com/pkg/wmi.isPropertyKeyProperty(0x291c2658, 0xcf3700, 0x0, 0x0)
              C:/Users/ZABBIX/build-agents-dev/zabbix-5.0.18rc1/src/go/pkg/wmi/wmi.go:90 +0x1cc
      zabbix.com/pkg/wmi.(*valueResult).write.func1.1(0xc0001cac00, 0x0, 0x0)
              C:/Users/ZABBIX/build-agents-dev/zabbix-5.0.18rc1/src/go/pkg/wmi/wmi.go:147 +0x224
      github.com/go-ole/go-ole/oleutil.ForEach(0x28fe2488, 0xc00027d918, 0x0, 0x0)
              C:/Users/ZABBIX/go/pkg/mod/github.com/go-ole/[email protected]/oleutil/oleutil.go:122 +0x20b
      zabbix.com/pkg/wmi.(*valueResult).write.func1(0xc0001caac0, 0x0, 0x0)
              C:/Users/ZABBIX/build-agents-dev/zabbix-5.0.18rc1/src/go/pkg/wmi/wmi.go:127 +0x18a
      github.com/go-ole/go-ole/oleutil.ForEach(0x290ae568, 0xc00027da88, 0x0, 0x0)
              C:/Users/ZABBIX/go/pkg/mod/github.com/go-ole/[email protected]/oleutil/oleutil.go:122 +0x20b
      zabbix.com/pkg/wmi.(*valueResult).write(0xc0004fe6e0, 0x290ae568, 0x0, 0x0)
              C:/Users/ZABBIX/build-agents-dev/zabbix-5.0.18rc1/src/go/pkg/wmi/wmi.go:114 +0x12f
      zabbix.com/pkg/wmi.performQuery(0xc0002af3c0, 0xa, 0xc00016e4c0, 0x3a, 0xf04240, 0xc0004fe6e0, 0x0, 0x0)
              C:/Users/ZABBIX/build-agents-dev/zabbix-5.0.18rc1/src/go/pkg/wmi/wmi.go:298 +0x43f
      zabbix.com/pkg/wmi.QueryValue(...)
              C:/Users/ZABBIX/build-agents-dev/zabbix-5.0.18rc1/src/go/pkg/wmi/wmi.go:306
      zabbix.com/plugins/windows/wmi.(*Plugin).Export(0x149de80, 0xc0002af3d0, 0x7, 0xc00020e1c0, 0x2, 0x2, 0xf1fbe0, 0xc0005fdd40, 0x0, 0x0, ...)
              C:/Users/ZABBIX/build-agents-dev/zabbix-5.0.18rc1/src/go/plugins/windows/wmi/wmi.go:44 +0x136
      zabbix.com/internal/agent/scheduler.(*directExporterTask).perform.func1(0xc0005fdd40, 0xf1c9a0, 0xc000250230, 0xc000106c80, 0x50)
              C:/Users/ZABBIX/build-agents-dev/zabbix-5.0.18rc1/src/go/internal/agent/scheduler/task.go:266 +0x492
      created by zabbix.com/internal/agent/scheduler.(*directExporterTask).perform
              C:/Users/ZABBIX/build-agents-dev/zabbix-5.0.18rc1/src/go/internal/agent/scheduler/task.go:250 +0x6c
      
      goroutine 1 [select, 6 minutes]:
      main.run(0xc0002512c0, 0x0)
              C:/Users/ZABBIX/build-agents-dev/zabbix-5.0.18rc1/src/go/cmd/zabbix_agent2/zabbix_agent2.go:147 +0x215
      main.main()
              C:/Users/ZABBIX/build-agents-dev/zabbix-5.0.18rc1/src/go/cmd/zabbix_agent2/zabbix_agent2.go:501 +0x1904
      
      goroutine 5 [select]:
      zabbix.com/internal/agent/scheduler.(*Manager).run(0xc000250230)
              C:/Users/ZABBIX/build-agents-dev/zabbix-5.0.18rc1/src/go/internal/agent/scheduler/manager.go:317 +0x1c4
      created by zabbix.com/internal/agent/scheduler.(*Manager).Start
              C:/Users/ZABBIX/build-agents-dev/zabbix-5.0.18rc1/src/go/internal/agent/scheduler/manager.go:457 +0x5f
      
      goroutine 6 [chan receive]:
      zabbix.com/internal/agent/resultcache.(*MemoryCache).run(0xc0000da240)
              C:/Users/ZABBIX/build-agents-dev/zabbix-5.0.18rc1/src/go/internal/agent/resultcache/memorycache.go:202 +0xb2
      created by zabbix.com/internal/agent/resultcache.(*MemoryCache).Start
              C:/Users/ZABBIX/build-agents-dev/zabbix-5.0.18rc1/src/go/internal/agent/resultcache/memorycache.go:232 +0x5f
      
      goroutine 7 [select]:
      zabbix.com/internal/agent/serverconnector.(*Connector).run(0xc0001a2000)
              C:/Users/ZABBIX/build-agents-dev/zabbix-5.0.18rc1/src/go/internal/agent/serverconnector/serverconnector.go:300 +0x207
      created by zabbix.com/internal/agent/serverconnector.(*Connector).Start
              C:/Users/ZABBIX/build-agents-dev/zabbix-5.0.18rc1/src/go/internal/agent/serverconnector/serverconnector.go:358 +0x77
      
      goroutine 8 [IO wait]:
      internal/poll.runtime_pollWait(0x2a0c0f08, 0x72, 0xf05160)
              c:/go/src/runtime/netpoll.go:203 +0x5c
      internal/poll.(*pollDesc).wait(0xc000016448, 0x72, 0xc1b400, 0x0, 0x0)
              c:/go/src/internal/poll/fd_poll_runtime.go:87 +0x4c
      internal/poll.(*ioSrv).ExecIO(0x149a130, 0xc000016298, 0xc00020e380, 0x1, 0x0, 0x43c)
              c:/go/src/internal/poll/fd_windows.go:228 +0x121
      internal/poll.(*FD).acceptOne(0xc000016280, 0x43c, 0xc00013b770, 0x2, 0x2, 0xc000016298, 0x5, 0x14, 0x5, 0x2)
              c:/go/src/internal/poll/fd_windows.go:896 +0xa9
      internal/poll.(*FD).Accept(0xc000016280, 0xc00006de08, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
              c:/go/src/internal/poll/fd_windows.go:930 +0x15f
      net.(*netFD).accept(0xc000016280, 0xadd6ca, 0xc00006def0, 0xadd6f8)
              c:/go/src/net/fd_windows.go:193 +0x7b
      net.(*TCPListener).accept(0xc0000986e0, 0x7f3, 0xc0002afac0, 0xe)
              c:/go/src/net/tcpsock_posix.go:139 +0x39
      net.(*TCPListener).Accept(0xc0000986e0, 0xc00020e340, 0xcf40e0, 0xc0004fe920, 0xcf40e0)
              c:/go/src/net/tcpsock.go:261 +0x6b
      zabbix.com/pkg/zbxcomms.(*Listener).Accept(0xc000098720, 0xb2d05e00, 0x1, 0x0, 0x1, 0x0)
              C:/Users/ZABBIX/build-agents-dev/zabbix-5.0.18rc1/src/go/pkg/zbxcomms/comms.go:307 +0x3e
      zabbix.com/internal/agent/serverlistener.(*ServerListener).run(0xc0002512c0)
              C:/Users/ZABBIX/build-agents-dev/zabbix-5.0.18rc1/src/go/internal/agent/serverlistener/serverlistener.go:71 +0x169
      created by zabbix.com/internal/agent/serverlistener.(*ServerListener).Start
              C:/Users/ZABBIX/build-agents-dev/zabbix-5.0.18rc1/src/go/internal/agent/serverlistener/serverlistener.go:111 +0x256
      
      goroutine 9 [syscall, 6 minutes]:
      os/signal.signal_recv(0x0)
              c:/go/src/runtime/sigqueue.go:147 +0xa3
      os/signal.loop()
              c:/go/src/os/signal/signal_unix.go:23 +0x29
      created by os/signal.Notify.func1
              c:/go/src/os/signal/signal.go:127 +0x4b
      
      goroutine 10 [syscall, 6 minutes, locked to thread]:
      syscall.Syscall(0x7ffb3b303920, 0x2, 0x5e0, 0xffffffff, 0x0, 0x0, 0x0, 0x0)
              c:/go/src/runtime/syscall_windows.go:188 +0xe9
      syscall.WaitForSingleObject(0x5e0, 0xc0ffffffff, 0x0, 0x0, 0x435d2d)
              c:/go/src/syscall/zsyscall_windows.go:738 +0x6b
      github.com/natefinch/npipe.waitForCompletion(0x5d0, 0xc000140160, 0xf05160, 0xc0001bbfd0, 0x0)
              C:/Users/ZABBIX/go/pkg/mod/github.com/natefinch/[email protected]/npipe_windows.go:182 +0x42
      github.com/natefinch/npipe.(*PipeListener).AcceptPipe(0xc0000e6000, 0x0, 0x0, 0x0)
              C:/Users/ZABBIX/go/pkg/mod/github.com/natefinch/[email protected]/npipe_windows.go:327 +0x2fa
      github.com/natefinch/npipe.(*PipeListener).Accept(0xc0000e6000, 0x0, 0x0, 0x0, 0x0)
              C:/Users/ZABBIX/go/pkg/mod/github.com/natefinch/[email protected]/npipe_windows.go:263 +0x36
      zabbix.com/internal/agent/remotecontrol.(*Conn).run(0xc000098740)
              C:/Users/ZABBIX/build-agents-dev/zabbix-5.0.18rc1/src/go/internal/agent/remotecontrol/remote.go:64 +0x4f
      created by zabbix.com/internal/agent/remotecontrol.(*Conn).Start
              C:/Users/ZABBIX/build-agents-dev/zabbix-5.0.18rc1/src/go/internal/agent/remotecontrol/remote.go:80 +0x56
      
      goroutine 11330 [select]:
      zabbix.com/internal/agent/scheduler.(*Manager).PerformTask(0xc000250230, 0xc000106c80, 0x50, 0xdf8475800, 0x1, 0x50, 0x0, 0x0, 0x0)
              C:/Users/ZABBIX/build-agents-dev/zabbix-5.0.18rc1/src/go/internal/agent/scheduler/manager.go:500 +0x246
      zabbix.com/internal/agent/serverlistener.(*passiveCheck).handleCheck(0xc00020e080, 0xc00017480d, 0x50, 0x7f3)
              C:/Users/ZABBIX/build-agents-dev/zabbix-5.0.18rc1/src/go/internal/agent/serverlistener/passivecheck.go:48 +0xd3
      created by zabbix.com/internal/agent/serverlistener.(*ServerListener).processConnection
              C:/Users/ZABBIX/build-agents-dev/zabbix-5.0.18rc1/src/go/internal/agent/serverlistener/serverlistener.go:61 +0x25a
      rax     0x5ade573fda4eef2b
      rbx     0x9
      rcx     0x291c3538
      rdi     0xc0001cb040
      rsi     0x0
      rbp     0x9
      rsp     0x2b0ffd50
      r8      0x0
      r9      0x0
      r10     0xbaac1f10365eb170
      r11     0xc0002af8b0
      r12     0x2000
      r13     0x1000ff7ffc
      r14     0x0
      r15     0x6000
      rip     0x7ffb3b7cdf04
      rflags  0x10202
      cs      0x33
      fs      0x53
      gs      0x2b
      

      Expected: no agent crashes should be observed

            asestakovs Aleksejs Sestakovs
            solonkins Sergejs Olonkins
            Team B
            Votes:
            1 Vote for this issue
            Watchers:
            7 Start watching this issue

              Created:
              Updated:
              Resolved: