[ZBX-9143] Zabbix agent does not see some processes since Windows 7 Created: 2014 Dec 14  Updated: 2017 May 30  Resolved: 2015 Feb 10

Status: Closed
Project: ZABBIX BUGS AND ISSUES
Component/s: Agent (G)
Affects Version/s: 2.2.7, 2.4.2
Fix Version/s: 2.2.9rc1, 2.4.4rc1, 2.5.0

Type: Incident report Priority: Blocker
Reporter: Alexey Pustovalov Assignee: Unassigned
Resolution: Fixed Votes: 1
Labels: agent, permissions, proc.num, process, windows
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Windows 7, 2012 and newer.

Attachments: PNG File Screen Shot 2014-10-19 at 0.51.00.png     File zabbix_agentd.exe     File zabbix_agentd_old.exe     File zabbix_agentd_x64.exe     File zabbix_agentd_x64_old.exe    
Issue Links:
Duplicate
is duplicated by ZBX-5849 proc.num is not calculating the 32bit... Closed

 Description   

Zabbix agent does not see some processes using proc.num item key.

 Comments   
Comment by Oleg Ivanivskyi [ 2015 Jan 26 ]

If this issue is related to permissions it will be great to provide some details under ZBXNEXT-2553.

Comment by dimir [ 2015 Feb 03 ]

There are 2 ways to run Zabbix agent on Windows:

  • as a service
  • as a console application

I guess we should deal here with the case when agent is run as a service. Because first of all, console application is mostly meant for debugging and secondly, there are usually no permission problems (the same user is used to start an agent as console application, agent as a client and Zabbix get).

As filipp.sudanov already mentioned above, when we run Zabbix agent as a service it gets started as SYSTEM user. Let's get a bit into detail here. In my case (Windows 2008), here is what we get if we log user name (using GetUserName() to get the user name) from within the agent run as a service:

  2720:20150203:210637.830 Current user:"SYSTEM"

Now, I've added some logging to proc.num command and this is what I get on the agent when I run zabbix_get -s 127.0.0.1 -k proc.num to request data from it:

  2720:20150203:210637.830 Current user:"SYSTEM", requested proc name:""
  2720:20150203:210637.830 MATCH: (p:smss.exe u:SYSTEM)
  2720:20150203:210637.830 MATCH: (p:csrss.exe u:SYSTEM)
  2720:20150203:210637.830 MATCH: (p:wininit.exe u:SYSTEM)
  2720:20150203:210637.830 MATCH: (p:services.exe u:SYSTEM)
  2720:20150203:210637.830 MATCH: (p:lsass.exe u:SYSTEM)
  2720:20150203:210637.846 MATCH: (p:lsm.exe u:SYSTEM)
  2720:20150203:210637.846 MATCH: (p:svchost.exe u:SYSTEM)
  2720:20150203:210637.846 MATCH: (p:svchost.exe u:NETWORK SERVICE)
  2720:20150203:210637.846 MATCH: (p:svchost.exe u:LOCAL SERVICE)
  2720:20150203:210637.846 MATCH: (p:svchost.exe u:SYSTEM)
  2720:20150203:210637.846 MATCH: (p:svchost.exe u:SYSTEM)
  2720:20150203:210637.846 MATCH: (p:SLsvc.exe u:NETWORK SERVICE)
  2720:20150203:210637.846 MATCH: (p:svchost.exe u:LOCAL SERVICE)
  2720:20150203:210637.861 MATCH: (p:svchost.exe u:SYSTEM)
  2720:20150203:210637.861 MATCH: (p:svchost.exe u:NETWORK SERVICE)
  2720:20150203:210637.861 MATCH: (p:svchost.exe u:LOCAL SERVICE)
  2720:20150203:210637.861 MATCH: (p:spoolsv.exe u:SYSTEM)
  2720:20150203:210637.861 MATCH: (p:artstartsvc.exe u:SYSTEM)
  2720:20150203:210637.861 MATCH: (p:dsNcService.exe u:SYSTEM)
  2720:20150203:210637.861 MATCH: (p:FileZilla Server.exe u:SYSTEM)
  2720:20150203:210637.861 MATCH: (p:svchost.exe u:NETWORK SERVICE)
  2720:20150203:210637.877 MATCH: (p:svchost.exe u:LOCAL SERVICE)
  2720:20150203:210637.877 MATCH: (p:snmp.exe u:SYSTEM)
  2720:20150203:210637.877 MATCH: (p:svchost.exe u:NETWORK SERVICE)
  2720:20150203:210637.877 MATCH: (p:tlntsvr.exe u:SYSTEM)
  2720:20150203:210637.877 MATCH: (p:tvnserver.exe u:SYSTEM)
  2720:20150203:210637.877 MATCH: (p:svchost.exe u:SYSTEM)
  2720:20150203:210637.877 MATCH: (p:zabbix_agentd.exe u:SYSTEM)
  2720:20150203:210637.877 MATCH: (p:zabbix_agentd.exe u:SYSTEM)
  2720:20150203:210637.877 MATCH: (p:vmware-usbarbitrator.exe u:SYSTEM)
  2720:20150203:210637.893 MATCH: (p:msdtc.exe u:NETWORK SERVICE)
  2720:20150203:210637.893 MATCH: (p:csrss.exe u:SYSTEM)
  2720:20150203:210637.893 MATCH: (p:winlogon.exe u:SYSTEM)
  2720:20150203:210637.893 MATCH: (p:LogonUI.exe u:SYSTEM)
  2720:20150203:210637.893 MATCH: (p:taskeng.exe u:SYSTEM)
  2720:20150203:210637.893 MATCH: (p:zabbix_agentd.exe u:SYSTEM)
  2720:20150203:210637.908 MATCH: (p:wmiprvse.exe u:SYSTEM)
  2720:20150203:210637.908 MATCH: (p:csrss.exe u:SYSTEM)
  2720:20150203:210637.908 MATCH: (p:winlogon.exe u:SYSTEM)
  2720:20150203:210637.908 MATCH: (p:taskeng.exe u:Administrator)
  2720:20150203:210637.908 MATCH: (p:rdpclip.exe u:Administrator)
  2720:20150203:210637.908 MATCH: (p:Dwm.exe u:Administrator)
  2720:20150203:210637.908 MATCH: (p:Explorer.EXE u:Administrator)
  2720:20150203:210637.908 MATCH: (p:tvnserver.exe u:Administrator)
  2720:20150203:210637.924 MATCH: (p:jusched.exe u:Administrator)
  2720:20150203:210637.924 MATCH: (p:GoogleToolbarNotifier.exe u:Administrator)
  2720:20150203:210637.924 MATCH: (p:ProcessHacker.exe u:Administrator)
  2720:20150203:210637.924 MATCH: (p:unsecapp.exe u:Administrator)
  2720:20150203:210637.924 MATCH: (p:csrss.exe u:SYSTEM)
  2720:20150203:210637.924 MATCH: (p:winlogon.exe u:SYSTEM)
  2720:20150203:210637.924 MATCH: (p:taskeng.exe u:dimir)
  2720:20150203:210637.924 MATCH: (p:rdpclip.exe u:dimir)
  2720:20150203:210637.924 MATCH: (p:jucheck.exe u:Administrator)
  2720:20150203:210637.939 MATCH: (p:Dwm.exe u:dimir)
  2720:20150203:210637.939 MATCH: (p:Explorer.EXE u:dimir)
  2720:20150203:210637.939 MATCH: (p:tvnserver.exe u:dimir)
  2720:20150203:210637.939 MATCH: (p:jusched.exe u:dimir)
  2720:20150203:210637.939 MATCH: (p:cmd.exe u:dimir)
  2720:20150203:210637.939 MATCH: (p:cmd.exe u:dimir)
  2720:20150203:210637.939 MATCH: (p:cmd.exe u:Administrator)
  2720:20150203:210637.939 MATCH: (p:conime.exe u:Administrator)
  2720:20150203:210637.955 MATCH: (p:wuauclt.exe u:Administrator)
  2720:20150203:210637.955 MATCH: (p:firefox.exe u:Administrator)
  2720:20150203:210637.955 MATCH: (p:dsNetworkConnect.exe u:Administrator)
  2720:20150203:210637.955 MATCH: (p:notepad++.exe u:Administrator)
  2720:20150203:210637.955 MATCH: (p:LogonUI.exe u:SYSTEM)
  2720:20150203:210637.955 MATCH: (p:SLUI.exe u:dimir)
  2720:20150203:210637.955 MATCH: (p:cmd.exe u:Administrator)
  2720:20150203:210637.955 MATCH: (p:cmd.exe u:Administrator)
  2720:20150203:210637.955 MATCH: (p:mspdbsrv.exe u:Administrator)
  2720:20150203:210637.955 MATCH: (p:zabbix_agentd.exe u:SYSTEM)
  2720:20150203:210637.955 MATCH: (p:zabbix_get.exe u:Administrator)
  2720:20150203:210637.955 Sending back [72]

72 processes matched user SYSTEM (despite many are run by different user). And this is what I get if I run zabbix_agentd -t proc.num:

zabbix_agentd.exe [3132]: Current user:"Administrator", requested proc name:""
zabbix_agentd.exe [3132]: MATCH: (p:taskeng.exe u:Administrator)
zabbix_agentd.exe [3132]: MATCH: (p:rdpclip.exe u:Administrator)
zabbix_agentd.exe [3132]: MATCH: (p:Dwm.exe u:Administrator)
zabbix_agentd.exe [3132]: MATCH: (p:Explorer.EXE u:Administrator)
zabbix_agentd.exe [3132]: MATCH: (p:tvnserver.exe u:Administrator)
zabbix_agentd.exe [3132]: MATCH: (p:jusched.exe u:Administrator)
zabbix_agentd.exe [3132]: MATCH: (p:GoogleToolbarNotifier.exe u:Administrator)
zabbix_agentd.exe [3132]: MATCH: (p:ProcessHacker.exe u:Administrator)
zabbix_agentd.exe [3132]: MATCH: (p:unsecapp.exe u:Administrator)
zabbix_agentd.exe [3132]: MATCH: (p:jucheck.exe u:Administrator)
zabbix_agentd.exe [3132]: MATCH: (p:cmd.exe u:Administrator)
zabbix_agentd.exe [3132]: MATCH: (p:conime.exe u:Administrator)
zabbix_agentd.exe [3132]: MATCH: (p:wuauclt.exe u:Administrator)
zabbix_agentd.exe [3132]: MATCH: (p:firefox.exe u:Administrator)
zabbix_agentd.exe [3132]: MATCH: (p:dsNetworkConnect.exe u:Administrator)
zabbix_agentd.exe [3132]: MATCH: (p:notepad++.exe u:Administrator)
zabbix_agentd.exe [3132]: MATCH: (p:cmd.exe u:Administrator)
zabbix_agentd.exe [3132]: MATCH: (p:cmd.exe u:Administrator)
zabbix_agentd.exe [3132]: MATCH: (p:mspdbsrv.exe u:Administrator)
zabbix_agentd.exe [3132]: MATCH: (p:zabbix_agentd.exe u:Administrator)
proc.num                                      [u|20]

It doesn't matter which user I run zabbix_get by because it just sends request to agent over network, which actually contacts operating system.

So this is why the different results. If I specify process name we will get different results anyway, because of different users:

> zabbix_get.exe -s 127.0.0.1 -k proc.num[Explorer.EXE]
2

This one matched 2 different users running Explorer.EXE.

> zabbix_agentd.exe -c \dimir\zabbix_agentd.conf -t proc.num[Explorer.EXE]
zabbix_agentd.exe [3132]: Current user:"Administrator", requested proc name:"Explorer.EXE"
zabbix_agentd.exe [3708]: NO MATCH: (p:taskeng.exe u:Administrator)
zabbix_agentd.exe [3708]: NO MATCH: (p:rdpclip.exe u:Administrator)
zabbix_agentd.exe [3708]: NO MATCH: (p:Dwm.exe u:Administrator)
zabbix_agentd.exe [3708]: MATCH: (p:Explorer.EXE u:Administrator)
zabbix_agentd.exe [3708]: NO MATCH: (p:tvnserver.exe u:Administrator)
zabbix_agentd.exe [3708]: NO MATCH: (p:jusched.exe u:Administrator)
zabbix_agentd.exe [3708]: NO MATCH: (p:GoogleToolbarNotifier.exe u:Administrator)
zabbix_agentd.exe [3708]: NO MATCH: (p:ProcessHacker.exe u:Administrator)
zabbix_agentd.exe [3708]: NO MATCH: (p:unsecapp.exe u:Administrator)
zabbix_agentd.exe [3708]: NO MATCH: (p:jucheck.exe u:Administrator)
zabbix_agentd.exe [3708]: NO MATCH: (p:cmd.exe u:Administrator)
zabbix_agentd.exe [3708]: NO MATCH: (p:conime.exe u:Administrator)
zabbix_agentd.exe [3708]: NO MATCH: (p:wuauclt.exe u:Administrator)
zabbix_agentd.exe [3708]: NO MATCH: (p:firefox.exe u:Administrator)
zabbix_agentd.exe [3708]: NO MATCH: (p:dsNetworkConnect.exe u:Administrator)
zabbix_agentd.exe [3708]: NO MATCH: (p:notepad++.exe u:Administrator)
zabbix_agentd.exe [3708]: NO MATCH: (p:cmd.exe u:Administrator)
zabbix_agentd.exe [3708]: NO MATCH: (p:cmd.exe u:Administrator)
zabbix_agentd.exe [3708]: NO MATCH: (p:mmc.exe u:Administrator)
zabbix_agentd.exe [3708]: NO MATCH: (p:mdm.exe u:Administrator)
zabbix_agentd.exe [3708]: NO MATCH: (p:zabbix_agentd.exe u:Administrator)
proc.num[Explorer.EXE]                        [u|1]

This one matched only one.

The solution might be just documenting that properly.

Comment by richlv [ 2015 Feb 05 ]

could ZBX-9283 be the same ?

Comment by dimir [ 2015 Feb 09 ]

Attaching binaries for testing.

  • zabbix_agentd.exe (x86, fixed)
  • zabbix_agentd_x64.exe (x64, fixed)
  • zabbix_agentd_old.exe (x86, as it is now)
  • zabbix_agentd_x64_old.exe (x64, as it is now)

Please feel free to test.

Comment by dimir [ 2015 Feb 09 ]

Fixed in development branch svn://svn.zabbix.com/branches/dev/ZBX-9143

The main idea was taken from: https://msdn.microsoft.com/en-us/library/windows/desktop/ms686701(v=vs.85).aspx

Comment by Filipp Sudanov (Inactive) [ 2015 Feb 09 ]

Under x64 Win2012 R2 server it now shows as many processes, as Task Manager sees. Hidden processes like smss.exe are reported.

Comment by Andris Zeila [ 2015 Feb 10 ]

(1) Please check my changes in r52120

<dimir> CLOSED

Comment by Andris Zeila [ 2015 Feb 10 ]

Successfully tested

Comment by dimir [ 2015 Feb 10 ]

Fixed in pre-2.2.9 (r52123), pre-2.2.4 (r52126), pre-2.5.0 (r52130)

Comment by dimir [ 2015 Feb 10 ]

(2) [D] Upgrade notes.

sasha CLOSED

Comment by dimir [ 2015 Feb 10 ]

Some implementation details.

Before this issue Zabbix agent used method EnumProcesses() to get a list of running process IDs. If there was a process name filter, OpenProcess() call was issued on each process ID to get process handle. After that process name (executable) would be requested from that handle. A handle maybe requested with different access level. We were requesting a hanlde with PROCESS_QUERY_INFORMATION and PROCESS_VM_READ access rights. This was actually not needed and resulted in "access denied" in many cases and incorrect number of running processes reported by Zabbix agent.

In this issue we decided to use another method of getting a list of running processes, CreateToolhelp32Snapshot(). This method allows getting the list of all running processes along with process names (executables). This allows skipping unnecessary calls to OpenProcess(), which gave errors before.

OpenProcess() is still called when a user name is specified in //proc.num// parameter, e. g. proc.num[zabbix_agentd.exe,administrator].

More information:

Comment by dimir [ 2015 Mar 10 ]

This also fixed ZBX-5849 (32-bit agent would not list 64-bit processes when run on a 64-bit machine).

Generated at Thu Apr 25 14:50:10 EEST 2024 using Jira 9.12.4#9120004-sha1:625303b708afdb767e17cb2838290c41888e9ff0.