[ZBXNEXT-760] [PATCH] SNMP "count"/"sum" operations, regexp matching and better index caching Created: 2011 Apr 20  Updated: 2021 Nov 16  Resolved: 2021 Nov 16

Status: Closed
Project: ZABBIX FEATURE REQUESTS
Component/s: Proxy (P), Server (S)
Affects Version/s: 1.8.5, 1.9.3 (alpha)
Fix Version/s: None

Type: New Feature Request Priority: Major
Reporter: Ilia Sotnikov Assignee: Unassigned
Resolution: Fixed Votes: 38
Labels: patch, performance, snmp, triggerfunctions
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: PNG File Screenshot 2021-06-16 at 14.36.11.png     PNG File Screenshot 2021-06-16 at 14.38.10.png     File zabbix-1.8.5-snmp.diff     File zabbix-snmp-svn-trunk-19060.diff    
Issue Links:
Duplicate
is duplicated by ZBXNEXT-3689 SNMp Dynamic indexes counting Closed
is duplicated by ZBXNEXT-179 count snmp entries by regexp Closed

 Description   

This patch adds the following features for SNMP monitoring:

  • "count" operation, which counts SNMP indexes with corresponding value
  • "sum" operation, which calculates the sum of SNMP values by SNMP indexes with corresponding value
  • textual representation of SNMP value is used when comparing, existing Zabbix versions work well only on string values. Thus, SNMP index being integer or quoted string will be matched correcly
  • regex matching is used for comparing SNMP index values in all modes: "index", "count", "sum"
  • internal caching infrstructure (hashset) is used to store SNMP indexes, which improves effectiveness and reuses existing code. Efficiency is achieved by sharing the SNMP index cache between SNMP poller processes. To control the size of the cache additional configuration option was introduced, 'SNMPCacheSize'

Several use cases for features implemented are below.

Use case 1. Counting number of specific processes (eg. 'httpd')
hrSWRunName["count","hrSWRunName","httpd"]

Use case 2. Getting the RAM amount consumed by specific process (eg. 'snmpd')
hrSWRunPerfMem["sum","hrSWRunName","snmpd"]

Use case 3. Get the version of packege installed (maybe OS specific and depends on SNMP daemon)
hrSWInstalledName["index","hrSWInstalledName","net-snmp-.\+"]

Use case 4. Get the number of processes running. Note that 'running' value is internally represented as integer with value '2'.
hrSWRunStatus["count","hrSWRunStatus","running"]



 Comments   
Comment by Oleksii Zagorskyi [ 2011 Apr 20 ]

Related (or similar) issue ZBX-3192

Comment by Ilia Sotnikov [ 2011 Apr 20 ]

Oleksiy, good catch - yes, the patch keeps separate cache entries for different hosts. This is additional point of improvement indeed. Will update the description appropriately.

Comment by Ilia Sotnikov [ 2011 Apr 21 ]

Another important point I forgot to mention is that "sum"/"count" operations aren't cached by design.

Comment by Ilia Sotnikov [ 2011 Apr 21 ]

Found manual merge bug - fixed and re-uploaded

Comment by dimir [ 2011 May 30 ]

Great work, thank you! I have tested it with 1.8 and it works great. One thing that I noticed, when you log hostid you use "%lu" format and the log entry that contains hostid is corrupted. Using "%llu" fixes that. E. g.:

zabbix_log(LOG_LEVEL_DEBUG, "In %s(hostid:%llu,oid:%s,value:%s,index:%d)",
__function_name, hostid, oid, value, index);

We should definitely consider it in the future.

Comment by Ilia Sotnikov [ 2011 May 31 ]

Vladimir,
much appreciated your review and feedback. Regarding, the %llu - sorry, my bad Should the patches be regenerated?

Comment by Aleksandrs Saveljevs [ 2011 May 31 ]

Just a friendly reminder that ZBX_FS_UI64 format specifier should be used to output values of type zbx_uint64_t.

Comment by Ilia Sotnikov [ 2011 May 31 ]

Aleksandrs,
ah, saw the modifier but it didn't come to my mind. This is preferable variant, in my opinion. Could regenerate the patches today or tomorrow.

Comment by Ilia Sotnikov [ 2011 May 31 ]

Patches regenerated to address %lu modifiers being used to output zbx_uint64_t - ZBX_FS_UI64 is used instead. This applies to 'hostid' spotted by Vladimir, as well as to sum/count values

Comment by dimir [ 2011 May 31 ]

Thank you for the update! Your patch is great but unfortunately we can not integrate it into 1.8 as we do mostly bugfixing until 2.0 and we can not allow such big changes at this moment. However after 2.0 release I suggest this be brought to a discussion again.

We have discovered another small issue, there might be 2 snmp targets running on the same host on different ports. So

item->host.hostid and
item->snmp_port

should be considered in order to correctly identify the target. This will be included in the fix for ZBX-3192 . And currently we are left with a separate cache instance for each poller.

Comment by Ilia Sotnikov [ 2011 May 31 ]

Vladimir,
thanks for good catch - indeed, being rare the situation doesn't become impossible
Ok, understood - will update the patches to include snmp_port into hashing algorithm and post them here. Then we could reiterate when you'll be ready to integrate them.

Comment by richlv [ 2011 Aug 31 ]

while at it, might also throw in min/max

Comment by Ilia Sotnikov [ 2011 Aug 31 ]

Rich,
interesting idea, should be rather easy to implement. Do we have plans/deadlines to align with?

Comment by richlv [ 2011 Sep 09 ]

Ilia, this isn't really strictly planned, so there are no guarantees of patch going in - i wouldn't want to imply otherwise. as such, the "schedule" is fully up to you

Comment by Jon [ 2012 Aug 26 ]

Has this patch made it into 2.0 or future plans. I can see this being a very useful feature.

Comment by dimir [ 2012 Aug 28 ]

It has not. But indeed, the idea is good. Keep voting.

Comment by Shriharsh Thanvi [ 2016 Sep 20 ]

I would like to bring attention back at this issue.
I would say dynamic indexes are most interesting feature of Zabbix.
No problem with patch, but it still needs to be applied !

And not sure if the patch is still applicable for 2.2, 3.0 or even 3.2 for that matter.

Comment by Michael P [ 2017 Feb 09 ]

I hope that this feature will not go unnoticed as it would be very helpful for for tracking Wireless Performance Data, as seen in my (already closed) request:
ZBXNEXT-3689

Comment by Jon Smith [ 2017 Aug 13 ]

I am a new Zabbix user and am just migrating all my monitoring from Nagios/Cacti. This feature would be icing on the cake at the moment. Like Michael above, my specific use case is monitoring high level numbers on a wireless controller.

Comment by dimir [ 2017 Aug 25 ]

It looks like there is a request to implement such feature using more general approach: ZBXNEXT-1420. Does anybody already use LLD to discover such metrics? Is it working for you? Do you think ZBXNEXT-1420 would help achieving your goals?

Comment by Franck VGRT [ 2020 Mar 16 ]

Hello, is this feature available on the last zabbix versions ? I'm using 4.4

I would like to count how many users are connected to a Stormshield VPN using this MIB https://www.stormshield.com/wp-content/uploads/STORMSHIELD-AUTHUSERS-MIB.txt

I can get the users list with snmpwalk on STORMSHIELD-AUTHUSERS-MIB::snsAuthUsersName but i would like to count it.

Is it possible ?

Comment by Alex Kalimulin [ 2021 Jun 16 ]

Is this still an issue in the recent versions of Zabbix? SNMP OID discovery[] item along with JavaScript preprocessing can easily do all sorts of custom calculations.

For example, I have a router that returns a bunch of network interfaces of all kinds (ethernet, wlans, bridges etc) under vendor-specific OID. I want to calculate only the number of Ethernet interfaces. Then I add the item like this:

In the Preprocessing tab I add the following script:

var c=0;
JSON.parse(value).forEach(function(e){c+=e['{#SNMPVALUE}'].includes('ether');});
return c;

That's it!

Alternatively, if you need to add discovered SNMP entities as separate items and then do calculations (e.g. count total traffic for all network interfaces) you can take advantage of the new calculated items feature (possible since 5.4) by using formulas like this:

sum(last_foreach(//snmp.if.key[*]))

This will get the last values for all interfaces (note wildcard in arguments) and calculate the total with sum().

Comment by Alexei Vladishev [ 2021 Nov 16 ]

I am closing it for now. Feel free to re-open if you think we missed something.

Generated at Fri Mar 29 15:53:45 EET 2024 using Jira 9.12.4#9120004-sha1:625303b708afdb767e17cb2838290c41888e9ff0.