index-by-interfaceid.patch
[ZBX-8833] cache for dynamic snmp items indexes values by hostid and port Created: 2014 Sep 29 Updated: 2017 May 30 Resolved: 2014 Oct 31 |
|
| Status: | Closed |
| Project: | ZABBIX BUGS AND ISSUES |
| Component/s: | Proxy (P), Server (S) |
| Affects Version/s: | 2.0.13 |
| Fix Version/s: | 2.0.14rc1, 2.2.8rc1, 2.4.2rc1, 2.5.0 |
| Type: | Incident report | Priority: | Blocker |
| Reporter: | Aleksandrs Saveljevs | Assignee: | Unassigned |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | cache, snmp, snmpdynamicindex | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Attachments: |
index-by-interfaceid.patch
|
| Description |
|
Suppose we have the following SNMP item with dynamic index: ifInOctets["index","ifDescr","GigabitEthernet0/1"] What a poller does when it encounters this item for the first time, is it walks "ifDescr" tree, remembering at which index each value resides. In this case, suppose "GigabitEthernet0/1" resides at index 5, so it remembers that and returns the value of "ifInOctets.5" as the value of the item. When it encounters this item for the second time, the poller queries the cache and gets 5 as the index for "GigabitEthernet0/1", asks the device for "ifDescr.5" to see whether cache entry is still valid. If it is, it proceeds with "ifInOctets.5". Otherwise, it rebuilds the cache as described above. The problem is that this caching is done using (hostid, port, OID, value) tuples. In the case above, it will cache 5 for key (hostid, port, "ifDescr", "GigabitEthernet0/1"). Usually this works OK, but breaks when a host has multiple interfaces with the same port, but different IP addresses, as we have seen people use in practice. In Zabbix 2.0, we introduced multiple interfaces per host, but, apparently, the way this cache works was not updated. The practical implication is significantly increased CPU usage, because on a large number of occasions the cache has to be rebuilt. |
| Comments |
| Comment by Aleksandrs Saveljevs [ 2014 Sep 29 ] |
|
Attached is a sample patch that replaces the (hostid, port) part of the index with just interfaceid. |
| Comment by Alexander Vladishev [ 2014 Oct 09 ] |
|
(1) Will better use CONN + PORT to cache instead of interfaceid.
asaveljevs RESOLVED in r50124. wiper CLOSED |
| Comment by Andris Zeila [ 2014 Oct 27 ] |
|
(2) Would be better to separate memory management functions from other parameters in zbx_hashset_create_ext() function. Also vectors have clean() function, so we should keep similar naming: void zbx_hashset_create_ext(zbx_hashset_t *hs, size_t init_size, zbx_hash_func_t hash_func, zbx_compare_func_t compare_func, zbx_clean_func_t clean_func, zbx_mem_malloc_func_t mem_malloc_func, zbx_mem_realloc_func_t mem_realloc_func, zbx_mem_free_func_t mem_free_func ) asaveljevs RESOLVED in r50220. wiper CLOSED |
| Comment by Andris Zeila [ 2014 Oct 28 ] |
|
Successfully tested, but please see the (2) subissue |
| Comment by Aleksandrs Saveljevs [ 2014 Oct 28 ] |
|
(3) Fix for 2.0, which simply replaces indexing by (hostid, port) with (addr, port) is available in development branch svn://svn.zabbix.com/branches/dev/ZBX-8833-2.0 . wiper CLOSED |
| Comment by Aleksandrs Saveljevs [ 2014 Oct 28 ] |
|
(4) After this issue is merged into trunk, we shall create a separate branch under this issue and change "zbx_vector_ptr_clear(zbx_vector_ptr_t *vector, zbx_mem_free_func_t free_func)" to "zbx_vector_ptr_clear(zbx_vector_ptr_t *vector, zbx_clean_func_t clean_func)". asaveljevs Fixed in development branch svn://svn.zabbix.com/branches/dev/ZBX-8833 . RESOLVED. wiper after lengthy discussions was decided that vector should not free values array in its clear() and clear_ext() functions. asaveljevs RESOLVED in r50294 through r50301. wiper CLOSED asaveljevs Merged into pre-2.5.0 (trunk) r50327. |
| Comment by Aleksandrs Saveljevs [ 2014 Oct 28 ] |
|
Fixed in pre-2.0.14 r50235, pre-2.2.8 r50240, pre-2.4.2 r50241, and pre-2.5.0 (trunk) r50242. |