Uploaded image for project: 'ZABBIX FEATURE REQUESTS'
  2. ZBXNEXT-3006

A cache to provide multiple metrics of a single custom function


    • Sprint 2, Sprint 4, Sprint 5, Sprint 6, Sprint 7, Sprint 8, Sprint 9, Sprint 10, Sprint 11, Sprint 12, Sprint 13, Sprint 14, Sprint 15
    • 24

      Custom functions (items) based on External check, User parameter or loadable module often parse output or content of commands resp. files that include information for lots of metrics.

      Thanks to parameter support in items keys a single function can be used to extract the value of a particular metric only.
      When being interested in many of the included metrics or even all of them, then this can become quite inefficient. For each dedicated metric all information is requested just to ignore values from the other ones (see ZBXNEXT-103).

      In best case a loadable module function is used to access a pseudo file.
      A rather worse scenario could be the need to run an expensive command or command chain by User parameter:

      UserParameter=openvz.ubc[*],/usr/bin/sudo /usr/bin/tac /proc/user_beancounters | awk '/$1/{print $(NF-5+$2);exit}'

      The previous User parameter extracts one of five possible values for one of 20 possible resources from OpenVZ's User Beancounters table

      When being interested in every available metric this means 100 User parameter calls each obtaining the very same information per OpenVZ container - and there may be hundreds of containers per physical host.

      This ticket is about providing a way to improve this by caching all potential information in memory, so that (dynamic provided) custom functions can then extract data from this regularly updated cache.

      What I currently think of is introducing a new cache, lets say Bulk Parameter Cache, which may be populated by Bulk parameters.
      A Bulk user parameter could look like this:

      BulkUserParameter=openvz.ubc,1,sudo tac /proc/user_beancounters | awk ...

      What means the agent executes the deposited command every minute and memorizes the output in the designated cache.
      The output must follow a common syntax of course, lets say it must result in a list of key value pairs like this:

      # sudo tac /proc/user_beancounters | awk '$1=="uid"{for(i=4;i>=0;i--)h[i]=$(7-i);next}$6~/./{r=$(NF-5);for(i=0;i<5;i++)printf("openvz.ubc[%s,%s] %d\n",r,h[i],$(NF-i))}'
      openvz.ubc[kmemsize,failcnt] 0
      openvz.ubc[kmemsize,limit] 1217787766
      openvz.ubc[kmemsize,barrier] 1217604266
      openvz.ubc[kmemsize,maxheld] 793927680
      openvz.ubc[kmemsize,held] 764308562
      openvz.ubc[lockedpages,failcnt] 0
      openvz.ubc[lockedpages,limit] 1368
      openvz.ubc[lockedpages,barrier] 1368
      openvz.ubc[lockedpages,maxheld] 123
      --- SNIP --- SNAP --- SNIP --- SNAP ---

      On each cache updated new keys get appended, existing key values get updated and missing keys get removed from the cache.

      Ideally each in cache existing key becomes dynamically a valid item key available to passive as well as to active agent checks returning the current key value on demand.

      By this the expensive part is significant reduced to one call per minimum desired item update interval.

      Of course this should not be limited to Bulk user parameters but be also supported for Bulk external checks or for use in loadable modules.

      PS: OpenVZ has been chosen as example just because of being easy to parse.

            Unassigned Unassigned
            okkuv9xh Marc
            Team C
            10 Vote for this issue
            23 Start watching this issue