Uploaded image for project: 'ZABBIX BUGS AND ISSUES'
  1. ZABBIX BUGS AND ISSUES
  2. ZBX-17065

strange lines in "/proc/diskstats" cause multiple issues on template for block devices

XMLWordPrintable

    • Sprint 60 (Jan 2020), Sprint 61 (Feb 2020), Sprint 62 (Mar 2020), Sprint 63 (Apr 2020), Sprint 64 (May 2020), Sprint 65 (Jun 2020)
    • 1

      In template "Template Module Linux block devices by Zabbix agent active" LLD rule "Block devices discovery" constantly had this error:

      Cannot accurately apply filter: no value received for macro "{#DEVNAME}".
Cannot accurately apply filter: no value received for macro "{#DEVNAME}".
Cannot accurately apply filter: no value received for macro "{#DEVNAME}".
Cannot accurately apply filter: no value received for macro "{#DEVNAME}".
Cannot accurately apply filter: no value received for macro "{#DEVNAME}".
....

      The LLD rule is a dependent item, which takes "$.lld" JSONPath.
      Its master item takes data by "vfs.file.contents[/proc/diskstats]" agent key and then process it by this JavaScript:

      var parsed = value.split("\n").reduce(function(acc, x, i) {
  parts = x.trim().split(/ +/)
  acc["values"][parts[2]] = parts
  acc["lld"].push({"{#DEVNAME}":parts[2]})
  return acc;
}, {"values":{}, "lld": []});

return JSON.stringify(parsed);

      Resulting JSON of it (we created temporal dependent to take "$.lld" only) is this:

      [{"{#DEVNAME}":"ram0"},{"{#DEVNAME}":"ram1"},{"{#DEVNAME}":"ram2"},{"{#DEVNAME}":"ram3"},{"{#DEVNAME}":"ram4"},{"{#DEVNAME}":"ram5"},{"{#DEVNAME}":"ram6"},{"{#DEVNAME}":"ram7"},{"{#DEVNAME}":"ram8"},{"{#DEVNAME}":"ram9"},{"{#DEVNAME}":"ram10"},{"{#DEVNAME}":"ram11"},{"{#DEVNAME}":"ram12"},{"{#DEVNAME}":"ram13"},{"{#DEVNAME}":"ram14"},{"{#DEVNAME}":"ram15"},{"{#DEVNAME}":"loop0"},{"{#DEVNAME}":"loop1"},{"{#DEVNAME}":"loop2"},{"{#DEVNAME}":"loop3"},{"{#DEVNAME}":"loop4"},{"{#DEVNAME}":"loop5"},{"{#DEVNAME}":"loop6"},{"{#DEVNAME}":"loop7"},{"{#DEVNAME}":"sdar"},{"{#DEVNAME}":"sdar1"},{"{#DEVNAME}":"sdar2"},{"{#DEVNAME}":"dm-6"},{"{#DEVNAME}":"sdft"},{"{#DEVNAME}":"dm-28"},{"{#DEVNAME}":"dm-30"},{"{#DEVNAME}":"dm-89"},{"{#DEVNAME}":"dm-90"},{"{#DEVNAME}":"dm-92"},{"{#DEVNAME}":"dm-95"},{"{#DEVNAME}":"dm-96"},{"{#DEVNAME}":"dm-97"},{"{#DEVNAME}":"dm-98"},{"{#DEVNAME}":"dm-99"},{"{#DEVNAME}":"dm-101"},{"{#DEVNAME}":"dm-102"},{"{#DEVNAME}":"dm-103"},{"{#DEVNAME}":"dm-106"},{"{#DEVNAME}":"dm-107"},{"{#DEVNAME}":"dm-110"},{"{#DEVNAME}":"dm-111"},{"{#DEVNAME}":"dm-112"},{"{#DEVNAME}":"dm-113"},{"{#DEVNAME}":"dm-114"},{"{#DEVNAME}":"dm-115"},{"{#DEVNAME}":"dm-116"},{"{#DEVNAME}":"dm-117"},{"{#DEVNAME}":"dm-119"},{},{"{#DEVNAME}":"dm-122"},{"{#DEVNAME}":"dm-123"},{"{#DEVNAME}":"dm-124"},{"{#DEVNAME}":"dm-125"},{"{#DEVNAME}":"dm-127"},{"{#DEVNAME}":"dm-128"},{"{#DEVNAME}":"dm-129"},{"{#DEVNAME}":"dm-130"},{"{#DEVNAME}":"dm-131"},{"{#DEVNAME}":"dm-132"},{"{#DEVNAME}":"dm-133"},{"{#DEVNAME}":"dm-134"},{"{#DEVNAME}":"dm-135"},{"{#DEVNAME}":"dm-136"},{"{#DEVNAME}":"dm-137"},{"{#DEVNAME}":"dm-138"},{"{#DEVNAME}":"dm-139"},{"{#DEVNAME}":"dm-140"},{"{#DEVNAME}":"dm-141"},{"{#DEVNAME}":"dm-142"},{"{#DEVNAME}":"dm-143"},{"{#DEVNAME}":"sdbd"},{"{#DEVNAME}":"sdbd1"},{"{#DEVNAME}":"sdbt"},{"{#DEVNAME}":"dm-40"},{"{#DEVNAME}":"dm-45"},{"{#DEVNAME}":"sdi"},{"{#DEVNAME}":"sdi1"},{"{#DEVNAME}":"dm-4"},{"{#DEVNAME}":"sdbb"},{"{#DEVNAME}":"sdbb1"},{},{"{#DEVNAME}":"dm-18"},{"{#DEVNAME}":"dm-19"},{"{#DEVNAME}":"dm-27"},{"{#DEVNAME}":"sdcv"},{"{#DEVNAME}":"sddl"},{"{#DEVNAME}":"dm-41"},{"{#DEVNAME}":"sdef"},{"{#DEVNAME}":"dm-49"},{"{#DEVNAME}":"sdem"},{"{#DEVNAME}":"dm-58"},{"{#DEVNAME}":"dm-60"},{"{#DEVNAME}":"sdfw"},{"{#DEVNAME}":"sdfz"},{"{#DEVNAME}":"dm-79"},{"{#DEVNAME}":"sdga"},{"{#DEVNAME}":"dm-82"},{"{#DEVNAME}":"sdge"},{"{#DEVNAME}":"dm-85"},{"{#DEVNAME}":"dm-88"},{"{#DEVNAME}":"sdgi"},{"{#DEVNAME}":"sdgk"},{"{#DEVNAME}":"dm-91"},{"{#DEVNAME}":"dm-151"},{"{#DEVNAME}":"sdhz"},{"{#DEVNAME}":"sdib"},{"{#DEVNAME}":"dm-152"},{"{#DEVNAME}":"sdie"},{"{#DEVNAME}":"dm-153"},{},{"{#DEVNAME}":"sdih"},{"{#DEVNAME}":"dm-154"},{"{#DEVNAME}":"sdiu"},{"{#DEVNAME}":"dm-159"},{"{#DEVNAME}":"sdiv"},{},{"{#DEVNAME}":"dm-160"},{"{#DEVNAME}":"sdja"},{"{#DEVNAME}":"dm-161"},{"{#DEVNAME}":"sdjd"},{"{#DEVNAME}":"dm-162"},{"{#DEVNAME}":"sdjm"},{"{#DEVNAME}":"dm-165"},{"{#DEVNAME}":"sdjr"},{"{#DEVNAME}":"dm-166"},{"{#DEVNAME}":"dm-173"},{},{"{#DEVNAME}":"sdkl"},{"{#DEVNAME}":"sdkm"},{"{#DEVNAME}":"dm-174"},{"{#DEVNAME}":"dm-175"},{"{#DEVNAME}":"sdkt"},{"{#DEVNAME}":"sdku"},{"{#DEVNAME}":"dm-176"},{"{#DEVNAME}":"dm-177"},{"{#DEVNAME}":"sdkz"},{"{#DEVNAME}":"sdlb"},{"{#DEVNAME}":"dm-178"},{"{#DEVNAME}":"dm-179"},{"{#DEVNAME}":"sdld"},{"{#DEVNAME}":"sdlf"},{"{#DEVNAME}":"dm-180"},{"{#DEVNAME}":"sdli"},{"{#DEVNAME}":"dm-181"},{"{#DEVNAME}":"sdll"},{"{#DEVNAME}":"dm-182"},{},{"{#DEVNAME}":"dm-183"},{"{#DEVNAME}":"sdlr"},{"{#DEVNAME}":"sdlt"},{"{#DEVNAME}":"dm-184"},{"{#DEVNAME}":"sdlu"},{"{#DEVNAME}":"dm-185"},{"{#DEVNAME}":"sdly"},{"{#DEVNAME}":"dm-186"},{"{#DEVNAME}":"sdmc"},{"{#DEVNAME}":"dm-187"},{"{#DEVNAME}":"sdme"},{"{#DEVNAME}":"dm-188"},{"{#DEVNAME}":"sdmg"},{},{"{#DEVNAME}":"dm-189"},{"{#DEVNAME}":"sdml"},{"{#DEVNAME}":"dm-190"},{"{#DEVNAME}":"dm-191"},{"{#DEVNAME}":"sdmn"},{"{#DEVNAME}":"sdmp"},{},{"{#DEVNAME}":"dm-192"},{"{#DEVNAME}":"dm-193"},{"{#DEVNAME}":"sdmt"},{"{#DEVNAME}":"sdmv"},{"{#DEVNAME}":"dm-194"},{"{#DEVNAME}":"sdmz"},{"{#DEVNAME}":"dm-195"},{"{#DEVNAME}":"sdna"},{"{#DEVNAME}":"dm-196"},{"{#DEVNAME}":"sdne"},{"{#DEVNAME}":"dm-197"},{"{#DEVNAME}":"sdnf"},{"{#DEVNAME}":"dm-198"},{"{#DEVNAME}":"sdf"},{},{"{#DEVNAME}":"sdf1"},{"{#DEVNAME}":"sdf2"},{},{"{#DEVNAME}":"sdk"},{"{#DEVNAME}":"sdk1"},{"{#DEVNAME}":"sdm"},{"{#DEVNAME}":"sdm1"},{"{#DEVNAME}":"sdp"},{"{#DEVNAME}":"sdp1"},{},{"{#DEVNAME}":"sdn"},{"{#DEVNAME}":"sdn1"},{"{#DEVNAME}":"sdo"},{"{#DEVNAME}":"sdo1"},{"{#DEVNAME}":"sdq"},{"{#DEVNAME}":"sdq1"},{"{#DEVNAME}":"sdr"},{"{#DEVNAME}":"sdr1"},{"{#DEVNAME}":"sdx"},{"{#DEVNAME}":"sdx1"},{"{#DEVNAME}":"sdac"},{"{#DEVNAME}":"sdac1"},{"{#DEVNAME}":"sdy"},{"{#DEVNAME}":"sdy1"},{"{#DEVNAME}":"sdad"},{"{#DEVNAME}":"sdad1"},{"{#DEVNAME}":"sdaf"},{},{"{#DEVNAME}":"sdaf1"},{"{#DEVNAME}":"sdah"},{"{#DEVNAME}":"sdah1"},{"{#DEVNAME}":"sdai"},{"{#DEVNAME}":"sdai1"},{"{#DEVNAME}":"sdaj"},{"{#DEVNAME}":"sdaj1"},{"{#DEVNAME}":"sdal"},{"{#DEVNAME}":"sdal1"},{"{#DEVNAME}":"sdak"},{"{#DEVNAME}":"sdak1"},{"{#DEVNAME}":"sdam"},{"{#DEVNAME}":"sdam1"},{"{#DEVNAME}":"sdan"},{},{"{#DEVNAME}":"sdan1"},{"{#DEVNAME}":"sdao"},{"{#DEVNAME}":"sdao1"},{"{#DEVNAME}":"sdap"},{"{#DEVNAME}":"sdap1"},{"{#DEVNAME}":"sdat"},{"{#DEVNAME}":"sdat1"},{},{"{#DEVNAME}":"sdaq"},{"{#DEVNAME}":"sdaq1"},{"{#DEVNAME}":"sdau"},{"{#DEVNAME}":"sdau1"},{"{#DEVNAME}":"sdav"},{"{#DEVNAME}":"sdav1"},{"{#DEVNAME}":"sday"},{"{#DEVNAME}":"sday1"},{"{#DEVNAME}":"sday2"},{"{#DEVNAME}":"sdba"},{"{#DEVNAME}":"sdba1"},{"{#DEVNAME}":"sdbe"},{"{#DEVNAME}":"sdbe1"},{"{#DEVNAME}":"sdbf"},{},{"{#DEVNAME}":"sdbf1"},{"{#DEVNAME}":"sdbg"},{"{#DEVNAME}":"sdbg1"},{"{#DEVNAME}":"sdbh"},{"{#DEVNAME}":"sdbh1"},{"{#DEVNAME}":"sdbi"},{},{"{#DEVNAME}":"sdbi1"},{"{#DEVNAME}":"sdbj"},{"{#DEVNAME}":"sdbj1"},{"{#DEVNAME}":"sdbp"},{"{#DEVNAME}":"sdbp1"},{"{#DEVNAME}":"sdbq"},{"{#DEVNAME}":"sdbq1"},{"{#DEVNAME}":"sdbu"},{},{"{#DEVNAME}":"sdbu1"},{},{"{#DEVNAME}":"sdbv"},{"{#DEVNAME}":"sdbv1"},{"{#DEVNAME}":"sdbx"},{"{#DEVNAME}":"sdbz"},{"{#DEVNAME}":"sdca"},{"{#DEVNAME}":"sdcb"},{"{#DEVNAME}":"sdcc"},{"{#DEVNAME}":"sdcd"},{"{#DEVNAME}":"sdce"},{},{"{#DEVNAME}":"sdcf"},{},{"{#DEVNAME}":"sdcg"},{"{#DEVNAME}":"sdch"},{"{#DEVNAME}":"sdci"},{},{"{#DEVNAME}":"sdcj"},{"{#DEVNAME}":"sdcj1"},{},{"{#DEVNAME}":"sdck"},{"{#DEVNAME}":"sdcl"},{"{#DEVNAME}":"sdct"},{"{#DEVNAME}":"sdct1"},{"{#DEVNAME}":"sdcy"},{"{#DEVNAME}":"sdcx"},{"{#DEVNAME}":"dm-5"},{"{#DEVNAME}":"sdcz"},{"{#DEVNAME}":"dm-7"},{"{#DEVNAME}":"sdde"},{"{#DEVNAME}":"sddf"},{"{#DEVNAME}":"sddg"},{"{#DEVNAME}":"sddi"},{},{"{#DEVNAME}":"dm-10"},{"{#DEVNAME}":"dm-11"},{"{#DEVNAME}":"sdu"},{"{#DEVNAME}":"sdu1"},{"{#DEVNAME}":"dm-2"},{"{#DEVNAME}":"sdco"},{"{#DEVNAME}":"sdco1"},{"{#DEVNAME}":"sdcq"},{"{#DEVNAME}":"sdcq1"},{"{#DEVNAME}":"sdcp"},{"{#DEVNAME}":"sdcp1"},{"{#DEVNAME}":"dm-3"},{"{#DEVNAME}":"dm-12"},{"{#DEVNAME}":"sdcs"},{"{#DEVNAME}":"sddk"},{"{#DEVNAME}":"dm-13"},{},{"{#DEVNAME}":"sddm"},{},{"{#DEVNAME}":"sddo"},{"{#DEVNAME}":"dm-14"},{"{#DEVNAME}":"sda"},{},{"{#DEVNAME}":"sda1"},{"{#DEVNAME}":"dm-0"},{"{#DEVNAME}":"sdb"},{"{#DEVNAME}":"sdb1"},{"{#DEVNAME}":"sdax"},{"{#DEVNAME}":"sdax1"},{},{"{#DEVNAME}":"sdcm"},{"{#DEVNAME}":"sdcm1"},{"{#DEVNAME}":"dm-1"},{"{#DEVNAME}":"sdh"},{"{#DEVNAME}":"dm-16"},{"{#DEVNAME}":"sdj"},{"{#DEVNAME}":"sdl"},{"{#DEVNAME}":"sdl1"},{"{#DEVNAME}":"sdw"},{},{"{#DEVNAME}":"sdw1"},{"{#DEVNAME}":"dm-20"},{"{#DEVNAME}":"sdaa"},{"{#DEVNAME}":"sdaz"},{},{},{"{#DEVNAME}":"sdbn"},{"{#DEVNAME}":"sdbw"},{"{#DEVNAME}":"dm-8"},{"{#DEVNAME}":"dm-9"},{},{"{#DEVNAME}":"dm-21"},{"{#DEVNAME}":"dm-22"},{"{#DEVNAME}":"sdda"},{},{"{#DEVNAME}":"sddb"},{},{"{#DEVNAME}":"dm-23"},{"{#DEVNAME}":"sddd"},{"{#DEVNAME}":"sddu"},{"{#DEVNAME}":"dm-24"},{"{#DEVNAME}":"sdag"},{"{#DEVNAME}":"sdby"},{"{#DEVNAME}":"dm-25"},{"{#DEVNAME}":"sddv"},{"{#DEVNAME}":"sddw"},{"{#DEVNAME}":"dm-26"},{"{#DEVNAME}":"dm-15"},{"{#DEVNAME}":"sdd"},{"{#DEVNAME}":"sdc"},{"{#DEVNAME}":"sde"},{},{"{#DEVNAME}":"sdg"},{"{#DEVNAME}":"sdt"},{},{"{#DEVNAME}":"sds"},{"{#DEVNAME}":"sdv"},{"{#DEVNAME}":"sdz"},{"{#DEVNAME}":"sdab"},{"{#DEVNAME}":"sdae"},{"{#DEVNAME}":"sdas"},{"{#DEVNAME}":"sdaw"},{"{#DEVNAME}":"sdbk"},{"{#DEVNAME}":"sdbc"},{"{#DEVNAME}":"sdbm"},{"{#DEVNAME}":"sdbs"},{"{#DEVNAME}":"sddh"},{},{},{},{"{#DEVNAME}":"sdbo"},{"{#DEVNAME}":"sdcw"},{"{#DEVNAME}":"sdcn"},{"{#DEVNAME}":"sdcu"},{},{"{#DEVNAME}":"sddp"},{"{#DEVNAME}":"sdbl"},{"{#DEVNAME}":"sddr"},{"{#DEVNAME}":"sddn"},{"{#DEVNAME}":"sddc"},{"{#DEVNAME}":"sdds"},{"{#DEVNAME}":"sddq"},{"{#DEVNAME}":"sddj"},{"{#DEVNAME}":"sddt"},{"{#DEVNAME}":"sdcr"},{},{"{#DEVNAME}":"sddy"},{"{#DEVNAME}":"sddz"},{"{#DEVNAME}":"sdea"},{"{#DEVNAME}":"sdbr"},{"{#DEVNAME}":"sddx"},{"{#DEVNAME}":"sdeb"},{"{#DEVNAME}":"sdec"},{"{#DEVNAME}":"sded"},{"{#DEVNAME}":"sdee"},{},{"{#DEVNAME}":"sdeh"},{"{#DEVNAME}":"sdeg"},{"{#DEVNAME}":"sdej"},{"{#DEVNAME}":"sdel"},{"{#DEVNAME}":"sdek"},{"{#DEVNAME}":"sdei"},{"{#DEVNAME}":"sden"},{},{"{#DEVNAME}":"sdep"},{"{#DEVNAME}":"sder"},{"{#DEVNAME}":"sdeu"},{"{#DEVNAME}":"sdeq"},{"{#DEVNAME}":"sdeo"},{},{"{#DEVNAME}":"sdes"},{"{#DEVNAME}":"sdew"},{"{#DEVNAME}":"sdet"},{},{"{#DEVNAME}":"sdey"},{"{#DEVNAME}":"sdev"},{"{#DEVNAME}":"sdez"},{"{#DEVNAME}":"sdex"},{"{#DEVNAME}":"sdfb"},{"{#DEVNAME}":"sdfa"},{"{#DEVNAME}":"sdfc"},{"{#DEVNAME}":"sdfd"},{"{#DEVNAME}":"sdff"},{"{#DEVNAME}":"sdfe"},{"{#DEVNAME}":"sdfg"},{"{#DEVNAME}":"sdfi"},{"{#DEVNAME}":"sdfh"},{"{#DEVNAME}":"sdfj"},{},{"{#DEVNAME}":"sdfk"}]

      We see multiple empty array members - {} which supposedly cause the rule error state.

      Original /proc/diskstats file for the JSON is here plxpr603Disk.txt (taken correctly, by "cat /proc/diskstats >/tmp/plxpr603Disk.txt ")

      Looks like the JavaScrip supplied with the build-in template is not really suitable for such /proc/diskstats and it must be improved.

      SOLUTION:
      Use this JavaScript for the master item:

      var parsed = value.split("\n").reduce(function(acc, x, i) {
  parts = x.trim().split(/ +/)
  if(typeof parts[13] !== 'undefined') {
    acc["values"][parts[2]] = parts
    acc["lld"].push({"{#DEVNAME}":parts[2]})
  }
  return acc;
}, {"values":{}, "lld": []});

return JSON.stringify(parsed);

      It basically checks for 14th column and if it's missing - it will be skipped for discovery.
      It will resolve both issues:
      1. LLD rule, having the error state.
      2. Unsupported items, having error like this:

      Preprocessing failed for: {"values":{"1119045":["16","sdff","1119045","54","438202004","2474215","4743961","0","1920967473"...
1. Failed: cannot extract value from json by path "$.values['1119045'][13]": no data matches the specified path

      This happens it there are less that 14 columns in a line. In this case, block device name is "1119045" as it's 3rd column, of course because of the bad /proc/diskstats file

      For example a few excerpts from the file: 

       128     144 sdeh 145738886 54 27337661151 988202351 42664270 0 2742765213 37889233 0 263254202 1026083002

2
 128 sdeg 604868962 53 10294296430 168656555 4776995 0 103178483 2637879 0 147627897 171339634
 130      64 sdfi 264688 29 37324848 505059 104327 0 39145592 1196794 0 153369 1701775

      Correct lines here are only 1st and 5th (last).
      Incorrect lines:
      2nd - empty
      3rd - have one strange number
      4th - does NOT have 14 values separated by spaces.

      2nd and 3rd - cause the issue for discovery rule.
      4th - creases unsupported item, which tries to get 14th value, which is missing, so causes JSONPath preprocessing step failure.

      This is real case and there are quite a lot real servers, having such /proc/diskstats file

        1. plxpr603Disk.txt
          40 kB
          Oleksii Zagorskyi

            tuskov Tikhon Uskov (Inactive)
            zalex_ua Oleksii Zagorskyi
            Team INT
            Votes:
            3 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: