Uploaded image for project: 'ZABBIX BUGS AND ISSUES'
  1. ZABBIX BUGS AND ISSUES
  2. ZBX-25395

Zabbix Agent 2 - broken smart.disk.discovery after agent upgrade

XMLWordPrintable

    • Icon: Problem report Problem report
    • Resolution: Fixed
    • Icon: Trivial Trivial
    • 6.0.36rc1, 7.0.6rc1, 7.2.0beta1
    • 6.0.33
    • Agent2 plugin (N)
    • None
    • smartctl 7.1 2019-12-30 r5022 [x86_64-linux-4.18.0-477.10.1.el8_8.x86_64]
    • Sprint candidates

      SMART monitoring appears broken after agent upgrade. Please find logs below.

      6.0.32

      [user@host2 ~]$ sudo -u zabbix -s
      Creating home directory for zabbix.
      bash-4.4$ sudo -n /usr/sbin/smartctl --scan -j
      {
        "json_format_version": [
          1,
          0
        ],
        "smartctl": {
          "version": [
            7,
            1
          ],
          "svn_revision": "5022",
          "platform_info": "x86_64-linux-4.18.0-477.10.1.el8_8.x86_64",
          "build_info": "(local build)",
          "argv": [
            "smartctl",
            "--scan",
            "-j"
          ],
          "exit_status": 0
        },
        "devices": [
          {
            "name": "/dev/sdb",
            "info_name": "/dev/sdb",
            "type": "scsi",
            "protocol": "SCSI"
          },
          {
            "name": "/dev/sdc",
            "info_name": "/dev/sdc",
            "type": "scsi",
            "protocol": "SCSI"
          }
        ]
      }
      
      [user@host2 ~]$ sudo -u zabbix zabbix_agent2 -t smart.disk.discovery
      Creating home directory for zabbix.
      smart.disk.discovery                          [m|ZBX_NOTSUPPORTED] [Timeout occurred while gathering data.]
      

      6.0.7

      [user@host1 ~]$ sudo -u zabbix -s
      Creating home directory for zabbix.
      bash-4.4$ sudo -n /usr/sbin/smartctl --scan -j
      {
        "json_format_version": [
          1,
          0
        ],
        "smartctl": {
          "version": [
            7,
            1
          ],
          "svn_revision": "5049",
          "platform_info": "x86_64-linux-4.18.0-477.10.1.el8_8.x86_64",
          "build_info": "(local build)",
          "argv": [
            "smartctl",
            "--scan",
            "-j"
          ],
          "exit_status": 0
        },
        "devices": [
          {
            "name": "/dev/sdb",
            "info_name": "/dev/sdb",
            "type": "scsi",
            "protocol": "SCSI"
          },
          {
            "name": "/dev/sdc",
            "info_name": "/dev/sdc",
            "type": "scsi",
            "protocol": "SCSI"
          }
        ]
      }
      
      [user@host1 ~]$ sudo -u zabbix zabbix_agent2 -t smart.disk.discovery
      Creating home directory for zabbix.
      smart.disk.discovery                          [s|[{"{#NAME}":"sdb cciss,0","{#DISKTYPE}":"hdd","{#MODEL}":"HP EG000300JWEBF","{#SN}":"W0K2XX25","{#PATH}":"/dev/sdb","{#RAIDTYPE}":"cciss,0","{#ATTRIBUTES}":""},{"{#NAME}":"sdb cciss,1","{#DISKTYPE}":"hdd","{#MODEL}":"HP EG000300JWEBF","{#SN}":"W0K2XZ2W","{#PATH}":"/dev/sdb","{#RAIDTYPE}":"cciss,1","{#ATTRIBUTES}":""},{"{#NAME}":"sdb cciss,2","{#DISKTYPE}":"hdd","{#MODEL}":"HP EG000300JWEBF","{#SN}":"W0K2Y1HQ","{#PATH}":"/dev/sdb","{#RAIDTYPE}":"cciss,2","{#ATTRIBUTES}":""},{"{#NAME}":"sdb cciss,3","{#DISKTYPE}":"hdd","{#MODEL}":"HP EG000300JWEBF","{#SN}":"W0K2XQDV","{#PATH}":"/dev/sdb","{#RAIDTYPE}":"cciss,3","{#ATTRIBUTES}":""},{"{#NAME}":"sdb cciss,4","{#DISKTYPE}":"ssd","{#MODEL}":"VK000480GWSXF","{#SN}":"18141F94EF30","{#PATH}":"/dev/sdb","{#RAIDTYPE}":"cciss,4","{#ATTRIBUTES}":"Raw_Read_Error_Rate Reallocated_Sector_Ct Power_On_Hours Program_Fail_Count_Chip Unused_Rsvd_Blk_Cnt_Tot Temperature_Celsius Reallocated_Event_Count"},{"{#NAME}":"sdb cciss,5","{#DISKTYPE}":"ssd","{#MODEL}":"VK000480GWSXF","{#SN}":"18141F94EF2C","{#PATH}":"/dev/sdb","{#RAIDTYPE}":"cciss,5","{#ATTRIBUTES}":"Raw_Read_Error_Rate Reallocated_Sector_Ct Power_On_Hours Program_Fail_Count_Chip Unused_Rsvd_Blk_Cnt_Tot Temperature_Celsius Reallocated_Event_Count"}]]
      

      Debug level 5:

      2024/08/23 06:43:23.870718 executing direct exporter task for key 'smart.disk.discovery'
      2024/08/23 06:43:23.870739 [Smart] executing smartctl command: sudo -n /usr/sbin/smartctl -j -V
      2024/08/23 06:43:24.037294 [Smart] executing smartctl command: sudo -n /usr/sbin/smartctl --scan -j
      2024/08/23 06:43:24.074984 [Smart] executing smartctl command: sudo -n /usr/sbin/smartctl --scan -d sat -j
      2024/08/23 06:43:24.109502 [Smart] executing smartctl command: sudo -n /usr/sbin/smartctl -a /dev/sda -d 3ware,0 -j
      2024/08/23 06:43:24.109526 [Smart] executing smartctl command: sudo -n /usr/sbin/smartctl -a /dev/sda -d sat -j
      2024/08/23 06:43:24.109544 [Smart] executing smartctl command: sudo -n /usr/sbin/smartctl -a /dev/sda -d cciss,0 -j
      2024/08/23 06:43:24.109578 [Smart] executing smartctl command: sudo -n /usr/sbin/smartctl -a /dev/sda -d scsi -j
      2024/08/23 06:43:24.109669 [Smart] executing smartctl command: sudo -n /usr/sbin/smartctl -a /dev/sdb -d 3ware,0 -j
      2024/08/23 06:43:24.109746 [Smart] executing smartctl command: sudo -n /usr/sbin/smartctl -a /dev/sda -d areca,1 -j
      2024/08/23 06:43:24.109829 [Smart] executing smartctl command: sudo -n /usr/sbin/smartctl -a /dev/sdc -d 3ware,0 -j
      2024/08/23 06:43:24.109971 [Smart] executing smartctl command: sudo -n /usr/sbin/smartctl -a /dev/sdc -d sat -j
      2024/08/23 06:43:24.110089 [Smart] executing smartctl command: sudo -n /usr/sbin/smartctl -a /dev/sdb -d cciss,0 -j
      2024/08/23 06:43:24.110721 [Smart] executing smartctl command: sudo -n /usr/sbin/smartctl -a /dev/sdb -d sat -j
      2024/08/23 06:43:24.110749 [Smart] executing smartctl command: sudo -n /usr/sbin/smartctl -a /dev/sdc -d scsi -j
      2024/08/23 06:43:24.112799 [Smart] executing smartctl command: sudo -n /usr/sbin/smartctl -a /dev/sdc -d areca,1 -j
      2024/08/23 06:43:24.112908 [Smart] executing smartctl command: sudo -n /usr/sbin/smartctl -a /dev/sdb -d areca,1 -j
      2024/08/23 06:43:24.112969 [Smart] executing smartctl command: sudo -n /usr/sbin/smartctl -a /dev/sdc -d cciss,0 -j
      2024/08/23 06:43:24.113051 [Smart] executing smartctl command: sudo -n /usr/sbin/smartctl -a /dev/sdb -d scsi -j
      2024/08/23 06:43:24.216411 [Smart] failed to get device "/dev/sdb" info by type "3ware": Smartctl returned error: unknown error from smartctl.
      2024/08/23 06:43:24.216447 [Smart] failed to get device "/dev/sda" info by type "sat": Smartctl returned error: unknown error from smartctl.
      2024/08/23 06:43:24.217154 [Smart] failed to get device "/dev/sdc" info by type "3ware": Smartctl returned error: unknown error from smartctl.
      2024/08/23 06:43:24.217810 [Smart] failed to get device "/dev/sdc" info by type "areca": Smartctl returned error: Smartctl open device: /dev/sdc [areca_disk#01_enc#01] failed: Input/output error.
      2024/08/23 06:43:24.218737 [Smart] failed to get device "/dev/sdb" info by type "sat": Smartctl returned error: unknown error from smartctl.
      2024/08/23 06:43:24.231012 [Smart] failed to get device "/dev/sda" info by type "cciss": Smartctl returned error: Smartctl open device: /dev/sda [cciss_disk_00] [SCSI/SAT] failed: Read-only file system.
      2024/08/23 06:43:24.231340 [Smart] failed to get device "/dev/sdb" info by type "areca": Smartctl returned error: Smartctl open device: /dev/sdb [areca_disk#01_enc#01] failed: Input/output error.
      2024/08/23 06:43:24.238622 [Smart] failed to get device "/dev/sda" info by type "3ware": Smartctl returned error: unknown error from smartctl.
      2024/08/23 06:43:24.247501 [Smart] failed to get device "/dev/sda" info by type "areca": Smartctl returned error: Smartctl open device: /dev/sda [areca_disk#01_enc#01] failed: Input/output error.
      2024/08/23 06:43:24.247778 [Smart] failed to get device "/dev/sdc" info by type "sat": Smartctl returned error: unknown error from smartctl.
      2024/08/23 06:43:24.254113 [Smart] failed to get device "/dev/sda" info by type "scsi": Smartctl returned no smart status.
      2024/08/23 06:43:25.500603 [Smart] executing smartctl command: sudo -n /usr/sbin/smartctl -a /dev/sdb -d cciss,0 -j
      2024/08/23 06:43:25.532901 [Smart] executing smartctl command: sudo -n /usr/sbin/smartctl -a /dev/sdc -d cciss,0 -j
      2024/08/23 06:43:26.203771 [Smart] executing smartctl command: sudo -n /usr/sbin/smartctl -a /dev/sdb -d cciss,0 -j
      

      From changelog

      Between 6.07 and 6.0.32, there were three changes directly related to smartctl (highlighted in changelog). From most recent:
      ZBXNEXT-9081 added Small Computer System Interface (SCSI) device type support to Zabbix agent 2 Smart plugin
      ZBX-23858 fixed injection vulnerability in Zabbix agent 2 smartctl plugin
      ZBX-21665 fixed SMART plugin infinite loop for Zabbix agent 2

      Downgrading agent to 6.0.13 works:

      [user@host1 ~]$ zabbix_get -s 127.0.0.1 -k smart.disk.discovery
      [{"{#NAME}":"sda cciss,0","{#DISKTYPE}":"hdd","{#MODEL}":"HP EG000300JWEBF","{#SN}":"W0K2XX25","{#PATH}":"/dev/sda","{#RAIDTYPE}":"cciss,0","{#ATTRIBUTES}":""},{"{#NAME}":"sda cciss,1","{#DISKTYPE}":"hdd","{#MODEL}":"HP EG000300JWEBF","{#SN}":"W0K2XZ2W","{#PATH}":"/dev/sda","{#RAIDTYPE}":"cciss,1","{#ATTRIBUTES}":""},{"{#NAME}":"sda cciss,2","{#DISKTYPE}":"hdd","{#MODEL}":"HP EG000300JWEBF","{#SN}":"W0K2Y1HQ","{#PATH}":"/dev/sda","{#RAIDTYPE}":"cciss,2","{#ATTRIBUTES}":""},{"{#NAME}":"sda cciss,3","{#DISKTYPE}":"hdd","{#MODEL}":"HP EG000300JWEBF","{#SN}":"W0K2XQDV","{#PATH}":"/dev/sda","{#RAIDTYPE}":"cciss,3","{#ATTRIBUTES}":""},{"{#NAME}":"sda cciss,4","{#DISKTYPE}":"ssd","{#MODEL}":"VK000480GWSXF","{#SN}":"18141F94EF30","{#PATH}":"/dev/sda","{#RAIDTYPE}":"cciss,4","{#ATTRIBUTES}":"Raw_Read_Error_Rate Reallocated_Sector_Ct Power_On_Hours Unknown_Intel_Attribute Power_Loss_Cap_Test Unused_Rsvd_Blk_Cnt_Tot Temperature_Internal Reallocated_Event_Count"},{"{#NAME}":"sda cciss,5","{#DISKTYPE}":"ssd","{#MODEL}":"VK000480GWSXF","{#SN}":"18141F94EF2C","{#PATH}":"/dev/sda","{#RAIDTYPE}":"cciss,5","{#ATTRIBUTES}":"Raw_Read_Error_Rate Reallocated_Sector_Ct Power_On_Hours Unknown_Intel_Attribute Power_Loss_Cap_Test Unused_Rsvd_Blk_Cnt_Tot Temperature_Internal Reallocated_Event_Count"}]

      Log for 6.0.13:
      smart.discovery.wtt8lcmshv001ls.6.0.13.txt

      Agent key appears broken in 6.0.24:
      smart.discovery.wtt8lcmshv001ls.6.0.24-1.txt

       

            sjurgensons Stanislavs Jurgensons
            mkudlacz Michal Kudlacz
            Team INT
            Votes:
            1 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated:
              Resolved: