Details

      Description

      Zabbix server receives value 0, for all ping request from proxy (passive).

      If same host monitored by Zabbix server (not proxy), then ping requests works fine.

      No error information and item state is ok.

        Activity

        Hide
        richlv added a comment -

        any error messages in the proxy log ?
        what's the exact item key ?

        Show
        richlv added a comment - any error messages in the proxy log ? what's the exact item key ?
        Hide
        Maris Danne added a comment - - edited

        No errors in proxy log and no erros in server log. For every ping check with proxy Zabbix server is receiving value 0.
        Tried to use this Item key's:
        icmpping[{HOST.IP},5,,64,]
        and
        icmpping[,5,,64,]

        Same problem (receiving 0):
        icmppingloss[,5,,64]
        icmppingsec[,5,,64]

        Show
        Maris Danne added a comment - - edited No errors in proxy log and no erros in server log. For every ping check with proxy Zabbix server is receiving value 0. Tried to use this Item key's: icmpping[{HOST.IP},5,,64,] and icmpping[,5,,64,] Same problem (receiving 0): icmppingloss[,5,,64] icmppingsec[,5,,64]
        Hide
        Aleksandrs Saveljevs added a comment - - edited

        The value of "0" for icmpping[] and icmppingsec[] means that the host is inaccessible. For icmppingloss[], the value of 100 means the same.

        Since you get 0 for all items (including icmppingloss[]), this is very strange. Could you please confirm that the value for icmppingloss[] is also 0? Are you sure the host is pingable from the proxy host? Which fping versions are used on the server and the proxy?

        Show
        Aleksandrs Saveljevs added a comment - - edited The value of "0" for icmpping[] and icmppingsec[] means that the host is inaccessible. For icmppingloss[], the value of 100 means the same. Since you get 0 for all items (including icmppingloss[]), this is very strange. Could you please confirm that the value for icmppingloss[] is also 0? Are you sure the host is pingable from the proxy host? Which fping versions are used on the server and the proxy?
        Hide
        Maris Danne added a comment -

        Server has fping: Version 2.4b2
        Proxy has fping: Version 3.10

        It can be an issue?

        Show
        Maris Danne added a comment - Server has fping: Version 2.4b2 Proxy has fping: Version 3.10 It can be an issue?
        Hide
        Aleksandrs Saveljevs added a comment -

        Zabbix server and proxy work well for me with fping 3.10.

        Show
        Aleksandrs Saveljevs added a comment - Zabbix server and proxy work well for me with fping 3.10.
        Hide
        richlv added a comment - - edited

        is zabbix source ip parameter used ?

        Show
        richlv added a comment - - edited is zabbix source ip parameter used ?
        Hide
        Raimonds Treimanis added a comment -

        Problem solved - Selinux with default settings on CentOS is not allowing fping. Whats worst thing - its supressing error messages also by default (dontaudit rule), so we get weird situation without any error messages in either zabbix or selinux logs
        1. Why isnt zabbix complaining that its not allowed to execute fping, but merely returning zeros?
        2. Would be nice to somehow correct zabbix 1.2.0 selinux module installed with CentOS/RHEL (selinux-policy-targeted.noarch 3.7.19-231.el6_5.3)

        Show
        Raimonds Treimanis added a comment - Problem solved - Selinux with default settings on CentOS is not allowing fping. Whats worst thing - its supressing error messages also by default (dontaudit rule), so we get weird situation without any error messages in either zabbix or selinux logs 1. Why isnt zabbix complaining that its not allowed to execute fping, but merely returning zeros? 2. Would be nice to somehow correct zabbix 1.2.0 selinux module installed with CentOS/RHEL (selinux-policy-targeted.noarch 3.7.19-231.el6_5.3)
        Hide
        Aleksandrs Saveljevs added a comment -

        Regarding (1), could you please show some DebugLevel=4 from the pinger? Note that since ZBXNEXT-101, which was implemented for 2.4.0, it is possible to increase log level for pinger processes only at runtime (no restart is necessary):

        $ zabbix_proxy -R log_level_increase=pinger
        
        Show
        Aleksandrs Saveljevs added a comment - Regarding (1), could you please show some DebugLevel=4 from the pinger? Note that since ZBXNEXT-101 , which was implemented for 2.4.0, it is possible to increase log level for pinger processes only at runtime (no restart is necessary): $ zabbix_proxy -R log_level_increase=pinger
        Hide
        Raimonds Treimanis added a comment - - edited

        19079:20140930:115235.959 In DCconfig_get_poller_nextcheck() poller_type:3
        19079:20140930:115235.959 End of DCconfig_get_poller_nextcheck():1412067158
        19079:20140930:115235.959 icmp pinger #1 [got 0 values in 0.000228 sec, idle 3 sec]
        19081:20140930:115238.944 icmp pinger #3 [getting values]
        19081:20140930:115238.944 In get_pinger_hosts()
        19081:20140930:115238.944 In DCconfig_get_poller_items() poller_type:3
        19081:20140930:115238.944 End of DCconfig_get_poller_items():4
        19081:20140930:115238.944 In substitute_key_macros() data:'icmpping[10.10.10.10,5,,64,]'
        19081:20140930:115238.944 End of substitute_key_macros():SUCCEED data:'icmpping[10.10.10.10,5,,64,]'
        19081:20140930:115238.944 In add_icmpping_item() addr:'10.10.10.10' count:5 interval:0 size:64 timeout:0
        19081:20140930:115238.944 End of add_icmpping_item()
        19081:20140930:115238.944 In substitute_key_macros() data:'icmppingsec[,5,,64]'
        19081:20140930:115238.944 End of substitute_key_macros():SUCCEED data:'icmppingsec[,5,,64]'
        19081:20140930:115238.944 In add_icmpping_item() addr:'10.10.10.10' count:5 interval:0 size:64 timeout:0
        19081:20140930:115238.944 End of add_icmpping_item()
        19081:20140930:115238.944 In substitute_key_macros() data:'icmpping[

        {HOST.IP},5,,64,]'
        19081:20140930:115238.944 In substitute_simple_macros() data:'{HOST.IP}

        '
        19081:20140930:115238.944 End substitute_simple_macros() data:'10.10.10.10'
        19081:20140930:115238.944 End of substitute_key_macros():SUCCEED data:'icmpping[10.10.10.10,5,,64,]'
        19081:20140930:115238.944 In add_icmpping_item() addr:'10.10.10.10' count:5 interval:0 size:64 timeout:0
        19081:20140930:115238.944 End of add_icmpping_item()
        19081:20140930:115238.944 In substitute_key_macros() data:'icmppingloss[,5,,64]'
        19081:20140930:115238.944 End of substitute_key_macros():SUCCEED data:'icmppingloss[,5,,64]'
        19081:20140930:115238.944 In add_icmpping_item() addr:'10.10.10.10' count:5 interval:0 size:64 timeout:0
        19081:20140930:115238.944 End of add_icmpping_item()
        19081:20140930:115238.944 End of get_pinger_hosts():4
        19081:20140930:115238.944 In process_pinger_hosts()
        19081:20140930:115238.945 In add_pinger_host() addr:'10.10.10.10'
        19081:20140930:115238.945 End of add_pinger_host()
        19081:20140930:115238.945 In add_pinger_host() addr:'10.10.10.10'
        19081:20140930:115238.945 In add_pinger_host() addr:'10.10.10.10'
        19081:20140930:115238.945 In add_pinger_host() addr:'10.10.10.10'
        19081:20140930:115238.945 icmp pinger #3 [pinging hosts]
        19081:20140930:115238.945 In do_ping() hosts_count:1
        19081:20140930:115238.945 In process_ping() hosts_count:1
        19081:20140930:115238.945 /tmp/zabbix_proxy_19081.pinger
        19081:20140930:115238.945 10.10.10.10
        19081:20140930:115238.945 /usr/sbin/fping -C5 -b64 2>&1 </tmp/zabbix_proxy_19081.pinger;/usr/sbin/fping6 -C5 -b64 2>&1 </tmp/zabbix_proxy_19081.pinger;
        19081:20140930:115238.951 End of process_ping()
        19081:20140930:115238.951 End of do_ping():SUCCEED
        19081:20140930:115238.951 In process_values()
        19081:20140930:115238.951 Host [10.10.10.10] cnt=0 rcv=0 min/max/avg=0.000000/0.000000/0.000000
        19081:20140930:115238.952 In process_value()
        19081:20140930:115238.952 End of process_value()

        And at the same time in audit.log
        type=AVC msg=audit(1412067285.990:993): avc: denied

        { read }

        for pid=30018 comm="fping" path="/tmp/zabbix_proxy_19079.pinger" dev=dm-0 ino=1840307 scontext=unconfined_u:system_r:ping_t:s0 tcontext=unconfined_u:object_r:initrc_tmp_t:s0 tclass=file

        Show
        Raimonds Treimanis added a comment - - edited 19079:20140930:115235.959 In DCconfig_get_poller_nextcheck() poller_type:3 19079:20140930:115235.959 End of DCconfig_get_poller_nextcheck():1412067158 19079:20140930:115235.959 icmp pinger #1 [got 0 values in 0.000228 sec, idle 3 sec] 19081:20140930:115238.944 icmp pinger #3 [getting values] 19081:20140930:115238.944 In get_pinger_hosts() 19081:20140930:115238.944 In DCconfig_get_poller_items() poller_type:3 19081:20140930:115238.944 End of DCconfig_get_poller_items():4 19081:20140930:115238.944 In substitute_key_macros() data:'icmpping [10.10.10.10,5,,64,] ' 19081:20140930:115238.944 End of substitute_key_macros():SUCCEED data:'icmpping [10.10.10.10,5,,64,] ' 19081:20140930:115238.944 In add_icmpping_item() addr:'10.10.10.10' count:5 interval:0 size:64 timeout:0 19081:20140930:115238.944 End of add_icmpping_item() 19081:20140930:115238.944 In substitute_key_macros() data:'icmppingsec [,5,,64] ' 19081:20140930:115238.944 End of substitute_key_macros():SUCCEED data:'icmppingsec [,5,,64] ' 19081:20140930:115238.944 In add_icmpping_item() addr:'10.10.10.10' count:5 interval:0 size:64 timeout:0 19081:20140930:115238.944 End of add_icmpping_item() 19081:20140930:115238.944 In substitute_key_macros() data:'icmpping[ {HOST.IP},5,,64,]' 19081:20140930:115238.944 In substitute_simple_macros() data:'{HOST.IP} ' 19081:20140930:115238.944 End substitute_simple_macros() data:'10.10.10.10' 19081:20140930:115238.944 End of substitute_key_macros():SUCCEED data:'icmpping [10.10.10.10,5,,64,] ' 19081:20140930:115238.944 In add_icmpping_item() addr:'10.10.10.10' count:5 interval:0 size:64 timeout:0 19081:20140930:115238.944 End of add_icmpping_item() 19081:20140930:115238.944 In substitute_key_macros() data:'icmppingloss [,5,,64] ' 19081:20140930:115238.944 End of substitute_key_macros():SUCCEED data:'icmppingloss [,5,,64] ' 19081:20140930:115238.944 In add_icmpping_item() addr:'10.10.10.10' count:5 interval:0 size:64 timeout:0 19081:20140930:115238.944 End of add_icmpping_item() 19081:20140930:115238.944 End of get_pinger_hosts():4 19081:20140930:115238.944 In process_pinger_hosts() 19081:20140930:115238.945 In add_pinger_host() addr:'10.10.10.10' 19081:20140930:115238.945 End of add_pinger_host() 19081:20140930:115238.945 In add_pinger_host() addr:'10.10.10.10' 19081:20140930:115238.945 In add_pinger_host() addr:'10.10.10.10' 19081:20140930:115238.945 In add_pinger_host() addr:'10.10.10.10' 19081:20140930:115238.945 icmp pinger #3 [pinging hosts] 19081:20140930:115238.945 In do_ping() hosts_count:1 19081:20140930:115238.945 In process_ping() hosts_count:1 19081:20140930:115238.945 /tmp/zabbix_proxy_19081.pinger 19081:20140930:115238.945 10.10.10.10 19081:20140930:115238.945 /usr/sbin/fping -C5 -b64 2>&1 </tmp/zabbix_proxy_19081.pinger;/usr/sbin/fping6 -C5 -b64 2>&1 </tmp/zabbix_proxy_19081.pinger; 19081:20140930:115238.951 End of process_ping() 19081:20140930:115238.951 End of do_ping():SUCCEED 19081:20140930:115238.951 In process_values() 19081:20140930:115238.951 Host [10.10.10.10] cnt=0 rcv=0 min/max/avg=0.000000/0.000000/0.000000 19081:20140930:115238.952 In process_value() 19081:20140930:115238.952 End of process_value() And at the same time in audit.log type=AVC msg=audit(1412067285.990:993): avc: denied { read } for pid=30018 comm="fping" path="/tmp/zabbix_proxy_19079.pinger" dev=dm-0 ino=1840307 scontext=unconfined_u:system_r:ping_t:s0 tcontext=unconfined_u:object_r:initrc_tmp_t:s0 tclass=file
        Hide
        Volker Fröhlich added a comment -

        The policy assumes the pinger files to be in /var/lib/zabbixsrv/tmp.

        https://bugzilla.redhat.com/show_bug.cgi?id=890554

        Show
        Volker Fröhlich added a comment - The policy assumes the pinger files to be in /var/lib/zabbixsrv/tmp. https://bugzilla.redhat.com/show_bug.cgi?id=890554
        Hide
        Aleksandrs Saveljevs added a comment -

        It seems that fping is executed, but it does not output anything. There are valid cases where fping may not output anything, like this conditional in src/libs/zbxicmpping/icmpping.c:

        if (NULL == (f = popen(tmp, "r")))
        {
        	zbx_snprintf(error, max_error_len, "%s: %s", tmp, zbx_strerror(errno));
        
        	unlink(filename);
        
        	return ret;
        }
        
        if (NULL == fgets(tmp, sizeof(tmp), f))
        {
        	ret = SUCCEED; /* fping does not output anything for DNS names that fail to resolve */
        }
        else
        ...
        

        So we cannot distinguish between DNS resolution failure and SELinux blocking the pings based on output alone.

        One thing we could do is make icmppingloss[] return 100% in such cases, instead of 0%. That would be more correct.

        Show
        Aleksandrs Saveljevs added a comment - It seems that fping is executed, but it does not output anything. There are valid cases where fping may not output anything, like this conditional in src/libs/zbxicmpping/icmpping.c: if (NULL == (f = popen(tmp, "r" ))) { zbx_snprintf(error, max_error_len, "%s: %s" , tmp, zbx_strerror(errno)); unlink(filename); return ret; } if (NULL == fgets(tmp, sizeof(tmp), f)) { ret = SUCCEED; /* fping does not output anything for DNS names that fail to resolve */ } else ... So we cannot distinguish between DNS resolution failure and SELinux blocking the pings based on output alone. One thing we could do is make icmppingloss[] return 100% in such cases, instead of 0%. That would be more correct.
        Hide
        Aleksandrs Saveljevs added a comment -

        Another idea would be to make the items go unsupported if we could not send pings to the host (i.e., in case 0 == hosts[h].cnt in process_values()).

        Show
        Aleksandrs Saveljevs added a comment - Another idea would be to make the items go unsupported if we could not send pings to the host (i.e., in case 0 == hosts [h] .cnt in process_values()).
        Hide
        Igors Homjakovs (Inactive) added a comment -

        Fixed in svn://svn.zabbix.com/branches/dev/ZBX-8828

        Show
        Igors Homjakovs (Inactive) added a comment - Fixed in svn://svn.zabbix.com/branches/dev/ZBX-8828
        Hide
        Aleksandrs Saveljevs added a comment - - edited

        Alternative fix which detects the inability to send ICMP ping packets to a host based on "0 == hosts[h].cnt" is available in svn://svn.zabbix.com/branches/dev/ZBX-8828-alt .

        If there is just one host to ping and it fails due to DNS resolution, the error message will be like this:

        fping failed: non-existing.dns.name address not found
        

        Here, the last line of fping output is used for the error message, following the principles of ZBX-1557.

        If some hosts are pingable and some are not, the error message for the latter will be like this:

        Cannot send ICMP ping packets to this host.
        

        If fping does not output anything at all, as is the case with SELinux originally reported here, the error message will be:

        fping failed: no output
        

        Alexander Vladishev I like it! I made some changes in r51073. Please review.

        TESTED

        Aleksandrs Saveljevs Looks good, thank you! CLOSED.

        Show
        Aleksandrs Saveljevs added a comment - - edited Alternative fix which detects the inability to send ICMP ping packets to a host based on "0 == hosts[h].cnt" is available in svn://svn.zabbix.com/branches/dev/ZBX-8828-alt . If there is just one host to ping and it fails due to DNS resolution, the error message will be like this: fping failed: non-existing.dns.name address not found Here, the last line of fping output is used for the error message, following the principles of ZBX-1557 . If some hosts are pingable and some are not, the error message for the latter will be like this: Cannot send ICMP ping packets to this host. If fping does not output anything at all, as is the case with SELinux originally reported here, the error message will be: fping failed: no output Alexander Vladishev I like it! I made some changes in r51073. Please review. TESTED Aleksandrs Saveljevs Looks good, thank you! CLOSED.
        Hide
        Aleksandrs Saveljevs added a comment -

        The alternative solution was chosen, because it is simpler and easier to merge from 2.0 all the way to trunk. However, the solution implemented by Igors Homjakovs is also good and might be implemented in future in trunk only.

        Fixed in pre-2.0.15 r51431, pre-2.2.9 r51432, pre-2.4.4 r51433, and pre-2.5.0 (trunk) r51435.

        Show
        Aleksandrs Saveljevs added a comment - The alternative solution was chosen, because it is simpler and easier to merge from 2.0 all the way to trunk. However, the solution implemented by Igors Homjakovs is also good and might be implemented in future in trunk only. Fixed in pre-2.0.15 r51431, pre-2.2.9 r51432, pre-2.4.4 r51433, and pre-2.5.0 (trunk) r51435.
        Show
        Aleksandrs Saveljevs added a comment - - edited (1) Documented at the following locations: https://www.zabbix.com/documentation/2.0/manual/installation/upgrade_notes_2015 https://www.zabbix.com/documentation/2.2/manual/installation/upgrade_notes_229 https://www.zabbix.com/documentation/2.4/manual/installation/upgrade_notes_244 Alexander Vladishev CLOSED

          People

          • Assignee:
            Unassigned
            Reporter:
            Maris Danne
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: