[ZBX-8828] Ping on proxy not working Created: 2014 Sep 26  Updated: 2017 May 30  Resolved: 2015 Jan 08

Status: Closed
Project: ZABBIX BUGS AND ISSUES
Component/s: Proxy (P), Server (S)
Affects Version/s: 2.4.0
Fix Version/s: 2.0.15rc1, 2.2.9rc1, 2.4.4rc1, 2.5.0

Type: Incident report Priority: Major
Reporter: Maris Danne Assignee: Unassigned
Resolution: Fixed Votes: 0
Labels: icmpping, selinux
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Zabbix server receives value 0, for all ping request from proxy (passive).

If same host monitored by Zabbix server (not proxy), then ping requests works fine.

No error information and item state is ok.



 Comments   
Comment by richlv [ 2014 Sep 28 ]

any error messages in the proxy log ?
what's the exact item key ?

Comment by Maris Danne [ 2014 Sep 29 ]

No errors in proxy log and no erros in server log. For every ping check with proxy Zabbix server is receiving value 0.
Tried to use this Item key's:
icmpping[{HOST.IP},5,,64,]
and
icmpping[,5,,64,]

Same problem (receiving 0):
icmppingloss[,5,,64]
icmppingsec[,5,,64]

Comment by Aleksandrs Saveljevs [ 2014 Sep 29 ]

The value of "0" for icmpping[] and icmppingsec[] means that the host is inaccessible. For icmppingloss[], the value of 100 means the same.

Since you get 0 for all items (including icmppingloss[]), this is very strange. Could you please confirm that the value for icmppingloss[] is also 0? Are you sure the host is pingable from the proxy host? Which fping versions are used on the server and the proxy?

Comment by Maris Danne [ 2014 Sep 29 ]

Server has fping: Version 2.4b2
Proxy has fping: Version 3.10

It can be an issue?

Comment by Aleksandrs Saveljevs [ 2014 Sep 29 ]

Zabbix server and proxy work well for me with fping 3.10.

Comment by richlv [ 2014 Sep 29 ]

is zabbix source ip parameter used ?

Comment by Raimonds Treimanis [ 2014 Sep 30 ]

Problem solved - Selinux with default settings on CentOS is not allowing fping. Whats worst thing - its supressing error messages also by default (dontaudit rule), so we get weird situation without any error messages in either zabbix or selinux logs
1. Why isnt zabbix complaining that its not allowed to execute fping, but merely returning zeros?
2. Would be nice to somehow correct zabbix 1.2.0 selinux module installed with CentOS/RHEL (selinux-policy-targeted.noarch 3.7.19-231.el6_5.3)

Comment by Aleksandrs Saveljevs [ 2014 Sep 30 ]

Regarding (1), could you please show some DebugLevel=4 from the pinger? Note that since ZBXNEXT-101, which was implemented for 2.4.0, it is possible to increase log level for pinger processes only at runtime (no restart is necessary):

$ zabbix_proxy -R log_level_increase=pinger
Comment by Raimonds Treimanis [ 2014 Sep 30 ]

19079:20140930:115235.959 In DCconfig_get_poller_nextcheck() poller_type:3
19079:20140930:115235.959 End of DCconfig_get_poller_nextcheck():1412067158
19079:20140930:115235.959 icmp pinger #1 [got 0 values in 0.000228 sec, idle 3 sec]
19081:20140930:115238.944 icmp pinger #3 [getting values]
19081:20140930:115238.944 In get_pinger_hosts()
19081:20140930:115238.944 In DCconfig_get_poller_items() poller_type:3
19081:20140930:115238.944 End of DCconfig_get_poller_items():4
19081:20140930:115238.944 In substitute_key_macros() data:'icmpping[10.10.10.10,5,,64,]'
19081:20140930:115238.944 End of substitute_key_macros():SUCCEED data:'icmpping[10.10.10.10,5,,64,]'
19081:20140930:115238.944 In add_icmpping_item() addr:'10.10.10.10' count:5 interval:0 size:64 timeout:0
19081:20140930:115238.944 End of add_icmpping_item()
19081:20140930:115238.944 In substitute_key_macros() data:'icmppingsec[,5,,64]'
19081:20140930:115238.944 End of substitute_key_macros():SUCCEED data:'icmppingsec[,5,,64]'
19081:20140930:115238.944 In add_icmpping_item() addr:'10.10.10.10' count:5 interval:0 size:64 timeout:0
19081:20140930:115238.944 End of add_icmpping_item()
19081:20140930:115238.944 In substitute_key_macros() data:'icmpping[

{HOST.IP},5,,64,]'
19081:20140930:115238.944 In substitute_simple_macros() data:'{HOST.IP}

'
19081:20140930:115238.944 End substitute_simple_macros() data:'10.10.10.10'
19081:20140930:115238.944 End of substitute_key_macros():SUCCEED data:'icmpping[10.10.10.10,5,,64,]'
19081:20140930:115238.944 In add_icmpping_item() addr:'10.10.10.10' count:5 interval:0 size:64 timeout:0
19081:20140930:115238.944 End of add_icmpping_item()
19081:20140930:115238.944 In substitute_key_macros() data:'icmppingloss[,5,,64]'
19081:20140930:115238.944 End of substitute_key_macros():SUCCEED data:'icmppingloss[,5,,64]'
19081:20140930:115238.944 In add_icmpping_item() addr:'10.10.10.10' count:5 interval:0 size:64 timeout:0
19081:20140930:115238.944 End of add_icmpping_item()
19081:20140930:115238.944 End of get_pinger_hosts():4
19081:20140930:115238.944 In process_pinger_hosts()
19081:20140930:115238.945 In add_pinger_host() addr:'10.10.10.10'
19081:20140930:115238.945 End of add_pinger_host()
19081:20140930:115238.945 In add_pinger_host() addr:'10.10.10.10'
19081:20140930:115238.945 In add_pinger_host() addr:'10.10.10.10'
19081:20140930:115238.945 In add_pinger_host() addr:'10.10.10.10'
19081:20140930:115238.945 icmp pinger #3 [pinging hosts]
19081:20140930:115238.945 In do_ping() hosts_count:1
19081:20140930:115238.945 In process_ping() hosts_count:1
19081:20140930:115238.945 /tmp/zabbix_proxy_19081.pinger
19081:20140930:115238.945 10.10.10.10
19081:20140930:115238.945 /usr/sbin/fping -C5 -b64 2>&1 </tmp/zabbix_proxy_19081.pinger;/usr/sbin/fping6 -C5 -b64 2>&1 </tmp/zabbix_proxy_19081.pinger;
19081:20140930:115238.951 End of process_ping()
19081:20140930:115238.951 End of do_ping():SUCCEED
19081:20140930:115238.951 In process_values()
19081:20140930:115238.951 Host [10.10.10.10] cnt=0 rcv=0 min/max/avg=0.000000/0.000000/0.000000
19081:20140930:115238.952 In process_value()
19081:20140930:115238.952 End of process_value()

And at the same time in audit.log
type=AVC msg=audit(1412067285.990:993): avc: denied

{ read }

for pid=30018 comm="fping" path="/tmp/zabbix_proxy_19079.pinger" dev=dm-0 ino=1840307 scontext=unconfined_u:system_r:ping_t:s0 tcontext=unconfined_u:object_r:initrc_tmp_t:s0 tclass=file

Comment by Volker Fröhlich [ 2014 Sep 30 ]

The policy assumes the pinger files to be in /var/lib/zabbixsrv/tmp.

https://bugzilla.redhat.com/show_bug.cgi?id=890554

Comment by Aleksandrs Saveljevs [ 2014 Sep 30 ]

It seems that fping is executed, but it does not output anything. There are valid cases where fping may not output anything, like this conditional in src/libs/zbxicmpping/icmpping.c:

if (NULL == (f = popen(tmp, "r")))
{
	zbx_snprintf(error, max_error_len, "%s: %s", tmp, zbx_strerror(errno));

	unlink(filename);

	return ret;
}

if (NULL == fgets(tmp, sizeof(tmp), f))
{
	ret = SUCCEED; /* fping does not output anything for DNS names that fail to resolve */
}
else
...

So we cannot distinguish between DNS resolution failure and SELinux blocking the pings based on output alone.

One thing we could do is make icmppingloss[] return 100% in such cases, instead of 0%. That would be more correct.

Comment by Aleksandrs Saveljevs [ 2014 Oct 02 ]

Another idea would be to make the items go unsupported if we could not send pings to the host (i.e., in case 0 == hosts[h].cnt in process_values()).

Comment by Igors Homjakovs (Inactive) [ 2014 Oct 29 ]

Fixed in svn://svn.zabbix.com/branches/dev/ZBX-8828

Comment by Aleksandrs Saveljevs [ 2014 Oct 31 ]

Alternative fix which detects the inability to send ICMP ping packets to a host based on "0 == hosts[h].cnt" is available in svn://svn.zabbix.com/branches/dev/ZBX-8828-alt .

If there is just one host to ping and it fails due to DNS resolution, the error message will be like this:

fping failed: non-existing.dns.name address not found

Here, the last line of fping output is used for the error message, following the principles of ZBX-1557.

If some hosts are pingable and some are not, the error message for the latter will be like this:

Cannot send ICMP ping packets to this host.

If fping does not output anything at all, as is the case with SELinux originally reported here, the error message will be:

fping failed: no output

sasha I like it! I made some changes in r51073. Please review.

TESTED

asaveljevs Looks good, thank you! CLOSED.

Comment by Aleksandrs Saveljevs [ 2015 Jan 07 ]

The alternative solution was chosen, because it is simpler and easier to merge from 2.0 all the way to trunk. However, the solution implemented by igorsh is also good and might be implemented in future in trunk only.

Fixed in pre-2.0.15 r51431, pre-2.2.9 r51432, pre-2.4.4 r51433, and pre-2.5.0 (trunk) r51435.

Comment by Aleksandrs Saveljevs [ 2015 Jan 07 ]

(1) Documented at the following locations:

sasha CLOSED

Generated at Tue Apr 23 15:11:14 EEST 2024 using Jira 9.12.4#9120004-sha1:625303b708afdb767e17cb2838290c41888e9ff0.