[ZBX-18467] fping interval detection breaks ICMP when IPv6 is disabled Created: 2020 Oct 08 Updated: 2025 Feb 05 |
|
Status: | Confirmed |
Project: | ZABBIX BUGS AND ISSUES |
Component/s: | Proxy (P), Server (S) |
Affects Version/s: | 5.0.4 |
Fix Version/s: | None |
Type: | Incident report | Priority: | Trivial |
Reporter: | Daniel Berteaud | Assignee: | dimir |
Resolution: | Unresolved | Votes: | 3 |
Labels: | None | ||
Remaining Estimate: | Not Specified | ||
Time Spent: | Not Specified | ||
Original Estimate: | Not Specified | ||
Environment: |
CentOS 7 x86_64 |
Issue Links: |
|
Description |
The new detection of the minimum interval is breaking all pings (even IPv4) if IPv6 is disabled on the server/proxy. Here's an extract of the logs on my EL7 proxy, with DebugLevel=4 oct. 08 08:39:22 adm zabbix_proxy[21669]: testing interval 1 using command "/usr/sbin/fping6 -c1 -t50 -i1 sw1.lapiole.org" oct. 08 08:39:22 adm zabbix_proxy[21669]: In zbx_popen() command:'/usr/sbin/fping6 -c1 -t50 -i1 sw1.lapiole.org' oct. 08 08:39:22 adm zabbix_proxy[21669]: End of zbx_popen():8 oct. 08 08:39:22 adm zabbix_proxy[21683]: zbx_popen(): executing script oct. 08 08:39:22 adm zabbix_proxy[21669]: In zbx_waitpid() oct. 08 08:39:22 adm zabbix_proxy[21669]: zbx_waitpid() exited, status:4 oct. 08 08:39:22 adm zabbix_proxy[21669]: End of zbx_waitpid():21683 oct. 08 08:39:22 adm zabbix_proxy[21669]: testing interval 10 using command "/usr/sbin/fping6 -c1 -t50 -i10 sw1.lapiole.org" oct. 08 08:39:22 adm zabbix_proxy[21669]: In zbx_popen() command:'/usr/sbin/fping6 -c1 -t50 -i10 sw1.lapiole.org' oct. 08 08:39:22 adm zabbix_proxy[21669]: End of zbx_popen():8 oct. 08 08:39:22 adm zabbix_proxy[21688]: zbx_popen(): executing script oct. 08 08:39:22 adm zabbix_proxy[21669]: In zbx_waitpid() oct. 08 08:39:22 adm zabbix_proxy[21669]: zbx_waitpid() exited, status:4 oct. 08 08:39:22 adm zabbix_proxy[21669]: End of zbx_waitpid():21688 oct. 08 08:39:22 adm zabbix_proxy[21669]: Cannot detect the minimum interval of /usr/sbin/fping6 oct. 08 08:39:22 adm zabbix_proxy[21669]: End of zbx_ping():NOTSUPPORTED oct. 08 08:39:22 adm zabbix_proxy[21669]: In process_values() oct. 08 08:39:22 adm zabbix_proxy[21669]: host [sw1.lapiole.org] Cannot detect the minimum interval of /usr/sbin/fping6 Then, no more pings are sent. Running the fping6 command manually fails, not because of privileges issue, but because IPv6 is completly disabled :
[root@adm ~]# /usr/sbin/fping6 -c1 -t50 -i10 sw1.lapiole.org
(null): can't create raw socket (must run as root?) : Address family not supported by protocol
[root@adm ~]#
To work arround this, I must add in my zabbix_proxy.conf : Fping6Location=/usr/sbin/fping This way, zabbix only tries to use the IPv4 variant, and pings works again |
Comments |
Comment by dimir [ 2020 Oct 14 ] |
You can either set Fping6Location to the same as FpingLocation or set Fping6Location to any non-existent file, including empty string, which is what I'd recommend to avoid confusion when someone else views the config file: Fping6Location= |
Comment by Daniel Berteaud [ 2020 Oct 14 ] |
OK, I didn't know we could set it to an empty string to disable it. Indeed, seems to be working |
Comment by xiong wei [ 2021 Jan 08 ] |
I have a similar problem, via fping6 ping ipv4 address will Interrupt all Ping checks, Trigger unsupported items, so must clear Fping6Location setting of sovled this isse? 118014:20210108:145330.316 In add_pinger_host() addr:'10.199.174.62' 118014:20210108:145330.316 End of add_pinger_host() 118014:20210108:145330.316 zbx_setproctitle() title:'icmp pinger #1 [pinging hosts]' 118014:20210108:145330.316 In zbx_ping() hosts_count:1 118014:20210108:145330.316 testing fping interval 0 ms 118014:20210108:145330.316 In zbx_popen() command:'/usr/sbin/fping6 -c1 -t50 -i0 10.199.174.62' 118014:20210108:145330.327 End of zbx_ping():NOTSUPPORTED 118014:20210108:145330.327 End of process_pinger_hosts() 118014:20210108:145330.328 zbx_setproctitle() title:'icmp pinger #1 [got 1 values in 0.012129 sec, idle 5 sec]' 117985:20210108:145331.278 End of evaluate() error:'Cannot evaluate expression: "Cannot evaluate function "TXLX6100008.bmwbrill.cn:icmpping.max(#3)": item is not supported.".' 117985:20210108:145331.278 evaluate_expressions():expression [(ZBX_UNKNOWN0)=0] cannot be evaluated: Cannot evaluate expression: "Cannot evaluate function "TXLX6100008.bmwbrill.cn:icmpping.max(#3)": item is not supported." |
Comment by Oleksii Zagorskyi [ 2021 Oct 27 ] |
See also ZBX-7209 |
Comment by Oleksii Zagorskyi [ 2021 Dec 08 ] |
I could reproduce it with zabbix 5.0 on CentOS7. To reproduce. During booting (let's assume it's a VM) we interrupt "grub" loading, press "e", move cursor to a line starting with "linux" or "linux16" and at the end we add " ipv6.disable=1", then press Ctrl+x to continue loading. Making sure that in server config Fping* parameters are not defined:
egrep -i "^Fping" /etc/zabbix/zabbix_server.conf
Start zabbix server and getting this repeating error: 1267:20211119:173402.192 /usr/sbin/fping6: can't create raw socket (must run as root?) : Address family not supported by protocol 1267:20211119:173502.327 /usr/sbin/fping6: can't create raw socket (must run as root?) : Address family not supported by protocol Uncommenting and making value empty, restarting - resolves the issue:
# egrep -i "^Fping" /etc/zabbix/zabbix_server.conf
Fping6Location=
Test on CentOS7: # fping -C2 10.33.0.1 10.33.0.1 : [0], 84 bytes, 1.77 ms (1.77 avg, 0% loss) 10.33.0.1 : [1], 84 bytes, 0.92 ms (1.34 avg, 0% loss) 10.33.0.1 : 1.77 0.92 # fping6 -C2 10.33.0.1 fping6: can't create raw socket (must run as root?) : Address family not supported by protocol # rpm -qa | grep fping fping-3.16-1.el7.x86_64 # rpm -q --whatprovides /usr/sbin/fping6 fping-3.16-1.el7.x86_64 Test on CentOS8: # fping -C2 10.33.0.1 10.33.0.1 : [0], 84 bytes, 1.90 ms (1.90 avg, 0% loss) 10.33.0.1 : [1], 84 bytes, 0.94 ms (1.42 avg, 0% loss) 10.33.0.1 : 1.90 0.94 # fping6 -C2 10.33.0.1 bash: fping6: command not found # rpm -qa | grep fping fping-4.2-2.el8.x86_64 difference - on CentOS8 "fping6" binary is missing at all. Ubuntu 21.10 - provides "fping6" binary together with classic one in single "fping" package.
Dirty hack on CentOS7 by masking out fpig6 binary while zabbix server is running: mv /usr/sbin/fping6 /usr/sbin/fping6X immediately resolves the issue and zabbix_server (without restart) starts to collect data on fly (latest data shows no data before 18:11): 20657:20211208:180511.777 server #36 started [icmp pinger #1] 20657:20211208:180532.849 /usr/sbin/fping6: can't create raw socket (must run as root?) : Address family not supported by protocol 20657:20211208:180602.863 /usr/sbin/fping6: can't create raw socket (must run as root?) : Address family not supported by protocol 20657:20211208:180702.934 /usr/sbin/fping6: can't create raw socket (must run as root?) : Address family not supported by protocol 20657:20211208:180802.967 /usr/sbin/fping6: can't create raw socket (must run as root?) : Address family not supported by protocol 20657:20211208:180903.057 /usr/sbin/fping6: can't create raw socket (must run as root?) : Address family not supported by protocol 20657:20211208:181002.097 /usr/sbin/fping6: can't create raw socket (must run as root?) : Address family not supported by protocol 20638:20211208:181102.951 item "test icmp:icmppingloss[,1]" became supported 20638:20211208:181102.951 item "test icmp:icmpping[,1]" became supported 20638:20211208:181102.951 item "test icmp:icmppingsec[,1]" became supported Related change form fping changelog: fping 4.0 (2017-04-23) ====================== ## Incompatible Changes ##### fping and fping6 unification fping and fping6 are now unified into one binary. It means that, for example, doing 'fping google.com' is going to ping the IPv6 IP of google.com on IPv6-enabled hosts. If you need exact compatibility with old versions, you can configure and install fping twice: once for ipv4, and once for ipv6: ./configure --disable-ipv6; make clean install ./configure --disable-ipv4 --program-suffix=6; make clean install supposedly change made in https://github.com/schweikert/fping/issues/80 I compiled recent fping 5.0 from sources on CentOS8, according to the steps to separate v4 v6 binaries. 30242:20211208:191507.580 server #9 started [preprocessing worker #3] 30269:20211208:191602.504 /etc/zabbix/fping6: can't create socket (must run as root?) 30252:20211208:191602.507 error reason for "icmp-test:icmppingloss[,1]" changed: /etc/zabbix/fping6: can't create socket (must run as root?) 30252:20211208:191602.507 error reason for "icmp-test:icmpping[,1]" changed: /etc/zabbix/fping6: can't create socket (must run as root?) 30252:20211208:191602.507 error reason for "icmp-test:icmppingsec[,1]" changed: /etc/zabbix/fping6: can't create socket (must run as root?) 30269:20211208:191702.526 /etc/zabbix/fping6: can't create socket (must run as root?) 30269:20211208:191802.548 /etc/zabbix/fping6: can't create socket (must run as root?) 30269:20211208:191902.578 /etc/zabbix/fping6: can't create socket (must run as root?) Manually compiled binaries: # ll /etc/zabbix/fping* -rwsr-xr-x 1 root zabbix 143216 Dec 8 19:05 /etc/zabbix/fping -rwsr-xr-x 1 root zabbix 158880 Dec 8 19:04 /etc/zabbix/fping6 # /etc/zabbix/fping -v /etc/zabbix/fping: Version 5.0 # /etc/zabbix/fping6 -v /etc/zabbix/fping6: Version 5.0 # /etc/zabbix/fping -C2 10.33.0.1 10.33.0.1 : [0], 64 bytes, 0.813 ms (0.813 avg, 0% loss) 10.33.0.1 : [1], 64 bytes, 1.05 ms (0.932 avg, 0% loss) 10.33.0.1 : 0.813 1.05 # /etc/zabbix/fping6 -C2 10.33.0.1 /etc/zabbix/fping6: can't create socket (must run as root?) again, moving away /etc/zabbix/fping6 resolved the issue immediately and data collection starts to work on the fly. |
Comment by Oleksii Zagorskyi [ 2021 Dec 09 ] |
Based on the comment above, as this issue still may be valid for quite long time, before we hopefully fix this fping6's issue in zabbix code (detect and workaround it), I'd add this note on the Known issues page: If OS kernel is configured with IPv6 support disabled, fping6 binary cannot be used and produces unexpected error, like missing SUID bit set. Requested in ZBX-20336 |