[ZBX-16082] If SourceIP= is set to IPv4 in /etc/zabbix/zabbix_proxy.conf, the curl should called with -4 flag. Created: 2019 May 04  Updated: 2019 Jun 13  Resolved: 2019 Jun 12

Status: Closed
Project: ZABBIX BUGS AND ISSUES
Component/s: Proxy (P)
Affects Version/s: 4.2.1
Fix Version/s: None

Type: Patch request Priority: Trivial
Reporter: Alex Grebenschikov Assignee: Viktors Tjarve
Resolution: Won't fix Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:
  • Zabbix Proxy with IPv4/IPv6 on its interface.
  • SourceIP= set to IPv4 in /etc/zabbix/zabbix_proxy.conf
  • Web-Checks n Zabbix are used to check a site which has both IPv4/IPv6

Attachments: Text File debug_log5.txt    
Team: Team A
Sprint: Sprint 52 (May 2019), Sprint 53 (Jun 2019)

 Description   

Steps to reproduce:

  1. Zabbix Proxy with IPv4/IPv6 on its interface.
  2. SourceIP= set to IPv4 in /etc/zabbix/zabbix_proxy.conf
  3. Web-Checks n Zabbix are used to check a site which has both IPv4/IPv6

Result:

  1. curl --interface 1.2.3.4 -I https://example.net/
    curl: (45) bind failed with errno 22: Invalid argument

Expected:

  1. curl --interface 1.2.3.4 -4 -I https://example.net/
    HTTP/1.1 200 OK
    Server: nginx
    ....

Possible solution:

If `SourceIP=` is set to `IPv4` in `/etc/zabbix/zabbix_proxy.conf`, the curl should called with `-4` flag.

Regards,
Alex.



 Comments   
Comment by Andrei Gushchin (Inactive) [ 2019 May 04 ]

Thank you for reporting this.
What curl version do you use?
Any IPv6 addressed on the same interface?
Acutally zabbix set this option in code:

  if (NULL != CONFIG_SOURCE_IP)
        {
                if (CURLE_OK != (err = curl_easy_setopt(easyhandle, CURLOPT_INTERFACE, CONFIG_SOURCE_IP)))
                {
                        *error = zbx_dsprintf(*error, "Cannot specify source interface for outgoing traffic: %s",
                                        curl_easy_strerror(err));
                        return FAIL;
                }
        }

Do you have some issues with getting data?

Comment by Alex Grebenschikov [ 2019 May 04 ]

Thank you for your time. Here you are:

CentOS Linux release 7.6.1810 (Core)
curl 7.29.0 (x86_64-redhat-linux-gnu) libcurl/7.29.0 NSS/3.36 zlib/1.2.7 libidn/1.28 libssh2/1.4.3

installed from a basic CentOS repository.

Connections from Zabbix Proxy per web-check fail. As curl fails to connect to a IPv6 with source ip set to IPv4.

A web monitoring page in zabbix shows 

Step "General Check via HTTPS" [1 of 1] failed: Failed binding local connection end: bind failed with errno 22: Invalid argument

in red, and no data retrieved. Confirmed through nginx's logs from a target server and tcpdump session.

The IPv4 and IPv6 are added to one and the same eth0:

eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
 inet 1.2.3.4 netmask 255.255.254.0 broadcast 1.2.3.255
 inet6 fe80::200:ff:fe42:8651 prefixlen 64 scopeid 0x20<link>
 inet6 1111:2222:db:1::1 prefixlen 64 scopeid 0x0<global>
 ether 00:00:00:42:86:51 txqueuelen 1000 (Ethernet)

I hope it helps.

Comment by Alex Grebenschikov [ 2019 May 04 ]

So I believe you need to add CURL_IPRESOLVE_V4 for this case.

 

https://curl.haxx.se/libcurl/c/CURLOPT_IPRESOLVE.html

Comment by Andrei Gushchin (Inactive) [ 2019 May 04 ]

Do domains in the web checks resolved to IPv4 or both?

Comment by Alex Grebenschikov [ 2019 May 04 ]

Resolve both to IPv6 and IPv4.

Comment by Andrei Gushchin (Inactive) [ 2019 May 04 ]

Thanks. Probably this makes sense.

Comment by Viktors Tjarve [ 2019 May 13 ]

Hi Alex,

Is it possible that you shear more information how it is possible to reproduce this issue from within Zabbix? Also could you please add here the errors from log files of Zabbx proxy and server that are related to this issue?

Does the error message appear in "Info" column in Web scenario page?

Thanks

Comment by Alex Grebenschikov [ 2019 May 13 ]

Hello Viktors,

Most details were already provided, anyway if you want them, here you are:

  1. Zabbix Proxy server with IPv4 + IPv6.
  2. SourceIP= set to IPv4, e.g. 1.2.3.4 in /etc/zabbix/zabbix_proxy.conf
  3. Add a site with a domain which resolves both to IPv4 and IPv6 in web-checks (access over DNS name).
  4. Web-monitoring page in Zabbix (Monitoring -> Web) will give Step "General Check via HTTPS" [1 of 1] failed: Failed binding local connection end: bind failed with errno 22: Invalid argument
  5. The Info column has a red squire with i, and whenever hover a cursor over I see the same error Step "General Check via HTTPS" [1 of 1] failed: Failed binding local connection end: bind failed with errno 22: Invalid argument
  6. Zabbix Proxy does not log anything related either at DebugLevel=2 or DebugLevel=3 levels. DebugLevel 5 is shown below:

debug_log5.txt

I really hope it is sufficient.

Regards.
Alex.

Comment by Alex Grebenschikov [ 2019 May 15 ]

Hello Viktors,

If you need anything else, kindly let me know.

I believe you might need to adjust the block

 

	if (NULL != CONFIG_SOURCE_IP)
	{
		if (CURLE_OK != (err = curl_easy_setopt(easyhandle, CURLOPT_INTERFACE, CONFIG_SOURCE_IP)))
		{
			*error = zbx_dsprintf(*error, "Cannot specify source interface for outgoing traffic: %s",
					curl_easy_strerror(err));
			return FAIL;
		}
	}

 

from https://github.com/zabbix/zabbix/blob/master/src/libs/zbxhttp/http.c#L52  and add a check of CONFIG_SOURCE_IP is IPv4, and if this is a case add instruction

 

curl_easy_setopt(easyhandle, CURLOPT_IPRESOLVE, CURL_IPRESOLVE_V4);

 

Sorry, my knowledge of C-language allows me at the moment only reading.

I've found this example: https://stackoverflow.com/a/3736377 

Regards.
Alex.

Comment by Viktors Tjarve [ 2019 May 15 ]

Hi Alex,

Thanks for your input. The extra information you provided confirmed some of our suspicions. I have been able to reproduce this issue when messing with SourceIP parameter in conf file.
It might turn out that to properly resolve this issue adding CURL_IPRESOLVE_V4 to CURL options in the code is not enough and more work has to be done.

Kind Regards,
Viktors

Comment by Alex Grebenschikov [ 2019 May 16 ]

Hello Viktors,

Thanks for the explanation. Kindly let me know if I can help you with anything else.

Regards,
Alex.

Comment by Alex Grebenschikov [ 2019 Jun 08 ]

Reported the issue to cURL developers here: https://github.com/curl/curl/issues/3993

Probably a quicker solution could be elaborated if to join the efforts.

 

Regards,
Alex.

Comment by Viktors Tjarve [ 2019 Jun 11 ]

Hi Alex,
Has the libcurl version 7.65.2 fixed the issue discussed here for your case?
Viktors

Comment by Alex Grebenschikov [ 2019 Jun 12 ]

Hello Viktors,

Yes, I can confirm, the issue has been fixed with a custom `libcurl` build.  The version `7.65.2` does not seem to be released yet. Anyway the master branch contains a fix.

Comment by Alex Grebenschikov [ 2019 Jun 12 ]

Fixed through upgrade of libcurl, 

bindlocal: detect and avoid IP version mismatches in bind() #4002

https://github.com/curl/curl/pull/4002

 

Comment by Viktors Tjarve [ 2019 Jun 12 ]

Great.
Thanks Alex.

Generated at Fri Mar 29 07:00:52 EET 2024 using Jira 9.12.4#9120004-sha1:625303b708afdb767e17cb2838290c41888e9ff0.