[ZBXNEXT-1816] Use Curl for the agent's web.page items Created: 2013 Jul 09 Updated: 2023 Mar 23 Resolved: 2019 Aug 22 |
|
Status: | Closed |
Project: | ZABBIX FEATURE REQUESTS |
Component/s: | Agent (G), Documentation (D) |
Affects Version/s: | 2.2.0 |
Fix Version/s: | None |
Type: | Change Request | Priority: | Major |
Reporter: | Volker Fröhlich | Assignee: | Unassigned |
Resolution: | Duplicate | Votes: | 45 |
Labels: | curl, http, httpcheck, webcheck | ||
Remaining Estimate: | Not Specified | ||
Time Spent: | Not Specified | ||
Original Estimate: | Not Specified |
Issue Links: |
|
Description |
web.page items don't use Curl to do their job. The current implementation in libs/zbxsysinfo/common/http.c does not allow for redirects, SSL negotiation or everything else that is more than trivial. Furthermore, this is not obvious from the documentation, plus the fact that the agent is linked to libcurl. |
Comments |
Comment by Oleksii Zagorskyi [ 2013 Jul 11 ] |
If we would use Curl for those agent items ... then logically we would need to provide ability to specify there: http proxy, auth, browser etc things which looks too much. Also it's another dependency for agent, what would be better to avoid, IMO. We once had one problem for FreeBSD because of the package maintainer's mistake - But it indeed could be additionally documented, because I recall such a technical detail was a news even for dotneft |
Comment by Volker Fröhlich [ 2013 Jul 11 ] |
I guess it's arguable whether this item serves a real purpose. You could very easily create a user parameter with curl or wget. It's no additional dependency though, the agent binary is linked to libcurl already. I'd be happy if it was just documented. |
Comment by richlv [ 2013 Jul 11 ] |
i believe agent uses curl for net.tcp.service[https] already, but i can't find that properly documented yet |
Comment by Oleksii Zagorskyi [ 2013 Jul 11 ] |
hmm, indeed already linked: # uname -a FreeBSD mon 8.1-RELEASE FreeBSD 8.1-RELEASE #0: Mon Jul 19 02:55:53 UTC 2010 [email protected]:/usr/obj/usr/src/sys/GENERIC i386 # ldd /usr/local/sbin/zabbix_agentd | grep curl libcurl.so.6 => /usr/local/lib/libcurl.so.6 (0x280b5000) # uname -a Linux it0 3.2.0-4-amd64 #1 SMP Debian 3.2.41-2 x86_64 GNU/Linux # ldd /usr/sbin/zabbix_agentd | grep curl libcurl-nss.so.4 => /usr/lib/x86_64-linux-gnu/libcurl-nss.so.4 (0x00007f212aae3000) |
Comment by Volker Fröhlich [ 2013 Jul 11 ] |
It's also linked to ldap, ber and ssh for the same reasons or probably false reasons. |
Comment by Volker Fröhlich [ 2013 Oct 24 ] |
I think that you should either purge out those not so important functions that cause the linking or consider utilizing curl. |
Comment by dimir [ 2014 Apr 10 ] |
As rich stated currently in Zabbix agent there is only one place where we use libcurl (if --with-libcurl was passed to configure script) which is net.tcp.service[https] and we should indeed document that. What if we do the same check for libcurl when processing web.page.get and if we have it, use that otherwise use existing code? asaveljevs As far as I know, everywhere we now depend on a library, if a library is linked, we work. If a library is not linked, we do not work at all (and make the item "not supported"). With your proposal, this would make "web.page.get[]" the only place where the behavior changes depending on the linked library. That might not be as apparent and clear to the user. dimir Good point, I agree. Let's keep it simple and clear to the user. |
Comment by Aleksandrs Saveljevs [ 2014 Apr 10 ] |
Volker, regarding linking:
|
Comment by Volker Fröhlich [ 2014 Apr 10 ] |
Doesn't linking to ldap and curl sound like a waste in the face of how little code is used? |
Comment by dimir [ 2014 Apr 10 ] |
We could basically rewrite net.tcp.service to initate SSL connection using OpenSSL and fetch the data when checking "https" service. I have recently written that code, that should be pretty easy. This would aslo allow "https://" in web.page.get . And in that case we would drop libcurl depencency in agent. |
Comment by dimir [ 2014 Apr 10 ] |
I guess the easier and cleaner solution would be to change web.page.get to use libcurl and as asaveljevs noted, just return NOTSUPPORTED for web.page.get when compiled without curl support. As for ldap, I don't see any problem. "If ldap service check is needed, recompile with --with-ldap, otherwise get NOTSUPPORTED" approach sounds totally acceptable to me. Regarding what zalex_ua mentioned about supporting other CURL parameters. Currently web.page.get syntax is: web.page.get[host,<path>,<port>] At this point we could leave it as it is and in the future if there will be a need we could add more options, like http proxy etc. This will bring no regression with one exception that agents compiled without curl support will start to fail. |
Comment by Aleksandrs Saveljevs [ 2014 Oct 21 ] |
One idea to add headers and cURL options would be to extend web.page.get[] syntax with array parameters, like so: web.page.get[host,<path>,<port>,<headers>,<options>] This can then be used like so: web.page.get[127.0.0.1,index.html,80,[Host: www.example.com,User-Agent: Zabbix 2.4.1],[PROXY=127.0.0.2,FOLLOWLOCATION=1]] Another unrelated idea would be to merge "host", "path" and "port" into "url". If the reason they are currently separated is the fact that we do not use cURL and such separation makes it easy for us to construct the request, we can safely merge them when cURL support is implemented. |
Comment by richlv [ 2015 Sep 20 ] |
specifying user agent specifically requested in |
Comment by Volker Fröhlich [ 2015 Sep 28 ] |
When you are implementing this, please consider the following: Resolving names in connection with libcurl follow a different method. This surfaces when you change resolv.conf, which is read on daemon startup. Thus, changes to the file are not reflected in operations – except for web checks. These apparently use libcurl's backend for name resolution. If your primary nameserver on the Zabbix server becomes unavailable, everything in Zabbix that uses names fails, except for the web checks, if you updated resolv.conf. This behaviour is confusing. |
Comment by Marc [ 2015 Oct 30 ] |
Could underlying issue of |
Comment by Viktors Tjarve [ 2015 Nov 11 ] |
Specification draft: http://zabbix.org/wiki/Docs/specs/ZBXNEXT-1816 |
Comment by dimir [ 2016 Jan 20 ] |
I definitely vote for idea of asaveljevs to merge 3 first parameters into url. In this case the spec could mention that we need to take care of that in upgrade patch. |
Comment by George Machitidze [ 2016 Feb 04 ] |
+1 for merging web.uri or web.urn URI or URN is for future use - who knows, maybe we want to support not only http and https for access methods, shall we change everything for every new address and access method? RFC 3986 and RFC2141 describes those two. |
Comment by MightyDok [ 2016 Feb 29 ] |
+1 for merging, need curl support for web page checks |
Comment by richlv [ 2016 May 13 ] |
note that with chunked transfer encoding, web.page.get actually returns bad data as per ZBX-9892 |
Comment by Mickael Martin (Cyres) [ 2017 Aug 04 ] |
Nothing since one year (open since 4 years..) ? I need to check a remote https URL from an agent and web.page.get not support https protocol. |
Comment by dimir [ 2017 Aug 04 ] |
We'll try to push this one further, let's see how it goes. |
Comment by Glebs Ivanovskis (Inactive) [ 2017 Nov 09 ] |
|
Comment by Mickael Martin (Cyres) [ 2018 Mar 30 ] |
Always nothing for #Zbx4... |
Comment by dimir [ 2018 Apr 02 ] |
Looks like we will have the needed functionality implemented in |
Comment by richlv [ 2018 Apr 02 ] |
that issue is super-vague, though - it doesn't even say whether the new item will work on server/proxy, agent or all of them. |
Comment by Mathieu CARBONNEAUX [ 2018 Apr 25 ] |
many devices metric now can be pooled with http... now quasi all devices use https... it's required to have complete http item pool solution.
you cant developing the actual solution with adding https support, header support, method, etc... or using curl and adding argument option...
for compatibility you can add new item like : web.curl[host,<path>,<port>,<headers>,<options>] in that way you are explicit that you use curl... and all there functionality...
|
Comment by Oleksii Zagorskyi [ 2019 Aug 22 ] |
As I understand, it has been implemented in |
Comment by Robert Szulist [ 2023 Mar 23 ] |
How is this implemented in What about custom headers or the ability to follow redirects? |
Comment by Glebs Ivanovskis [ 2023 Mar 23 ] |
Checking the latest source code I see that agent sets the CURLOPT_FOLLOWLOCATION option, so redirects are followed. |
Comment by Robert Szulist [ 2023 Mar 23 ] |
The docs state that `0L` means disabled. So this code explicitly turns off HTTP redirections. |
Comment by Glebs Ivanovskis [ 2023 Mar 23 ] |
Oops, my bad. You are right. |