-
Type:
Problem report
-
Resolution: Unresolved
-
Priority:
Trivial
-
Affects Version/s: 8.0.0rc1, 8.0.0rc2
-
Component/s: Server (S)
-
Environment:Ubuntu 24.04, zabbix-server-pgsql 2:8.0.0~rc2-1+ubuntu24.04, PostgreSQL 16.15, libcurl 8.5.0, Elasticsearch 9.5.5 (deb package, default security: HTTPS and authentication)
Steps to reproduce:
- Install Elasticsearch 9.5.5 with the default security configuration (HTTPS and authentication enabled).
- Add the Elasticsearch HTTP CA certificate (/etc/elasticsearch/certs/http_ca.crt) to the OS trust store, so that the TLS handshake succeeds.
- In zabbix_server.conf, set a URL without credentials:
HistoryProvider=elasticsearch;value_types="str,log,text,json",url=https://localhost:9200
- Start Zabbix server.
Result:
Zabbix server crashes during the history provider version check; systemd restarts it in a loop:
retrieving history provider "elasticsearch" information cannot query Elasticsearch, HTTP error message: The requested URL returned error: 401 Got signal [signal:11(SIGSEGV),reason:1,refaddr:(nil)]. Crashing ... ... === Backtrace: === 7: /usr/sbin/zabbix_server(zbx_json_open+0x19) [0x5bc563429c49] 6: /usr/sbin/zabbix_server(+0x399e2f) [0x5bc5633f6e2f] 5: /usr/sbin/zabbix_server(zbx_history_check_version+0x134) [0x5bc5633f4eb4] 4: /usr/sbin/zabbix_server(MAIN_ZABBIX_ENTRY+0x862) [0x5bc56315d2c2] 3: /usr/sbin/zabbix_server(main+0x369) [0x5bc5631511c9]
The same crash occurs with a wrong password (HTTP 401) and with a URL that returns HTTP 404 (for example, url=https://user:password@localhost:9200/no_such_path). HTTP 403 (a user without the monitor cluster privilege) goes through the same code path.
The full log is attached.
Expected:
Zabbix server logs the error and exits without crashing, as it does for other connection errors, for example:
cannot query Elasticsearch: SSL certificate problem: self-signed certificate in certificate chain cannot retrieve history provider "elasticsearch" information: Cannot perform Elasticsearch query
Analysis:
- history_elastic_perform_once(): for CURLE_HTTP_RETURNED_ERROR (except 429), the handle is neither added to the retry list nor marked as completed (conn->status stays FAIL), and the function returns 0.
- history_elastic_query() with ELASTIC_RETRIES_OFF treats 0 as SUCCEED.
- history_elastic_get_info() then calls zbx_json_open(conn.resp.page.data, ...) with page.data == NULL (CURLOPT_FAILONERROR is set, so no response body is received).
- Other callers (history_elastic_fetch(), history_elastic_fetch_batch()) check conn.resp.page.offset before parsing, so only the startup version check is affected.
- The code was introduced with the history provider refactoring (commits 7d52251ea6d "refactored elastic history storage provider" and f9f750d655f "removed retries from initial elasticsearch get_info request") and is first included in 8.0.0beta2. Zabbix 7.0 and 7.4 are not affected: zbx_elastic_version_extract() checks the curl_easy_perform() result before parsing.
Suggested fix:
In history_elastic_get_info(), return FAIL with an error (preferably including the HTTP status code) if conn.status != SUCCEED or the response is empty, before parsing it.