-
Incident report
-
Resolution: Unresolved
-
Trivial
-
None
-
None
-
None
-
None
Each history syncer allocates ~33 MB on 5.0 or ~28 MB on master with the following malloc:
items = (DC_ITEM *)zbx_malloc(NULL, sizeof(DC_ITEM) * (size_t)history_num);
If DC_ITEM structure is analyzed we see that it totals in 34584
sizeof(DC_ITEM):34584 sizeof(DC_HOST):18176 sizeof(DC_INTERFACE):416
On master it is a little bit better
sizeof(DC_ITEM):28408 sizeof(DC_HOST):9928 sizeof(DC_INTERFACE):2480
Notice on master that 10 KB are required by DC_HOST structure to hold following data:
#if defined(HAVE_GNUTLS) || defined(HAVE_OPENSSL) char tls_issuer[HOST_TLS_ISSUER_LEN_MAX]; char tls_subject[HOST_TLS_SUBJECT_LEN_MAX]; char tls_psk_identity[HOST_TLS_PSK_IDENTITY_LEN_MAX]; char tls_psk[HOST_TLS_PSK_LEN_MAX]; #endif
The problem is that it is never used and same goes for DC_INTERFACE structure which makes it 12 KB
But going even further DC_ITEM only uses key buffer for history syncer and int, char, uint64 variables, those buffers below are only needed for other processes:
char trapper_hosts[ITEM_TRAPPER_HOSTS_LEN_MAX]; ... char snmp_community_orig[ITEM_SNMP_COMMUNITY_LEN_MAX], *snmp_community; char snmp_oid_orig[ITEM_SNMP_OID_LEN_MAX], *snmp_oid; char snmpv3_securityname_orig[ITEM_SNMPV3_SECURITYNAME_LEN_MAX], *snmpv3_securityname; char snmpv3_authpassphrase_orig[ITEM_SNMPV3_AUTHPASSPHRASE_LEN_MAX], *snmpv3_authpassphrase; char snmpv3_privpassphrase_orig[ITEM_SNMPV3_PRIVPASSPHRASE_LEN_MAX], *snmpv3_privpassphrase; char ipmi_sensor[ITEM_IPMI_SENSOR_LEN_MAX]; char *params; char username_orig[ITEM_USERNAME_LEN_MAX], *username; char publickey_orig[ITEM_PUBLICKEY_LEN_MAX], *publickey; char privatekey_orig[ITEM_PRIVATEKEY_LEN_MAX], *privatekey; char password_orig[ITEM_PASSWORD_LEN_MAX], *password; char snmpv3_contextname_orig[ITEM_SNMPV3_CONTEXTNAME_LEN_MAX], *snmpv3_contextname; char jmx_endpoint_orig[ITEM_JMX_ENDPOINT_LEN_MAX], *jmx_endpoint; char timeout_orig[ITEM_TIMEOUT_LEN_MAX], *timeout; char url_orig[ITEM_URL_LEN_MAX], *url; char query_fields_orig[ITEM_QUERY_FIELDS_LEN_MAX], *query_fields; char *posts; char status_codes_orig[ITEM_STATUS_CODES_LEN_MAX], *status_codes; char http_proxy_orig[ITEM_HTTP_PROXY_LEN_MAX], *http_proxy; char *headers; char ssl_cert_file_orig[ITEM_SSL_CERT_FILE_LEN_MAX], *ssl_cert_file; char ssl_key_file_orig[ITEM_SSL_KEY_FILE_LEN_MAX], *ssl_key_file; char ssl_key_password_orig[ITEM_SSL_KEY_PASSWORD_LEN_MAX], *ssl_key_password;
This leaves 16 KB for item where half of it is for key and half for poller related data. Key is also not so important and is only for error messages.
After ZBX-19307, fields that are not used by history syncer are marked and can easily be spotted.
Expected:
One of solutions can be to replace all poller/trapper related fields with pointers but that could result in allot of allocations.
Other solution could be to keep existing DC_ITEM but put parts of it in structures as subsets. Then move code that retrieve history syncer data to new function, effectively Zabbix history syncer would allocate only subset structures but other processes can continue working as before.
