-
Documentation task
-
Resolution: Unresolved
-
Trivial
-
None
-
None
-
None
-
Support backlog
It should be documented that it's possible to use tcmalloc to profile memory for Zabbix server/Zabbix proxy when there is suspicion that more memory is consumed than should on big installations.
When compiling do not strip debug symbols also best to compile with following CFLAGS:
export CFLAGS="-g -O0"
Run with tcmalloc, profiling will be automatically dumbed when peak memory for process exceeds 4 gigabytes.
LD_PRELOAD="/usr/lib/aarch64-linux-gnu/libtcmalloc.so" HEAPPROFILE=./heap_profile HEAP_PROFILE_ALLOCATION_INTERVAL=0 HEAP_PROFILE_INUSE_INTERVAL=4294967296 HEAPPROFILESIGNAL=5 MALLOCSTATS=1 ./sbin/zabbix_server -f -c /etc/zabbix/zabbix_server.conf
To dump profiling manually identify pid that is consuming lots of memory and send signal, replace 1234 with pid of preprocessing manager as an example:
kill -5 1234
Profile should be generated at this moment and can be printed with following command
pprof-symbolize -text ./sbin/zabbix_server ./heap_profile.0001.heap
Using local file ./sbin/zabbix_server.
Using local file ./heap_profile.0001.heap.
Total: 1078.1 MB
1076.8 99.9% 99.9% 1076.8 99.9% zbx_malloc2
1.0 0.1% 100.0% 1.0 0.1% __GI___strdup
0.2 0.0% 100.0% 0.2 0.0% CRYPTO_zalloc@@OPENSSL_3.0.0
0.1 0.0% 100.0% 0.1 0.0% OPENSSL_LH_insert@@OPENSSL_3.0.0
0.0 0.0% 100.0% 0.0 0.0% zbx_realloc2
0.0 0.0% 100.0% 0.1 0.0% PKCS7_decrypt@@OPENSSL_3.0.0
0.0 0.0% 100.0% 0.0 0.0% find_best_tree_node
0.0 0.0% 100.0% 0.0 0.0% CRYPTO_strndup@@OPENSSL_3.0.0
...
0.0 0.0% 100.0% 0.0 0.0% preprocessing_flush_value
0.0 0.0% 100.0% 1074.0 99.6% preprocessor_add_request
- part of
-
ZBX-25050 System uses virtual memory instead of available memory for preprocessing
-
- Sign off by Support
-