[ZBX-24483] Zabbix Server Trapper processes consume memory for configuration sync over time Created: 2024 May 15  Updated: 2024 Jul 01  Resolved: 2024 Jun 08

Status: Closed
Project: ZABBIX BUGS AND ISSUES
Component/s: Server (S)
Affects Version/s: 6.0.29, 6.4.14, 7.0.0rc1
Fix Version/s: 6.0.31rc1, 6.4.16rc1, 7.0.1rc1, 7.2.0alpha1

Type: Problem report Priority: Blocker
Reporter: Edgar Akhmetshin Assignee: Vladislavs Sokurenko
Resolution: Fixed Votes: 0
Labels: Active, Configuration, Memory, Proxy, Sync, Trapper, Usage
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

RHEL 8.9
Zabbix Server 6.4.13
11 x Zabbix Active Proxy 6.4.x


Issue Links:
Duplicate
Team: Team B
Sprint: S24-W24/25
Story Points: 2

 Description   

Steps to reproduce:

  1. Configure clean instance with Active Proxies, for example 20 proxies with 250MB of the configuration each
  2. Start Server with StartTrappers=30
  3. Check memory usage of Trappers:
           646.85 Mb /srv/zabbix/sbin/zabbix_server: trapper #22 [processed data in 0.002103 sec, waiting for connection]
           107.39 Mb /srv/zabbix/sbin/zabbix_server: trapper #25 [processed data in 0.003412 sec, waiting for connection]
            61.17 Mb /srv/zabbix/sbin/zabbix_server: trapper #27 [processed data in 0.023760 sec, waiting for connection]
            42.52 Mb /srv/zabbix/sbin/zabbix_server: trapper #30 [processed data in 0.017782 sec, waiting for connection]
            37.07 Mb /srv/zabbix/sbin/zabbix_server: trapper #17 [processed data in 0.003034 sec, waiting for connection]
            34.94 Mb /srv/zabbix/sbin/zabbix_server: trapper #10 [processed data in 0.002147 sec, waiting for connection]
            15.73 Mb /srv/zabbix/sbin/zabbix_server: trapper #21 [processed data in 0.002440 sec, waiting for connection]
            14.13 Mb /srv/zabbix/sbin/zabbix_server: trapper #15 [processed data in 0.002891 sec, waiting for connection]
            14.02 Mb /srv/zabbix/sbin/zabbix_server: trapper #3 [processed data in 0.002281 sec, waiting for connection]
            13.66 Mb /srv/zabbix/sbin/zabbix_server: trapper #11 [processed data in 0.001165 sec, waiting for connection]
            13.59 Mb /srv/zabbix/sbin/zabbix_server: trapper #20 [processed data in 0.001359 sec, waiting for connection]
            13.42 Mb /srv/zabbix/sbin/zabbix_server: trapper #1 [processed data in 0.002193 sec, waiting for connection]
            13.36 Mb /srv/zabbix/sbin/zabbix_server: trapper #23 [processed data in 0.000864 sec, waiting for connection]
            13.35 Mb /srv/zabbix/sbin/zabbix_server: trapper #4 [processed data in 0.001452 sec, waiting for connection]
            13.12 Mb /srv/zabbix/sbin/zabbix_server: trapper #6 [processed data in 0.021679 sec, waiting for connection]
            13.11 Mb /srv/zabbix/sbin/zabbix_server: trapper #28 [processed data in 0.012474 sec, waiting for connection]
            13.10 Mb /srv/zabbix/sbin/zabbix_server: trapper #19 [processed data in 0.038730 sec, waiting for connection]
            13.09 Mb /srv/zabbix/sbin/zabbix_server: trapper #16 [processed data in 0.001426 sec, waiting for connection]
            13.03 Mb /srv/zabbix/sbin/zabbix_server: trapper #9 [processed data in 0.001753 sec, waiting for connection]
            13.02 Mb /srv/zabbix/sbin/zabbix_server: trapper #26 [processed data in 0.006001 sec, waiting for connection]
            12.41 Mb /srv/zabbix/sbin/zabbix_server: trapper #2 [processed data in 0.002594 sec, waiting for connection]
            12.37 Mb /srv/zabbix/sbin/zabbix_server: trapper #24 [processed data in 0.014503 sec, waiting for connection]
            12.22 Mb /srv/zabbix/sbin/zabbix_server: trapper #12 [processed data in 0.014370 sec, waiting for connection]
            10.89 Mb /srv/zabbix/sbin/zabbix_server: trapper #29 [processed data in 0.004934 sec, waiting for connection]
            10.71 Mb /srv/zabbix/sbin/zabbix_server: trapper #8 [processed data in 0.002488 sec, waiting for connection]
            10.05 Mb /srv/zabbix/sbin/zabbix_server: trapper #14 [processed data in 0.005085 sec, waiting for connection]
            10.00 Mb /srv/zabbix/sbin/zabbix_server: trapper #7 [processed data in 0.003139 sec, waiting for connection]
             9.96 Mb /srv/zabbix/sbin/zabbix_server: trapper #5 [processed data in 0.001367 sec, waiting for connection]
             9.84 Mb /srv/zabbix/sbin/zabbix_server: trapper #18 [processed data in 0.002454 sec, waiting for connection]
             9.83 Mb /srv/zabbix/sbin/zabbix_server: trapper #13 [processed data in 0.001173 sec, waiting for connection]
    

Result:
Run for 4 weeks:

       687.58 Mb /srv/zabbix/sbin/zabbix_server: trapper #9 [processed data in 0.007407 sec, waiting for connection]
       637.02 Mb /srv/zabbix/sbin/zabbix_server: trapper #24 [processed data in 0.002869 sec, waiting for connection]
       385.02 Mb /srv/zabbix/sbin/zabbix_server: trapper #11 [processed data in 0.007880 sec, waiting for connection]
       381.76 Mb /srv/zabbix/sbin/zabbix_server: trapper #28 [processed data in 0.011287 sec, waiting for connection]
       350.50 Mb /srv/zabbix/sbin/zabbix_server: trapper #10 [processed data in 0.002109 sec, waiting for connection]
       344.48 Mb /srv/zabbix/sbin/zabbix_server: trapper #17 [processed data in 0.003371 sec, waiting for connection]
       343.95 Mb /srv/zabbix/sbin/zabbix_server: trapper #4 [processed data in 0.002909 sec, waiting for connection]
       340.44 Mb /srv/zabbix/sbin/zabbix_server: trapper #29 [processed data in 0.104053 sec, waiting for connection]
       339.59 Mb /srv/zabbix/sbin/zabbix_server: trapper #1 [processed data in 0.007121 sec, waiting for connection]
       338.58 Mb /srv/zabbix/sbin/zabbix_server: trapper #18 [processed data in 0.051870 sec, waiting for connection]
       336.91 Mb /srv/zabbix/sbin/zabbix_server: trapper #22 [processed data in 0.007664 sec, waiting for connection]
       336.38 Mb /srv/zabbix/sbin/zabbix_server: trapper #27 [processed data in 0.004210 sec, waiting for connection]
       336.35 Mb /srv/zabbix/sbin/zabbix_server: trapper #13 [processed data in 0.008547 sec, waiting for connection]
       335.69 Mb /srv/zabbix/sbin/zabbix_server: trapper #20 [processed data in 0.012327 sec, waiting for connection]
       334.06 Mb /srv/zabbix/sbin/zabbix_server: trapper #7 [processed data in 0.125559 sec, waiting for connection]
       331.04 Mb /srv/zabbix/sbin/zabbix_server: trapper #6 [processed data in 0.002818 sec, waiting for connection]
       320.46 Mb /srv/zabbix/sbin/zabbix_server: trapper #2 [processed data in 0.008709 sec, waiting for connection]
       309.27 Mb /srv/zabbix/sbin/zabbix_server: trapper #3 [processed data in 0.003158 sec, waiting for connection]
       309.07 Mb /srv/zabbix/sbin/zabbix_server: trapper #16 [processed data in 0.008097 sec, waiting for connection]
       298.89 Mb /srv/zabbix/sbin/zabbix_server: trapper #5 [processed data in 0.007172 sec, waiting for connection]
       298.58 Mb /srv/zabbix/sbin/zabbix_server: trapper #30 [processed data in 0.006757 sec, waiting for connection]
       298.37 Mb /srv/zabbix/sbin/zabbix_server: trapper #19 [processed data in 0.010692 sec, waiting for connection]
       296.31 Mb /srv/zabbix/sbin/zabbix_server: trapper #14 [processed data in 0.010674 sec, waiting for connection]
       290.80 Mb /srv/zabbix/sbin/zabbix_server: trapper #23 [processed data in 0.006419 sec, waiting for connection]
       282.42 Mb /srv/zabbix/sbin/zabbix_server: trapper #15 [processed data in 0.013625 sec, waiting for connection]
       237.82 Mb /srv/zabbix/sbin/zabbix_server: trapper #26 [processed data in 0.053090 sec, waiting for connection]
       199.45 Mb /srv/zabbix/sbin/zabbix_server: trapper #21 [processed data in 0.002947 sec, waiting for connection]
       193.30 Mb /srv/zabbix/sbin/zabbix_server: trapper #25 [processed data in 0.008955 sec, waiting for connection]
        65.67 Mb /srv/zabbix/sbin/zabbix_server: trapper #12 [processed data in 0.010875 sec, waiting for connection]
        27.41 Mb /srv/zabbix/sbin/zabbix_server: trapper #8 [processed data in 0.002676 sec, waiting for connection]

Memory dump can be collected with the following example for Trappers:

dump-all-memory-of-pid.sh
#!/bin/bash

grep rw-p /proc/$1/maps \
| sed -n 's/^\([0-9a-f]*\)-\([0-9a-f]*\) .*$/\1 \2/p' \
| while read start stop; do \
    gdb --batch --pid $1 -ex \
        "dump memory $1-$start-$stop.dump 0x$start 0x$stop"; \
done

It will contain configuration data for Proxies.

Expected:
Efficient memory usage without caching unneeded information, especially for version 6.4+ with delta configuration sync. Configuration sync for Active proxies uses a lot of memory with no reason.

Same issue as already fixed for Active Proxies https://support.zabbix.com/browse/ZBX-20236.



 Comments   
Comment by Vladislavs Sokurenko [ 2024 May 21 ]

Calling malloc_trim explicitly after sending large configuration clears memory.

Comment by Vladislavs Sokurenko [ 2024 May 21 ]

Fixed in pull request feature/ZBX-24483-6.5

Comment by Vladislavs Sokurenko [ 2024 Jun 07 ]

Fixed in:

Generated at Tue Jun 17 07:09:18 EEST 2025 using Jira 9.12.4#9120004-sha1:625303b708afdb767e17cb2838290c41888e9ff0.