[ZBX-25162] LLD processing causes high memory usage Created: 2024 Sep 03  Updated: 2025 Mar 13  Resolved: 2024 Nov 07

Status: Closed
Project: ZABBIX BUGS AND ISSUES
Component/s: Proxy (P), Server (S)
Affects Version/s: 7.0.3
Fix Version/s: 7.0.6rc1

Type: Problem report Priority: Blocker
Reporter: Edgar Akhmetshin Assignee: Andris Zeila
Resolution: Fixed Votes: 2
Labels: Usage, memory
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: PNG File 70-er_message.png     HTML File dependency_validation_7_0,diff     PNG File zbx-25162-er_message.png    
Issue Links:
Duplicate
duplicates ZBXNEXT-9233 Increasing the limit for Zabbix depen... Closed
Team: Team A
Sprint: Prev.Sprint, S24-W36/37, S24-W38/39
Story Points: 1

 Description   

Steps to reproduce:

  1. use this template https://git.zabbix.com/projects/ZBX/repos/zabbix/browse/templates/san/hpe_primera_http?at=refs%2Fheads%2Frelease%2F7.0 on the storage system with 1000 volumes and 27 items per each.
zabbix_server -R diaginfo="lld"
== LLD diagnostic information ==
Rules:3 values:38044 time:0.003263
Top.values:
  itemid:407998 values:8227
  itemid:371310 values:6695
  itemid:408110 values:6532
  itemid:407997 values:4237
  itemid:371309 values:4038
  itemid:408109 values:3990
  itemid:407993 values:1792
  itemid:371305 values:1154
  itemid:408105 values:1144
  itemid:408106 values:112
  itemid:371306 values:111
  itemid:407994 values:4
  itemid:407996 values:3
  itemid:408107 values:1
  itemid:408108 values:1
  itemid:407995 values:1
  itemid:371307 values:1
  itemid:371308 values:1
==
 ps -eo pss,pid,user,command --sort -size | \
    awk '{ hr=$1/1024 ; printf("%13.2f Mb ",hr) } { for ( x=4 ; x<=NF ; x++ ) { printf("%s ",$x) } print "" }' |\
    cut -d "" -f2 | cut -d "-" -f1
         0.00 Mb COMMAND
     55570.99 Mb /usr/sbin/zabbix_server: lld manager #1 [processed 67 LLD rules, idle 5.168091sec during 5.171266 sec]
       900.58 Mb /usr/sbin/zabbix_server: configuration syncer [synced configuration in 0.479355 sec, idle 10 sec]
       456.40 Mb /usr/sbin/zabbix_server: lld worker #4 [processed 1 LLD rules, idle 0.002939 sec during 185647.368918 sec]
        27.60 Mb /usr/sbin/zabbix_server: preprocessing manager #1 [queued 0, processed 0 values, idle 5.014507 sec during 5.022935 sec]
       187.46 Mb /usr/sbin/zabbix_server: lld worker #5 [processed 16 LLD rules, idle 4.795398 sec during 5.083439 sec]
       182.67 Mb /usr/sbin/zabbix_server: lld worker #3 [processed 1 LLD rules, idle 0.000579 sec during 31.693582 sec]
       104.03 Mb /usr/sbin/zabbix_server: lld worker #7 [processed 1 LLD rules, idle 0.000622 sec during 4561.818499 sec]
        51.97 Mb /usr/sbin/zabbix_server: trapper #4 [processed data in 0.040791 sec, waiting for connection]
        31.81 Mb /usr/sbin/zabbix_server: trapper #2 [processed data in 0.031796 sec, waiting for connection]
         2.13 Mb /usr/sbin/zabbix_server: poller #1 [got 0 values in 0.000020 sec, idle 5 sec]
         1.70 Mb /usr/sbin/zabbix_server: proxy group manager #1 started
         2.12 Mb /usr/sbin/zabbix_server: poller #2 [got 0 values in 0.000013 sec, idle 5 sec]
         2.04 Mb /usr/sbin/zabbix_server: poller #4 [got 0 values in 0.000011 sec, idle 5 sec]
         2.06 Mb /usr/sbin/zabbix_server: poller #5 [got 0 values in 0.000011 sec, idle 5 sec]
         2.04 Mb /usr/sbin/zabbix_server: poller #3 [got 0 values in 0.000012 sec, idle 5 sec]
         1.72 Mb /usr/sbin/zabbix_server: alerter #3 [sent 0, failed 0 alerts, idle 223.544909 sec ...

Result:
'Volum discovery' LLD consumes 55G os RAM.

Expected:
Don't spend 55GB of RAM on processing text data.



 Comments   
Comment by Vladislavs Sokurenko [ 2024 Sep 04 ]

There are 38000 values in queue, it could help if there would be a way to drop older values, however DEV-3784 should improve performance and there will be less values queued.

Comment by Vladislavs Sokurenko [ 2024 Sep 13 ]

Please provide json for itemid 407998 and we can try reproducing the issue

Comment by Andris Zeila [ 2024 Nov 07 ]

Released ZBX-25162 in:

  • pre-7.0.6rc1 1c053f95ac0
Generated at Sun Mar 30 16:01:21 EEST 2025 using Jira 9.12.4#9120004-sha1:625303b708afdb767e17cb2838290c41888e9ff0.