[ZBX-12802] Optimize memory usage of L2 item configuration cache in preprocessing manager Created: 2017 Aug 31  Updated: 2019 Feb 07  Resolved: 2017 Oct 06

Status: Closed
Project: ZABBIX BUGS AND ISSUES
Component/s: Server (S)
Affects Version/s: 3.4.1, 4.0.0alpha1
Fix Version/s: 3.4.3rc1, 4.0.0alpha1, 4.0 (plan)

Type: Problem report Priority: Major
Reporter: Vjaceslavs Bogdanovs Assignee: Andris Zeila
Resolution: Fixed Votes: 0
Labels: cache, memory, optimization, preprocessing
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: GIF File queue_processing.gif    
Issue Links:
Causes
causes ZBX-15611 Possible crash when all preprocessing... Closed
Epic Link: DEV-591
Team: Team A
Sprint: Sprint 16, Sprint 17
Story Points: 3

 Description   

Preprocessing manager is creating L2 cache of item configuration to minimize cache locking operations.

Current solution uses DC_ITEM structure which is 12080 bytes (per item!) to hold item data, so it should be replaced with more lightweight structure containing only fields required in preprocessing manager.



 Comments   
Comment by Rostislav Palivoda [ 2017 Aug 31 ]

<ros> should we pick up for implementation?
<wiper> probably, 1m items would take 12gb, which is a lot

vjaceslavs Well, it is a 1m of "preprocessable" items, but yeah, impact is noticable.

Comment by Andris Zeila [ 2017 Aug 31 ]

Depending on the cached data we might also consider using string pool. Probably not worth it, but first need to limit the scope of the cached data to get clearer picture.

Comment by Andris Zeila [ 2017 Sep 06 ]

Fixed in development branch svn://svn.zabbix.com/branches/dev/DEV-658

Comment by Vjaceslavs Bogdanovs [ 2017 Sep 11 ]

Server side tested

Comment by Andris Zeila [ 2017 Sep 26 ]

Released in:

  • pre-3.4.3rc1 r72927
  • pre-4.0.0alpha1 r72932
Comment by Glebs Ivanovskis (Inactive) [ 2017 Sep 26 ]

(4) Looks like it introduced CID 167249.

wiper Yes, it's false positive, so I was not rushing to fix it. Not sure if removing NULL from declaration would satisfy coverity, or we should add some explicit NULL checks later.
REOPENED

wiper Fixed in development branch svn://svn.zabbix.com/branches/dev/DEV-658

vjaceslavs CLOSED

glebs.ivanovskis OK, we traded of NULL dereference for use of uninitialized variable. If you are sure that this is a false positive, just mark it as false positive and put a comment describing why you think so. However it would be nice to simplify logic so that analyzers can follow it and prove to themselves that there is nothing wrong.

wiper As I said - I wasn't sure it will satisfy coverity.
REOPENED

wiper RESOLVED and CLOSED (reviewed by glebs.ivanovskis)

glebs.ivanovskis And Coverity seems to be happy.

Comment by Andris Zeila [ 2017 Sep 28 ]

Released in:

  • pre-3.4.3rc1 r73023
  • 4.0.0alpha1 r73024
Comment by Andris Zeila [ 2017 Sep 28 ]

Released in:

  • pre-3.4.3rc1 r73041
  • pre-4.0.0alpha1 r73042
Generated at Fri Mar 29 16:11:14 EET 2024 using Jira 9.12.4#9120004-sha1:625303b708afdb767e17cb2838290c41888e9ff0.