[ZBX-9699] server does not start with the minimum CacheSize=128K Created: 2015 Jul 13  Updated: 2017 May 30  Resolved: 2015 Aug 19

Status: Closed
Project: ZABBIX BUGS AND ISSUES
Component/s: Proxy (P), Server (S)
Affects Version/s: 2.4.5
Fix Version/s: 2.5.0

Type: Incident report Priority: Minor
Reporter: Aleksandrs Saveljevs Assignee: Unassigned
Resolution: Fixed Votes: 0
Labels: cachesize, memory, startup
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

We have the following in the default Zabbix server configuration file, which specifies that the minimum value for CacheSize is 128K:

### Option: CacheSize
#	Size of configuration cache, in bytes.
#	Shared memory size for storing host, item and trigger data.
#
# Mandatory: no
# Range: 128K-8G
# Default:
# CacheSize=8M

However, the server does not start even with CacheSize=256K, even with an empty database:

 19115:20150713:112836.980 In init_configuration_cache() size:262144
 19115:20150713:112836.980 In zbx_mem_create() descr:'configuration cache' param:'CacheSize' size:222823
 19115:20150713:112836.980 zbx_shmget() removing existing shm_id:20545569
 19115:20150713:112836.980 valid user addresses: [0x7fbcb145f170, 0x7fbcb1495658] total size: 222440
 19115:20150713:112836.980 End of zbx_mem_create()
 19115:20150713:112836.981 __mem_malloc: skipped 1 asked 8072 skip_min 1432 skip_max 1432
 19115:20150713:112836.981 [file:dbconfig.c,line:446] zbx_mem_malloc(): out of memory (requested 8072 bytes)
 19115:20150713:112836.981 [file:dbconfig.c,line:446] zbx_mem_malloc(): please increase CacheSize configuration parameter

It seems it does not even manage to create all hashsets in init_configuration_cache().



 Comments   
Comment by Aleksandrs Saveljevs [ 2015 Jul 14 ]

If possible, we should not increase the minimum memory limitation: Zabbix should start with CacheSize=128K. Currently, we create quite a lot of hashsets with 1000 slots, but on small installations 1000 slots is too much. We should probably create hashsets with a smaller number of slots (e.g., 50) and they will increase as needed.

Maybe we should also introduce zbx_hashset_reserve() method, similar to zbx_vector_XXX_reserve(), that will preallocate the number of slots necessary for storing the given number of elements.

Comment by Igors Homjakovs (Inactive) [ 2015 Jul 22 ]

Fixed in svn://svn.zabbix.com/branches/dev/ZBX-9699

Comment by Aleksandrs Saveljevs [ 2015 Jul 27 ]

(1) It makes sense to make "hashset_size" the second parameter to CREATE_HASHSET_EXT(), for consistency with zbx_hashset_create_ext().

igorsh RESOLVED in r54530.

asaveljevs It does not seem to compile now. REOPENED.

igorsh RESOLVED in r54535.

asaveljevs CLOSED

Comment by Aleksandrs Saveljevs [ 2015 Jul 27 ]

(2) Hashset "config->interfaces_ht" should probably have the same number of slots by default as "config->interfaces". Currently it has 0.

igorsh RESOLVED in r54531.

asaveljevs CLOSED

Comment by Aleksandrs Saveljevs [ 2015 Jul 27 ]

(3) If we do CREATE_HASHSET(config->snmpitems, 0), then actually a hashset with 2 slots will be created. Was this intended?

This is inefficient on two levels. First, if we never use the hashset, it still uses these two slots. Second, if we start inserting items into the hashset, then it will perform many reallocations, because SLOT_GROWTH_FACTOR is 3/2, which is inefficient with small numbers. The number of slots will grow in the following sequence: 2, 3, 5, 7, 11, etc.

Three suggestions are thus proposed:

  • if 0 is specified as the initial hashset size, do not allocate space for the slots;
  • set a minimum number of slots that a hashset can have (e.g., 11 - the first prime over 10);
  • consider introducing and using zbx_hashset_reserve(), so as to reduce the number of allocation operations.

asaveljevs Note that whatever improvements are made to hashset internals, they should also be made for hashmap, so that the two data structures are in sync.

wiper RESOLVED in r54699

asaveljevs Please take a look at r54765. Among other things, it adds a check for 0 slots to zbx_hashmap_get() and removes one from zbx_hashset_iter_next().

wiper Thanks, CLOSED

Comment by Aleksandrs Saveljevs [ 2015 Jul 27 ]

(4) Consider investigating whether it is possible to start Zabbix server with the default installation if INIT_HASHSET_SIZE is reduced in string pool (hint: it is possible).

asaveljevs Consider also reducing it in dbcache.c for trends.

wiper Reduced initial string pool and trends cache size also to 100 slots.
RESOLVED in r54699

asaveljevs CLOSED

wiper With minimal cache sizes and default Zabbix server installation the configuration cache had 10% free space left. So it's quite close.

Comment by Andris Zeila [ 2015 Aug 10 ]

Released in:

  • pre-2.5.0 r54785
Generated at Fri Mar 29 14:57:56 EET 2024 using Jira 9.12.4#9120004-sha1:625303b708afdb767e17cb2838290c41888e9ff0.