-
Problem report
-
Resolution: Unresolved
-
Trivial
-
7.0.4rc1, 7.2.0alpha1 (master)
-
None
-
Sprint candidates
-
0.5
It's better not to hold all items in memory for insert and then inserting them together but to insert once there is batch of 10000 to avoid high memory usage while holding all of this data since it is done under transaction anyway.
https://git.zabbix.com/projects/ZBX/repos/zabbix/pull-requests?create&targetBranch=refs%2Fheads%2Ffeature%2FDEV-3784-7.1&sourceBranch=refs%2Fheads%2Ffeature%2FDEV-3784-7.1-branch&targetRepoId=152#src/libs/zbxdbhigh/db.c
Example of only necessary changes:
int zbx_db_insert_execute_overflowed(zbx_db_insert_t *self) { int ret; if (10000 > self->rows.values_num) return SUCCEED; ret = zbx_db_insert_execute(self); zbx_db_insert_clear(self); return ret; }
zbx_db_insert_add_values(db_insert_idiscovery, (*itemdiscoveryid)++, *itemid, item->parent_itemid, item_prototype->key); zbx_db_insert_add_values(db_insert_irtdata, *itemid); zbx_db_insert_add_values(db_insert_irtname, *itemid, item->name, item->name); zbx_db_insert_execute_overflowed(db_insert_items); zbx_db_insert_execute_overflowed(db_insert_idiscovery); zbx_db_insert_execute_overflowed(db_insert_irtdata); zbx_db_insert_execute_overflowed(db_insert_irtname);
Steps:
Try to discover as many items as possible at once, for example 100000, observe that memory usage high and after the changes it will drop at least twice or more
Might be related to ZBX-22504
