-
Problem report
-
Resolution: Fixed
-
Trivial
-
None
-
None
-
S24-W8/9, S24-W10/11
-
2
When adding new items to the hashset it is searched if such already exists, however itemid is unique and for first sync this check is redundant.
Another issue is that hashset slots need to be reserved to item count retrieved from database instead of reallocing over and over again, this should be done for items_hk and items
Furthermore numeric items are in own hashset which is also slow in our case when there are 5 million of items or more.
For 5 million trapper items difference on MySQL was:
354581:20240216:131456.516 DCsync_configuration() items : sql:14.247019 sync:3.813254 sec (5000103/0/0).
Before:
356501:20240216:132210.549 DCsync_configuration() items : sql:22.408493 sync:7.245631 sec (5000103/0/0).
Full statistics:
356501:20240216:132210.548 DCsync_configuration() config : sql:0.000208 sync:0.000046 sec (1/0/0). 356501:20240216:132210.548 DCsync_configuration() autoreg : sql:0.000218 sync:0.000005 sec (1/0/0). 356501:20240216:132210.548 DCsync_configuration() hosts : sql:0.005547 sync:0.001495 sec (5001/0/0). 356501:20240216:132210.548 DCsync_configuration() host_invent: sql:0.000305 sync:0.000004 sec (0/0/0). 356501:20240216:132210.548 DCsync_configuration() templates : sql:0.000204 sync:0.000048 sec (30/0/0). 356501:20240216:132210.549 DCsync_configuration() globmacros : sql:0.000203 sync:0.000006 sec (1/0/0). 356501:20240216:132210.549 DCsync_configuration() hostmacros : sql:0.003498 sync:0.004960 sec (5487/0/0). 356501:20240216:132210.549 DCsync_configuration() interfaces : sql:0.000345 sync:0.000083 sec (26/0/0). 356501:20240216:132210.549 DCsync_configuration() items : sql:22.408493 sync:7.245631 sec (5000103/0/0). 356501:20240216:132210.549 DCsync_configuration() template_items : sql:0.045589 sync:0.002078 sec (14472/0/0). 356501:20240216:132210.549 DCsync_configuration() prototype_items : sql:2.126065 sync:0.000909 sec (6535/0/0). 356501:20240216:132210.549 DCsync_configuration() item_discovery : sql:0.001385 sync:0.000881 sec (6535/0/0). 356501:20240216:132210.549 DCsync_configuration() triggers : sql:0.007696 sync:0.007448 sec (5865/0/0). 356501:20240216:132210.549 DCsync_configuration() trigdeps : sql:0.000504 sync:0.000775 sec (1751/0/0). 356501:20240216:132210.549 DCsync_configuration() trig. tags : sql:0.002211 sync:0.002196 sec (7638/0/0). 356501:20240216:132210.549 DCsync_configuration() host tags : sql:0.000527 sync:0.000341 sec (900/0/0). 356501:20240216:132210.549 DCsync_configuration() item tags : sql:0.006426 sync:0.004213 sec (23269/0/0). 356501:20240216:132210.549 DCsync_configuration() functions : sql:0.003221 sync:0.003032 sec (9630/0/0). 356501:20240216:132210.549 DCsync_configuration() expressions: sql:0.000468 sync:0.000011 sec (10/0/0). 356501:20240216:132210.549 DCsync_configuration() actions : sql:0.000268 sync:0.000003 sec (0/0/0). 356501:20240216:132210.549 DCsync_configuration() operations : sql:0.000364 sync:0.000012 sec (0/0/0). 356501:20240216:132210.549 DCsync_configuration() conditions : sql:0.000195 sync:0.000009 sec (0/0/0). 356501:20240216:132210.549 DCsync_configuration() corr : sql:0.000282 sync:0.000005 sec (0/0/0). 356501:20240216:132210.549 DCsync_configuration() corr_cond : sql:0.000315 sync:0.000003 sec (0/0/0). 356501:20240216:132210.549 DCsync_configuration() corr_op : sql:0.000176 sync:0.000003 sec (0/0/0). 356501:20240216:132210.549 DCsync_configuration() hgroups : sql:0.000745 sync:0.000023 sec (18/0/0). 356501:20240216:132210.549 DCsync_configuration() item pproc : sql:0.022925 sync:0.000026 sec (27/0/0). 356501:20240216:132210.549 DCsync_configuration() item script param: sql:0.000793 sync:0.000003 sec (0/0/0). 356501:20240216:132210.549 DCsync_configuration() maintenance: sql:0.000644 sync:0.000030 sec (0/0/0). 356501:20240216:132210.549 DCsync_configuration() reindex : 0.000429 sec. 356501:20240216:132210.549 DCsync_configuration() total sql : 24.638502 sec. 356501:20240216:132210.549 DCsync_configuration() total sync : 7.274366 sec. 356501:20240216:132210.549 DCsync_configuration() proxies : 0 (0 slots) 356501:20240216:132210.549 DCsync_configuration() hosts : 5001 (9029 slots) 356501:20240216:132210.549 DCsync_configuration() hosts_h : 2 (11 slots) 356501:20240216:132210.549 DCsync_configuration() hosts_p : 0 (0 slots) 356501:20240216:132210.549 DCsync_configuration() psks : 0 (0 slots) 356501:20240216:132210.549 DCsync_configuration() ipmihosts : 0 (0 slots) 356501:20240216:132210.549 DCsync_configuration() host_invent: 0 (0 slots) 356501:20240216:132210.549 DCsync_configuration() htmpls : 29 (43 slots) 356501:20240216:132210.549 DCsync_configuration() gmacros : 1 (11 slots) 356501:20240216:132210.549 DCsync_configuration() gmacros_m : 1 (11 slots) 356501:20240216:132210.549 DCsync_configuration() hmacros : 5487 (9029 slots) 356501:20240216:132210.549 DCsync_configuration() hmacros_hm : 5209 (9029 slots) 356501:20240216:132210.549 DCsync_configuration() kvs_paths : 0 356501:20240216:132210.549 DCsync_configuration() interfaces : 1 (11 slots) 356501:20240216:132210.549 DCsync_configuration() interfaces_snmp : 0 (0 slots) 356501:20240216:132210.549 DCsync_configuration() interfac_ht: 1 (11 slots) 356501:20240216:132210.549 DCsync_configuration() if_snmpitms: 0 (0 slots) 356501:20240216:132210.549 DCsync_configuration() if_snmpaddr: 0 (0 slots) 356501:20240216:132210.549 DCsync_configuration() item_discovery : 6535 (9029 slots) 356501:20240216:132210.549 DCsync_configuration() items : 5000103 (8911141 slots) 356501:20240216:132210.549 DCsync_configuration() items_hk : 5000103 (8911141 slots) 356501:20240216:132210.549 DCsync_configuration() numitems : 5000089 (8911141 slots) 356501:20240216:132210.549 DCsync_configuration() preprocitems: 27 (43 slots) 356501:20240216:132210.549 DCsync_configuration() preprocops : 27 (43 slots) 356501:20240216:132210.549 DCsync_configuration() snmpitems : 0 (0 slots) 356501:20240216:132210.549 DCsync_configuration() ipmiitems : 0 (0 slots) 356501:20240216:132210.549 DCsync_configuration() trapitems : 0 (0 slots) 356501:20240216:132210.549 DCsync_configuration() dependentitems : 3 (11 slots) 356501:20240216:132210.549 DCsync_configuration() logitems : 0 (0 slots) 356501:20240216:132210.549 DCsync_configuration() dbitems : 0 (0 slots) 356501:20240216:132210.549 DCsync_configuration() sshitems : 0 (0 slots) 356501:20240216:132210.549 DCsync_configuration() telnetitems: 0 (0 slots) 356501:20240216:132210.549 DCsync_configuration() simpleitems: 0 (0 slots) 356501:20240216:132210.549 DCsync_configuration() jmxitems : 0 (0 slots) 356501:20240216:132210.549 DCsync_configuration() calcitems : 0 (0 slots) 356501:20240216:132210.549 DCsync_configuration() httpitems : 0 (0 slots) 356501:20240216:132210.549 DCsync_configuration() scriptitems : 0 (0 slots) 356501:20240216:132210.549 DCsync_configuration() functions : 67 (101 slots) 356501:20240216:132210.549 DCsync_configuration() triggers : 5865 (9029 slots) 356501:20240216:132210.549 DCsync_configuration() trigdeps : 2511 (4007 slots) 356501:20240216:132210.549 DCsync_configuration() trig. tags : 7638 (13553 slots) 356501:20240216:132210.549 DCsync_configuration() expressions: 10 (17 slots) 356501:20240216:132210.549 DCsync_configuration() actions : 0 (0 slots) 356501:20240216:132210.549 DCsync_configuration() conditions : 0 (0 slots) 356501:20240216:132210.549 DCsync_configuration() corr. : 0 (0 slots) 356501:20240216:132210.549 DCsync_configuration() corr. conds: 0 (0 slots) 356501:20240216:132210.549 DCsync_configuration() corr. ops : 0 (0 slots) 356501:20240216:132210.549 DCsync_configuration() hgroups : 18 (29 slots) 356501:20240216:132210.549 DCsync_configuration() item procs : 27 (43 slots) 356501:20240216:132210.549 DCsync_configuration() maintenance: 0 (0 slots) 356501:20240216:132210.549 DCsync_configuration() maint tags : 0 (0 slots) 356501:20240216:132210.549 DCsync_configuration() maint time : 0 (0 slots) 356501:20240216:132210.549 DCsync_configuration() queue[0] : 42 (48 allocated) 356501:20240216:132210.549 DCsync_configuration() queue[1] : 0 (0 allocated) 356501:20240216:132210.549 DCsync_configuration() queue[2] : 0 (0 allocated) 356501:20240216:132210.549 DCsync_configuration() queue[3] : 0 (0 allocated) 356501:20240216:132210.549 DCsync_configuration() queue[4] : 0 (0 allocated) 356501:20240216:132210.549 DCsync_configuration() queue[5] : 58 (72 allocated) 356501:20240216:132210.549 DCsync_configuration() queue[6] : 0 (0 allocated) 356501:20240216:132210.549 DCsync_configuration() pqueue : 0 (0 allocated) 356501:20240216:132210.549 DCsync_configuration() timer queue: 0 (0 allocated) 356501:20240216:132210.549 DCsync_configuration() configfree : 59.401876% 356501:20240216:132210.549 DCsync_configuration() strings : 8696 (13553 slots)
- part of
-
-
- Confirmed
-
