[#ZBXNEXT-3659] More efficient sync of configuration cache

Comment by Rostislav Palivoda (Inactive) [ 2017 Feb 15 ]

wiper please proceed with specification.

Comment by Andris Zeila [ 2017 Mar 02 ]

3.0 changes finished in development branch svn://svn.zabbix.com/branches/dev/ZBXNEXT-3659

Comment by Andris Zeila [ 2017 Mar 08 ]

(1) [S] After configuration cache synchronization syncer must dump the static configuration cache data with log level trace.

vso RESOLVED In r66233-66494

wiper Great, I slightly changed the formatting, grouping relevant data togeteher and compacting it a little. Please review r66861

vso CLOSED with small style fixes in r66886

wiper Thanks, CLOSED

Comment by Vladislavs Sokurenko [ 2017 Mar 08 ]

(2) If config has more than one row then on each configuration sync it will sync different row.

Noticed that first row is only added if row needs updating which makes sense, but later second row is added always if present.
So first sync will add row 1 but second sync will add row 2 then 3rd sync adds row 1 again etc..

	if (ZBX_DBSYNC_ROW_NONE != tag)
		dbsync_add_row(sync, 0, tag, row);

	while (NULL != (row = DBfetch(result)))
		dbsync_add_row(sync, 0, ZBX_DBSYNC_ROW_ADD, row);

Furthermore the message no records in table 'config' is always displayed, even if there is one row and needs no updating.

I suggest that we either don't read more than one config row or handle it properly.

wiper I think the safest way would be to always sync config table directly from database. Otherwise we would have to move the warning messages to dbsync.c file.

vso as far as I understand there is no guarantee on order of select if there are more than one config so I would suggest that if there are more than one row then zbx_dbsync_compare_config() should return failure similar to when select fails.

wiper No, I added order by configid, so at least there is some consistency. Failing when multiple config rows are found might break existing setups.
RESOLVED in r66782

vso CLOSED

Comment by Vladislavs Sokurenko [ 2017 Mar 09 ]

(3) lastaccess is selected from database while is never used in zbx_dbsync_compare_hosts() and DCsync_hosts()

before that it was used for proxy

proxy->lastaccess = atoi(row[24]);

wiper RESOLVED in r66763

vso CLOSED

Comment by Vladislavs Sokurenko [ 2017 Mar 10 ]

(4) If macro name is changed to invalid then it is not removed from cache, old value remains until restart of server.

This is very minor and just for information.

create global macro {$MAC}=test
reload cache then set it to invalid name.

update globalmacro set macro='{$MAC' where globalmacroid=4;

reload cache

expected:
There is no more {$MAC}
Actual:
It still remains in cache.

wiper yes, it's really minor, especially when cosidering that macro names ought to be validated by API. I think this can be closed.

vso CLOSED

Comment by Dimitri Bellini [ 2017 Mar 10 ]

Is not completely related to this case but i hope in a possible resolution of this case ~~ZBXNEXT-3448~~, i think is very useful.
Thanks

Comment by Glebs Ivanovskis (Inactive) [ 2017 Mar 10 ]

Dear dimitri.bellini, ~~ZBXNEXT-3448~~ is not considered as a part of this ticket, but I can assure that we keep your request in mind. In fact as a part of ~~ZBXNEXT-1675~~ we implement a protocol for the frontend to get Reports->Status of Zabbix from Zabbix server (mainly to offload Required performance calculation with macros in update intervals). This can be a good basis for a more detailed Status of Zabbix in the future.

Comment by Dimitri Bellini [ 2017 Mar 10 ]

Hi Glebs,
perfect, thanks so much.

Comment by Vladislavs Sokurenko [ 2017 Mar 10 ]

(5) Redundant units field in ZBX_DC_ITEM struct
It is not used anywhere.

wiper RESOLVED in r66784

vso CLOSED

Comment by Vladislavs Sokurenko [ 2017 Mar 10 ]

(6) Something impossible just happened is displayed when deleting item and then reloading cache, when there are triggers on this item.

message:

zabbix_server [17422]: ERROR [file:dbconfig.c,line:2653] Something impossible has just happened.

Log attached

wiper RESOLVED in r66295

vso CLOSED

Comment by Vladislavs Sokurenko [ 2017 Mar 10 ]

(7) Cache for numeric item is different between proxy and server

However it works correctly, maybe it's redundant ?

Server:

 25833:20170310:170612.056   ====================
 25833:20170310:170612.056   itemid 23712
 25833:20170310:170612.056   formula '2'
 25833:20170310:170612.056   units ''
 25833:20170310:170612.056   trends 365
 25833:20170310:170612.056   delta 0
 25833:20170310:170612.056   multiplier 1
 25833:20170310:170612.056   ====================

Proxy:

 25899:20170310:170004.836   In DCdump_numitems()
 25899:20170310:170004.836   itemid 23712
 25899:20170310:170004.836   formula ''
 25899:20170310:170004.836   units ''
 25899:20170310:170004.836   trends 365
 25899:20170310:170004.836   delta 0
 25899:20170310:170004.836   multiplier 0
 25899:20170310:170004.836   ====================

wiper Yes, data used only by server is not sent to the proxy. That includes item parameters used in pre-processing (multiplier, formula etc), which is done by server.

vso CLOSED

Comment by Vladislavs Sokurenko [ 2017 Mar 13 ]

(8) SSH check crash.

log attached.

I think separate ticket should be created for this , since it also crash before the cache changes.

Cache

 20548:20170313:130246.106   itemid 23725
 20548:20170313:130246.106   hostid 10105
 20548:20170313:130246.106   interfaceid 3
 20548:20170313:130246.106   lastlogsize 0
 20548:20170313:130246.106   valuemapid 0
 20548:20170313:130246.106   key 'ssh.run[""]'
 20548:20170313:130246.107   port ''
 20548:20170313:130246.107   db_error 'Password authentication failed: Authentication failed (username/password)'
 20548:20170313:130246.107   delay 30
 20548:20170313:130246.107   nextcheck 1489402975
 20548:20170313:130246.107   lastclock 0
 20548:20170313:130246.107   mtime 0
 20548:20170313:130246.107   data_expected_from 1489402965
 20548:20170313:130246.107   history 90
 20548:20170313:130246.107   type 13
 20548:20170313:130246.107   data_type 0
 20548:20170313:130246.107   value_type 3
 20548:20170313:130246.107   poller_type 0
 20548:20170313:130246.108   state 1
 20548:20170313:130246.108   db_state 1
 20548:20170313:130246.108   inventory_link 0
 20548:20170313:130246.108   location 1
 20548:20170313:130246.108   flags 0
 20548:20170313:130246.108   status 0
 20548:20170313:130246.108   unreachable 0
 20548:20170313:130246.108   update_triggers 1

 20548:20170313:130246.230   In DCdump_sshitems()
 20548:20170313:130246.231   itemid 23725
 20548:20170313:130246.231   username 'test'
 20548:20170313:130246.232   publickey ''
 20548:20170313:130246.232   privatekey ''
 20548:20170313:130246.232   password 'test'
 20548:20170313:130246.232   params 'echo "hello there"'
 20548:20170313:130246.233   authtype 0
 20548:20170313:130246.233   ====================
 20548:20170313:130246.233   End of DCdump_sshitems()

This is only reproduced when using Valgrind.

vso probably Valgrind fault CLOSED as WONTFIX

Comment by Vladislavs Sokurenko [ 2017 Mar 13 ]

(9) SSH check stuck in not supported when there is no data.

This looks like minor issue.

Create ssh check with incorrect password and so it becomes unsupported.
Change password to correct one and reload cache.

Cache is reloaded OK but item is still unsupported.

It looks like because dc_add_history
Is not launched if there is no data

		case ITEM_VALUE_TYPE_UINT64:
			if (0 != (value_flags & ZBX_DC_FLAG_NOVALUE) || GET_UI64_RESULT(result))
			{
				dc_local_add_history_uint(itemid, ts, result->ui64, result->lastlogsize, result->mtime,
						value_flags);
			}
			break;

wiper This turns out to be a bug in SSH checks, moved it to a new issue ~~ZBX-11980~~.
CLOSED

Comment by Vladislavs Sokurenko [ 2017 Mar 13 ]

(10) Reconfiguration of SNMP trap adds duplicate itemid, caused by missing semicolon, but still compiles

Steps:
Create snmptrap with key and reload cache

snmptrap[""]

Then change key and reload cache

snmptrap[]

Result:

 17343:20170313:154245.838   In DCdump_interface_snmpitem()
 17343:20170313:154245.838   interfaceid 8
 17343:20170313:154245.838   itemid 23734
 17343:20170313:154245.838   itemid 23734
 17343:20170313:154245.838   itemid 23734
 17343:20170313:154245.838   itemid 23734
 17343:20170313:154245.838   itemid 23734
 17343:20170313:154245.838   ====================
 17343:20170313:154245.838   End of DCdump_interface_snmpitem()

Part of code missing semicolon

	if (FAIL == (index = zbx_vector_uint64_search(&ifitem->itemids, item->itemid, ZBX_DEFAULT_UINT64_COMPARE_FUNC)))
		return
zbx_vector_uint64_remove_noorder(&ifitem->itemids, index);

wiper RESOLVED in r66465

vso CLOSED

Comment by Vladislavs Sokurenko [ 2017 Mar 14 ]

(11) Conditional jump or move depends on uninitialised value(s)

This is not false positive.

==6583== Conditional jump or move depends on uninitialised value(s)
==6583==    at 0x4684C6: DCreset_hosts_availability (dbconfig.c:8896)
==6583==    by 0x45B7A8: DCupdate_hosts_availability (dbcache.c:3490)
==6583==    by 0x41F284: dbconfig_thread (dbconfig.c:94)
==6583==    by 0x47B914: zbx_thread_start (threads.c:128)
==6583==    by 0x41E509: MAIN_ZABBIX_ENTRY (server.c:938)
==6583==    by 0x47A4A2: daemon_start (daemon.c:392)
==6583==    by 0x41947D: main (server.c:757)
==6583==  Uninitialised value was created by a stack allocation
==6583==    at 0x45C36D: DCfind_id (dbconfig.c:414)

wiper This was happening because of (3) subissue, however during investigation I found problem with host availability resetting.
RESOLVED in r66812

vso REOPENED, deleting disabled item decrements count of active items.

Zabbix only increment count for active items, but decrements for any items, this can't be correct.

		if (NULL != (host = zbx_hashset_search(&config->hosts, &item->hostid)))
			dc_host_update_agent_stats(host, item->type, -1);

wiper Right, only the active items must be tracked.
RESOLVED in r66862

vso CLOSED

Comment by Vladislavs Sokurenko [ 2017 Mar 14 ]

(12) Redundant tag check in DCsync_functions

First this is checked so on row remove it breaks

/* removed rows will be always added at the end */
if (ZBX_DBSYNC_ROW_REMOVE == tag)
	break;

But later there is check for tag which can never be true

if (NULL != item->triggers)
{
	if (ZBX_DBSYNC_ROW_REMOVE == tag)
	{
		config->items.mem_free_func(item->triggers);
		item->triggers = NULL;
	}
	else
		item->triggers[0] = NULL;
}

wiper RESOLVED in r66817

vso CLOSED

Comment by Vladislavs Sokurenko [ 2017 Mar 14 ]

(13) Trigger dependency removal is broken.

Create 3 triggers where 2 triggers rely on one.
Reload cache.
Delete dependency from one of triggers, see incorrect trigger dependencies in triggerdeps dump

wiper RESOLVED in r66356

vso CLOSED, also removed unused vector initializations in r66364

Comment by Vladislavs Sokurenko [ 2017 Mar 14 ]

(14) SIGSEGV during DCconfig_check_trigger_dependencies if trigger depends another trigger.

Log attached.

No such problem before configuration cache improvement.

wiper RESOLVED in r66361

vso CLOSED

Comment by Vladislavs Sokurenko [ 2017 Mar 14 ]

(15) In DCsync_trigdeps() on some conditions vector is recreated using non shared memory allocation functions.

See dc_trigder_deplist_init() and notice it creates like this

zbx_vector_ptr_create_ext(&trigdep->dependencies, __config_mem_malloc_func, __config_mem_realloc_func,
			__config_mem_free_func);

See DCsync_trigdeps() and notice that it's different now.

zbx_vector_ptr_destroy(&trigdep_down->dependencies);
zbx_vector_ptr_create(&trigdep_down->dependencies)

wiper RESOLVED in r66371

vso CLOSED

Comment by Vladislavs Sokurenko [ 2017 Mar 16 ]

(16) Memory statistics dump should not be called if log level is lower than debug

Those functions are always called and during cache lock

	zbx_mem_dump_stats(config_mem);
	zbx_mem_dump_stats(strpool->mem_info);

As well as many other debug functions

	update_sec = zbx_time() - sec;

	strpool = zbx_strpool_info();

	total = csec + hsec + hisec + htsec + gmsec + hmsec + ifsec + isec + tsec + dsec + fsec + expr_sec +
			action_sec + action_condition_sec;
	total2 = csec2 + hsec2 + hisec2 + htsec2 + gmsec2 + hmsec2 + ifsec2 + isec2 + tsec2 + dsec2 + fsec2 +
			expr_sec2 + action_sec2 + action_condition_sec2 + update_sec;

	zabbix_log(LOG_LEVEL_DEBUG, "%s() config     : sql:" ZBX_FS_DBL " sync:" ZBX_FS_DBL " sec (%d/%d/%d).",
			__function_name, csec, csec2, config_sync.add_num, config_sync.update_num,
			config_sync.remove_num);
	zabbix_log(LOG_LEVEL_DEBUG, "%s() hosts      : sql:" ZBX_FS_DBL " sync:" ZBX_FS_DBL " sec. (%d/%d/%d)",
			__function_name, hsec, hsec2, hosts_sync.add_num, hosts_sync.update_num,
			hosts_sync.remove_num);
	zabbix_log(LOG_LEVEL_DEBUG, "%s() host_invent: sql:" ZBX_FS_DBL " sync:" ZBX_FS_DBL " sec (%d/%d/%d).",
			__function_name, hisec, hisec2, hi_sync.add_num, hi_sync.update_num,
			hi_sync.remove_num);
	zabbix_log(LOG_LEVEL_DEBUG, "%s() templates  : sql:" ZBX_FS_DBL " sync:" ZBX_FS_DBL " sec (%d/%d/%d).",
			__function_name, htsec, htsec2, htmpl_sync.add_num, htmpl_sync.update_num,
			htmpl_sync.remove_num);
	zabbix_log(LOG_LEVEL_DEBUG, "%s() globmacros : sql:" ZBX_FS_DBL " sync:" ZBX_FS_DBL " sec (%d/%d/%d).",
			__function_name, gmsec, gmsec2, gmacro_sync.add_num, gmacro_sync.update_num,
			gmacro_sync.remove_num);
	zabbix_log(LOG_LEVEL_DEBUG, "%s() hostmacros : sql:" ZBX_FS_DBL " sync:" ZBX_FS_DBL " sec (%d/%d/%d).",
			__function_name, hmsec, hmsec2, hmacro_sync.add_num, hmacro_sync.update_num,
			hmacro_sync.remove_num);
	zabbix_log(LOG_LEVEL_DEBUG, "%s() interfaces : sql:" ZBX_FS_DBL " sync:" ZBX_FS_DBL " sec (%d/%d/%d).",
			__function_name, ifsec, ifsec2, if_sync.add_num, if_sync.update_num,
			if_sync.remove_num);
	zabbix_log(LOG_LEVEL_DEBUG, "%s() items      : sql:" ZBX_FS_DBL " sync:" ZBX_FS_DBL " sec (%d/%d/%d).",
			__function_name, isec, isec2, items_sync.add_num, items_sync.update_num,
			items_sync.remove_num);
	zabbix_log(LOG_LEVEL_DEBUG, "%s() triggers   : sql:" ZBX_FS_DBL " sync:" ZBX_FS_DBL " sec (%d/%d/%d).",
			__function_name, tsec, tsec2, triggers_sync.add_num, triggers_sync.update_num,
			triggers_sync.remove_num);
	zabbix_log(LOG_LEVEL_DEBUG, "%s() trigdeps   : sql:" ZBX_FS_DBL " sync:" ZBX_FS_DBL " sec (%d/%d/%d).",
			__function_name, dsec, dsec2, tdep_sync.add_num, tdep_sync.update_num,
			tdep_sync.remove_num);
	zabbix_log(LOG_LEVEL_DEBUG, "%s() functions  : sql:" ZBX_FS_DBL " sync:" ZBX_FS_DBL " sec (%d/%d/%d).",
			__function_name, fsec, fsec2, func_sync.add_num, func_sync.update_num,
			func_sync.remove_num);
	zabbix_log(LOG_LEVEL_DEBUG, "%s() expressions: sql:" ZBX_FS_DBL " sync:" ZBX_FS_DBL " sec (%d/%d/%d).",
			__function_name, expr_sec, expr_sec2, expr_sync.add_num, expr_sync.update_num,
			expr_sync.remove_num);
	zabbix_log(LOG_LEVEL_DEBUG, "%s() actions    : sql:" ZBX_FS_DBL " sync:" ZBX_FS_DBL " sec (%d/%d/%d).",
			__function_name, action_sec, action_sec2, action_sync.add_num, action_sync.update_num,
			action_sync.remove_num);
	zabbix_log(LOG_LEVEL_DEBUG, "%s() conditions : sql:" ZBX_FS_DBL " sync:" ZBX_FS_DBL " sec (%d/%d/%d).",
			__function_name, action_condition_sec, action_condition_sec2, action_condition_sync.add_num,
			action_condition_sync.update_num, action_condition_sync.remove_num);

	zabbix_log(LOG_LEVEL_DEBUG, "%s() reindex    : " ZBX_FS_DBL " sec.", __function_name, update_sec);

	zabbix_log(LOG_LEVEL_DEBUG, "%s() total sql  : " ZBX_FS_DBL " sec.", __function_name, total);
	zabbix_log(LOG_LEVEL_DEBUG, "%s() total sync : " ZBX_FS_DBL " sec.", __function_name, total2);

	zabbix_log(LOG_LEVEL_DEBUG, "%s() proxies    : %d (%d slots)", __function_name,
			config->proxies.num_data, config->proxies.num_slots);
	zabbix_log(LOG_LEVEL_DEBUG, "%s() hosts      : %d (%d slots)", __function_name,
			config->hosts.num_data, config->hosts.num_slots);
	zabbix_log(LOG_LEVEL_DEBUG, "%s() hosts_h    : %d (%d slots)", __function_name,
			config->hosts_h.num_data, config->hosts_h.num_slots);
	zabbix_log(LOG_LEVEL_DEBUG, "%s() hosts_p    : %d (%d slots)", __function_name,
			config->hosts_p.num_data, config->hosts_p.num_slots);
#if defined(HAVE_POLARSSL) || defined(HAVE_GNUTLS) || defined(HAVE_OPENSSL)
	zabbix_log(LOG_LEVEL_DEBUG, "%s() psks       : %d (%d slots)", __function_name,
			config->psks.num_data, config->psks.num_slots);
#endif
	zabbix_log(LOG_LEVEL_DEBUG, "%s() ipmihosts  : %d (%d slots)", __function_name,
			config->ipmihosts.num_data, config->ipmihosts.num_slots);
	zabbix_log(LOG_LEVEL_DEBUG, "%s() host_invent: %d (%d slots)", __function_name,
			config->host_inventories.num_data, config->host_inventories.num_slots);
	zabbix_log(LOG_LEVEL_DEBUG, "%s() htmpls     : %d (%d slots)", __function_name,
			config->htmpls.num_data, config->htmpls.num_slots);
	zabbix_log(LOG_LEVEL_DEBUG, "%s() gmacros    : %d (%d slots)", __function_name,
			config->gmacros.num_data, config->gmacros.num_slots);
	zabbix_log(LOG_LEVEL_DEBUG, "%s() gmacros_m  : %d (%d slots)", __function_name,
			config->gmacros_m.num_data, config->gmacros_m.num_slots);
	zabbix_log(LOG_LEVEL_DEBUG, "%s() hmacros    : %d (%d slots)", __function_name,
			config->hmacros.num_data, config->hmacros.num_slots);
	zabbix_log(LOG_LEVEL_DEBUG, "%s() hmacros_hm : %d (%d slots)", __function_name,
			config->hmacros_hm.num_data, config->hmacros_hm.num_slots);
	zabbix_log(LOG_LEVEL_DEBUG, "%s() interfaces : %d (%d slots)", __function_name,
			config->interfaces.num_data, config->interfaces.num_slots);
	zabbix_log(LOG_LEVEL_DEBUG, "%s() interfac_ht: %d (%d slots)", __function_name,
			config->interfaces_ht.num_data, config->interfaces_ht.num_slots);
	zabbix_log(LOG_LEVEL_DEBUG, "%s() if_snmpitms: %d (%d slots)", __function_name,
			config->interface_snmpitems.num_data, config->interface_snmpitems.num_slots);
	zabbix_log(LOG_LEVEL_DEBUG, "%s() if_snmpaddr: %d (%d slots)", __function_name,
			config->interface_snmpaddrs.num_data, config->interface_snmpaddrs.num_slots);
	zabbix_log(LOG_LEVEL_DEBUG, "%s() items      : %d (%d slots)", __function_name,
			config->items.num_data, config->items.num_slots);
	zabbix_log(LOG_LEVEL_DEBUG, "%s() items_hk   : %d (%d slots)", __function_name,
			config->items_hk.num_data, config->items_hk.num_slots);
	zabbix_log(LOG_LEVEL_DEBUG, "%s() numitems   : %d (%d slots)", __function_name,
			config->numitems.num_data, config->numitems.num_slots);
	zabbix_log(LOG_LEVEL_DEBUG, "%s() snmpitems  : %d (%d slots)", __function_name,
			config->snmpitems.num_data, config->snmpitems.num_slots);
	zabbix_log(LOG_LEVEL_DEBUG, "%s() ipmiitems  : %d (%d slots)", __function_name,
			config->ipmiitems.num_data, config->ipmiitems.num_slots);
	zabbix_log(LOG_LEVEL_DEBUG, "%s() flexitems  : %d (%d slots)", __function_name,
			config->flexitems.num_data, config->flexitems.num_slots);
	zabbix_log(LOG_LEVEL_DEBUG, "%s() trapitems  : %d (%d slots)", __function_name,
			config->trapitems.num_data, config->trapitems.num_slots);
	zabbix_log(LOG_LEVEL_DEBUG, "%s() logitems   : %d (%d slots)", __function_name,
			config->logitems.num_data, config->logitems.num_slots);
	zabbix_log(LOG_LEVEL_DEBUG, "%s() dbitems    : %d (%d slots)", __function_name,
			config->dbitems.num_data, config->dbitems.num_slots);
	zabbix_log(LOG_LEVEL_DEBUG, "%s() sshitems   : %d (%d slots)", __function_name,
			config->sshitems.num_data, config->sshitems.num_slots);
	zabbix_log(LOG_LEVEL_DEBUG, "%s() telnetitems: %d (%d slots)", __function_name,
			config->telnetitems.num_data, config->telnetitems.num_slots);
	zabbix_log(LOG_LEVEL_DEBUG, "%s() simpleitems: %d (%d slots)", __function_name,
			config->simpleitems.num_data, config->simpleitems.num_slots);
	zabbix_log(LOG_LEVEL_DEBUG, "%s() jmxitems   : %d (%d slots)", __function_name,
			config->jmxitems.num_data, config->jmxitems.num_slots);
	zabbix_log(LOG_LEVEL_DEBUG, "%s() calcitems  : %d (%d slots)", __function_name,
			config->calcitems.num_data, config->calcitems.num_slots);
	zabbix_log(LOG_LEVEL_DEBUG, "%s() deltaitems : %d (%d slots)", __function_name,
			config->deltaitems.num_data, config->deltaitems.num_slots);
	zabbix_log(LOG_LEVEL_DEBUG, "%s() functions  : %d (%d slots)", __function_name,
			config->functions.num_data, config->functions.num_slots);
	zabbix_log(LOG_LEVEL_DEBUG, "%s() triggers   : %d (%d slots)", __function_name,
			config->triggers.num_data, config->triggers.num_slots);
	zabbix_log(LOG_LEVEL_DEBUG, "%s() trigdeps   : %d (%d slots)", __function_name,
			config->trigdeps.num_data, config->trigdeps.num_slots);
	for (i = 0; i < CONFIG_TIMER_FORKS; i++)
	{
		zabbix_log(LOG_LEVEL_DEBUG, "%s() t_trigs[%d] : %d (%d allocated)", __function_name,
				i, config->time_triggers[i].values_num, config->time_triggers[i].values_alloc);
	}
	zabbix_log(LOG_LEVEL_DEBUG, "%s() expressions: %d (%d slots)", __function_name,
			config->expressions.num_data, config->expressions.num_slots);

	zabbix_log(LOG_LEVEL_DEBUG, "%s() actions    : %d (%d slots)", __function_name,
			config->actions.num_data, config->actions.num_slots);
	zabbix_log(LOG_LEVEL_DEBUG, "%s() conditions : %d (%d slots)", __function_name,
			config->action_conditions.num_data, config->action_conditions.num_slots);

	for (i = 0; ZBX_POLLER_TYPE_COUNT > i; i++)
	{
		zabbix_log(LOG_LEVEL_DEBUG, "%s() queue[%d]   : %d (%d allocated)", __function_name,
				i, config->queues[i].elems_num, config->queues[i].elems_alloc);
	}

	zabbix_log(LOG_LEVEL_DEBUG, "%s() pqueue     : %d (%d allocated)", __function_name,
			config->pqueue.elems_num, config->pqueue.elems_alloc);

	zabbix_log(LOG_LEVEL_DEBUG, "%s() configfree : " ZBX_FS_DBL "%%", __function_name,
			100 * ((double)config_mem->free_size / config_mem->orig_size));

	zabbix_log(LOG_LEVEL_DEBUG, "%s() strings    : %d (%d slots)", __function_name,
			strpool->hashset->num_data, strpool->hashset->num_slots);

	zabbix_log(LOG_LEVEL_DEBUG, "%s() strpoolfree: " ZBX_FS_DBL "%%", __function_name,
			100 * ((double)strpool->mem_info->free_size / strpool->mem_info->orig_size));

I personally would prefer moving it all to separate function if possible and invoke this code part only if LOG_LEVEL_DEBUG is set

wiper moving to separate function would be make too many changes (we would need to wrap statistical data into a structure), so I simply added log level check.
RESOLVED in r67536

vso 3.0 tested with small style fix in r67546, is 3.4 also finished ?

wiper Yes, r67545

vso CLOSED with small style fix in r67548

Comment by Vladislavs Sokurenko [ 2017 Mar 16 ]

(17) SNMP trap interface is not updated properly.

This is what happens if you change interface, it adds more to interface_snmpitems

  3246:20170316:135311.143   In DCdump_interface_snmpitems()
  3246:20170316:135311.143   ====================
  3246:20170316:135311.143   interfaceid 11
  3246:20170316:135311.144   itemid 23750
  3246:20170316:135311.144   itemid 23750
  3246:20170316:135311.144   ====================
  3246:20170316:135311.144   interfaceid 8
  3246:20170316:135311.145   itemid 23750
  3246:20170316:135311.145   ====================
  3246:20170316:135311.145   End of DCdump_interface_snmpitems()

But after server reboot it's

  3349:20170316:135430.595   In DCdump_interface_snmpitems()
  3349:20170316:135430.595   ====================
  3349:20170316:135430.596   interfaceid 11
  3349:20170316:135430.597   itemid 23750
  3349:20170316:135430.597   ====================
  3349:20170316:135430.597   End of DCdump_interface_snmpitems()

vso CLOSED

Comment by Vladislavs Sokurenko [ 2017 Mar 16 ]

(18) Changing monitor by proxy or not does not reload interface_snmpitems

From diff it looks like it only happens to interface_snmpitems

Change server to be monitored by proxy.

Server has:

3351:20170316:135638.440   In DCdump_interface_snmpitems()
  3351:20170316:135638.440   ====================
  3351:20170316:135638.440   interfaceid 11
  3351:20170316:135638.440   itemid 23750
  3351:20170316:135638.440   ====================
  3351:20170316:135638.440   End of DCdump_interface_snmpitems()

Restart server

See that there are no more snmp interface

 3512:20170316:135742.255   In DCdump_interface_snmpitems()
 3512:20170316:135742.255   End of DCdump_interface_snmpitems()

Change server not to be monitored by proxy.

No interface_snmpitems is present

  3514:20170316:135853.642   In DCdump_interface_snmpitems()
  3514:20170316:135853.642   End of DCdump_interface_snmpitems()

Restart server

Now you have it loaded

  3622:20170316:140013.206   In DCdump_interface_snmpitems()
  3622:20170316:140013.206   ====================
  3622:20170316:140013.207   interfaceid 11
  3622:20170316:140013.207   itemid 23750
  3622:20170316:140013.207   ====================
  3622:20170316:140013.208   End of DCdump_interface_snmpitems()

wiper RESOLVED in r66470

vso REOPENED, this is very minor and maybe need not fixing but
If item has no triggers then update_triggers is 1 anyway.

update_triggers 1

vso CLOSED, looks like fixed in r66820

Comment by Vladislavs Sokurenko [ 2017 Mar 17 ]

Branch svn://svn.zabbix.com/branches/dev/ZBXNEXT-3659 successfully tested.

~~Sub issue (16) still open but are simply recommendation.~~

Comment by richlv [ 2017 Mar 22 ]

"Fix Version/s" does not list 3.2 - will this change be included in the 3.2 branch as well ?

sasha It will be included only in 3.0 and 3.4 (trunk) branches.

<richlv> thanks - sounds like an exception to the support rule, probably worth documenting. maybe in "known issues" for 3.2 ?

Comment by Andris Zeila [ 2017 Mar 30 ]

Configuration synchronization changes ported to trunk in development branch svn://svn.zabbix.com/branches/dev/ZBXNEXT-3659_33

Comment by Vladislavs Sokurenko [ 2017 Mar 30 ]

(19) Sort is sorted and made unique in 3.0 but no in trunk

This piece of code is present in 3.0 but not in trunk

	zbx_vector_ptr_sort(&sort, ZBX_DEFAULT_UINT64_PTR_COMPARE_FUNC);
	zbx_vector_ptr_uniq(&sort, ZBX_DEFAULT_UINT64_PTR_COMPARE_FUNC);

	/* sort the template lists with new rows */

wiper RESOLVED in r67067

vso CLOSED

Comment by Vladislavs Sokurenko [ 2017 Apr 03 ]

(20) Cache dump should be moved to separate source file also for 3.0 as it is in 3.4

wiper RESOLVED in r67103

vso CLOSED

Comment by Vladislavs Sokurenko [ 2017 Apr 03 ]

(21) Comparison that can never be true for 3.4, same as (12)

wiper RESOLVED in r67069

vso CLOSED

Comment by Vladislavs Sokurenko [ 2017 Apr 04 ]

(22) Functionids are sorted and made unique 3 times.
This is very minor but in get_functionids it is sorted in the end so no need to call sort again.

	get_functionids(&functionids, row[2]);
	get_functionids(&functionids, row[11]);

	zbx_vector_uint64_sort(&functionids, ZBX_DEFAULT_UINT64_COMPARE_FUNC);
	zbx_vector_uint64_uniq(&functionids, ZBX_DEFAULT_UINT64_COMPARE_FUNC);

In dbsync_trigger_preproc_row() redundant

zbx_vector_uint64_create(&itemids);

wiper RESOLVED in r67068

vso looks good, some small style fixes in r67070, please review and close

wiper Thanks, CLOSED

Comment by Vladislavs Sokurenko [ 2017 Apr 04 ]

(22) Trigger dependency issues, possibly not related to this development

Note sure yet if this is related to this task but:
trigger 1 depend on trigger 2.
Send value that makes trigger 2 to go in problem state, then do the same for trigger 1.

See that only trigger 2 is in problem state.

Now resolve trigger 2

Expected:
In frontend, see that trigger 1 is still in problem so problem is shown

Actual:
There are no problems.

Now resolve both triggers and send the other way around.
Send value that makes trigger 1 to go in problem state, then do the same for trigger 2.

See that only trigger 2 is in problem state.

Now resolve trigger 2.

Actual/Expected
In frontend, see that trigger 1 is still in problem so problem is shown

wiper Actually after reading it over - it works as supposed to do. In the first case after trigger2 is in problem state trigger1 will not change it's value event if problem data is sent to corresponding item. So trigger1 will still be in OK state.

In the second case trigger1 will be in problem state, but when trigger2 is switched to problem state frontend simply hides triggers depending on triggers in problem state (but in database trigger1 still has problem value).

CLOSED

Comment by Vladislavs Sokurenko [ 2017 Apr 05 ]

(23) Each refresh trigger dependency grows with same dependency.

Log attached.

mysql> select d.triggerid_down,d.triggerid_up,h.hostid,i.itemid from trigger_depends d,hosts h,items i,functions f where h.hostid=i.hostid and i.itemid=f.itemid and f.triggerid=d.triggerid_down and h.status in (0,1);
+----------------+--------------+--------+--------+
| triggerid_down | triggerid_up | hostid | itemid |
+----------------+--------------+--------+--------+
|             19 |           22 |      1 |     70 |
|             19 |           22 |      1 |     71 |
+----------------+--------------+--------+--------+
2 rows in set (0,00 sec)

wiper RESOLVED in r67114 and r67115 in 3.0 dev branch

vso CLOSED

Comment by Vladislavs Sokurenko [ 2017 Apr 06 ]

(24) Trigger tags are being added each sync

Don't know if it's something to fix, since overall it works much faster now anyway.
Looks like triggers tags are created for template as well as for host.
Then each cache reload it will try to add those but will not add since trigger for them does not exist.
This consume some sync time.

18238:20170406:093040.542 forced reloading of the configuration cache
 18238:20170406:093040.543 In DCsync_configuration()
 18238:20170406:093041.557 DCsync_configuration() config     : sql:0.001200 sync:0.000307 sec (1/0/0).
 18238:20170406:093041.557 DCsync_configuration() hosts      : sql:0.001382 sync:0.000297 sec (0/0/0).
 18238:20170406:093041.558 DCsync_configuration() host_invent: sql:0.000536 sync:0.000255 sec (0/0/0).
 18238:20170406:093041.558 DCsync_configuration() templates  : sql:0.000582 sync:0.000255 sec (0/0/0).
 18238:20170406:093041.558 DCsync_configuration() globmacros : sql:0.000728 sync:0.000252 sec (0/0/0).
 18238:20170406:093041.558 DCsync_configuration() hostmacros : sql:0.000602 sync:0.000252 sec (0/0/0).
 18238:20170406:093041.558 DCsync_configuration() interfaces : sql:0.000762 sync:0.000190 sec (0/0/0).
 18238:20170406:093041.559 DCsync_configuration() items      : sql:0.230752 sync:0.000188 sec (0/0/0).
 18238:20170406:093041.559 DCsync_configuration() triggers   : sql:0.162763 sync:0.000199 sec (0/0/0).
 18238:20170406:093041.559 DCsync_configuration() trigdeps   : sql:0.001305 sync:0.000192 sec (0/0/0).
 18238:20170406:093041.559 DCsync_configuration() trig. tags : sql:0.392203 sync:0.017240 sec (10000/0/0).
 18238:20170406:093041.559 DCsync_configuration() functions  : sql:0.092815 sync:0.000190 sec (0/0/0).
 18238:20170406:093041.560 DCsync_configuration() expressions: sql:0.000719 sync:0.000193 sec (0/0/0).
 18238:20170406:093041.560 DCsync_configuration() actions    : sql:0.000510 sync:0.000189 sec (0/0/0).
 18238:20170406:093041.560 DCsync_configuration() conditions : sql:0.000555 sync:0.000191 sec (0/0/0).
 18238:20170406:093041.560 DCsync_configuration() corr       : sql:0.009167 sync:0.000196 sec (0/0/0).
 18238:20170406:093041.560 DCsync_configuration() corr_cond  : sql:0.001124 sync:0.000184 sec (0/0/0).
 18238:20170406:093041.561 DCsync_configuration() corr_op    : sql:0.000684 sync:0.000180 sec (0/0/0).
 18238:20170406:093041.561 DCsync_configuration() hgroups    : sql:0.000664 sync:0.000180 sec (0/0/0).
 18238:20170406:093041.561 DCsync_configuration() item pproc : sql:0.093753 sync:0.000182 sec (0/0/0).
 18238:20170406:093041.561 DCsync_configuration() reindex    : 0.000005 sec.
 18238:20170406:093041.561 DCsync_configuration() total sql  : 0.992806 sec.
 18238:20170406:093041.562 DCsync_configuration() total sync : 0.021315 sec.
 18238:20170406:093041.562 DCsync_configuration() proxies    : 1 (11 slots)
 18238:20170406:093041.562 DCsync_configuration() hosts      : 2 (11 slots)
 18238:20170406:093041.562 DCsync_configuration() hosts_h    : 1 (11 slots)
 18238:20170406:093041.562 DCsync_configuration() hosts_p    : 1 (11 slots)
 18238:20170406:093041.562 DCsync_configuration() psks       : 0 (0 slots)
 18238:20170406:093041.563 DCsync_configuration() ipmihosts  : 1 (11 slots)
 18238:20170406:093041.563 DCsync_configuration() host_invent: 1 (11 slots)
 18238:20170406:093041.563 DCsync_configuration() htmpls     : 1 (11 slots)
 18238:20170406:093041.563 DCsync_configuration() gmacros    : 3 (11 slots)
 18238:20170406:093041.563 DCsync_configuration() gmacros_m  : 3 (11 slots)
 18238:20170406:093041.563 DCsync_configuration() hmacros    : 1 (11 slots)
 18238:20170406:093041.564 DCsync_configuration() hmacros_hm : 1 (11 slots)
 18238:20170406:093041.564 DCsync_configuration() interfaces : 2 (11 slots)
 18238:20170406:093041.564 DCsync_configuration() interfac_ht: 2 (11 slots)
 18238:20170406:093041.564 DCsync_configuration() if_snmpitms: 0 (0 slots)
 18238:20170406:093041.564 DCsync_configuration() if_snmpaddr: 1 (11 slots)
 18238:20170406:093041.565 DCsync_configuration() items      : 5000 (9029 slots)
 18238:20170406:093041.565 DCsync_configuration() items_hk   : 5000 (9029 slots)
 18238:20170406:093041.565 DCsync_configuration() numitems   : 0 (1777 slots)
 18238:20170406:093041.565 DCsync_configuration() snmpitems  : 0 (0 slots)
 18238:20170406:093041.565 DCsync_configuration() ipmiitems  : 0 (0 slots)
 18238:20170406:093041.565 DCsync_configuration() flexitems  : 0 (0 slots)
 18238:20170406:093041.566 DCsync_configuration() trapitems  : 5000 (9029 slots)
 18238:20170406:093041.566 DCsync_configuration() logitems   : 0 (0 slots)
 18238:20170406:093041.566 DCsync_configuration() dbitems    : 0 (0 slots)
 18238:20170406:093041.566 DCsync_configuration() sshitems   : 0 (0 slots)
 18238:20170406:093041.566 DCsync_configuration() telnetitems: 0 (0 slots)
 18238:20170406:093041.567 DCsync_configuration() simpleitems: 0 (0 slots)
 18238:20170406:093041.567 DCsync_configuration() jmxitems   : 0 (0 slots)
 18238:20170406:093041.567 DCsync_configuration() calcitems  : 0 (0 slots)
 18238:20170406:093041.567 DCsync_configuration() deltaitems : 0 (0 slots)
 18238:20170406:093041.567 DCsync_configuration() functions  : 5000 (9029 slots)
 18238:20170406:093041.567 DCsync_configuration() triggers   : 5000 (9029 slots)
 18238:20170406:093041.568 DCsync_configuration() trigdeps   : 0 (0 slots)
 18238:20170406:093041.568 DCsync_configuration() trig. tags : 10000 (13553 slots)
 18238:20170406:093041.568 DCsync_configuration() t_trigs[0] : 0 (0 allocated)
 18238:20170406:093041.568 DCsync_configuration() expressions: 4 (11 slots)
 18238:20170406:093041.568 DCsync_configuration() actions    : 1 (11 slots)
 18238:20170406:093041.569 DCsync_configuration() conditions : 1 (11 slots)
 18238:20170406:093041.569 DCsync_configuration() corr.      : 1 (11 slots)
 18238:20170406:093041.569 DCsync_configuration() corr. conds: 1 (11 slots)
 18238:20170406:093041.569 DCsync_configuration() corr. ops  : 1 (11 slots)
 18238:20170406:093041.569 DCsync_configuration() hgroups    : 10 (17 slots)
 18238:20170406:093041.569 DCsync_configuration() item procs : 5000 (9029 slots)
 18238:20170406:093041.570 DCsync_configuration() queue[0]   : 0 (0 allocated)
 18238:20170406:093041.570 DCsync_configuration() queue[1]   : 0 (0 allocated)
 18238:20170406:093041.570 DCsync_configuration() queue[2]   : 0 (0 allocated)
 18238:20170406:093041.570 DCsync_configuration() queue[3]   : 0 (0 allocated)
 18238:20170406:093041.570 DCsync_configuration() queue[4]   : 0 (0 allocated)
 18238:20170406:093041.571 DCsync_configuration() pqueue     : 0 (0 allocated)
 18238:20170406:093041.571 DCsync_configuration() configfree : 1.540240%
 18238:20170406:093041.571 DCsync_configuration() strings    : 15239 (20333 slots)
 18238:20170406:093041.571 DCsync_configuration() strpoolfree: 25.695169%

wiper RESOLVED in r67114

vso CLOSED

Comment by Andris Zeila [ 2017 Apr 25 ]

(25) Backported dbsync environment initialization changes from 3.4 (use explicit initialization instead of automatic reference counting based one).

RESOLVED in r67535

vso CLOSED with small style fix in r67547

Comment by Vladislavs Sokurenko [ 2017 Apr 25 ]

Successfully tested

Comment by Andris Zeila [ 2017 Apr 25 ]

(26) Fixed possible crash in the case of inconsistencies of cached data - having trigger without cached host.
RESOLED in r67558

vso it's false positive sync item will not sync if it does not have hostid, let's fix anyway CLOSED

Comment by Andris Zeila [ 2017 Apr 26 ]

Released in:

pre-3.0.10 r67560
pre-3.4.0 r67575

Comment by Andris Zeila [ 2017 Apr 26 ]

(27) [D] Documented in:

vso CLOSED

martins-v I've changed the sentence grammar a bit. Normally sentences do not start with a verb. Still CLOSED.

Comment by Glebs Ivanovskis (Inactive) [ 2017 Apr 26 ]

(28) Seems like calculated item formula is not updated in cache. Responsible code is missing from dbsync_compare_item().

wiper The code was missing from trunk, RESOLVED in r67595

vso CLOSED

Comment by Andris Zeila [ 2017 May 03 ]

Released in:

pre-3.0.10 r67560
pre-3.4.0 r67658

Comment by Glebs Ivanovskis (Inactive) [ 2017 May 14 ]

(29) Have a look at ~~ZBX-12174~~.

dbconfig.c: In function ‘DCsync_configuration’:
dbconfig.c:4628:2: warning: call to function ‘zbx_dbsync_free_env’ without a real prototype [-Wunprototyped-calls]
  zbx_dbsync_free_env(config);
  ^
In file included from dbconfig.c:41:0:
dbsync.h:109:6: note: ‘zbx_dbsync_free_env’ was declared here
 void zbx_dbsync_free_env();
      ^

wiper Released in:

pre-3.0.10rc1 r68129
pre-3.4.0alpha1 r68130

glebs.ivanovskis Looks good!
CLOSED

[ZBXNEXT-3659] More efficient sync of configuration cache Created: 2017 Jan 17 Updated: 2024 Apr 10 Resolved: 2017 May 15
Status:	Closed
Project:	ZABBIX FEATURE REQUESTS
Component/s:	Proxy (P), Server (S)
Affects Version/s:	None
Fix Version/s:	3.0.10, 3.4.0alpha1, 3.4 (plan)

[ZBXNEXT-3659] More efficient sync of configuration cache Created: 2017 Jan 17 Updated: 2024 Apr 10 Resolved: 2017 May 15