[ZBX-14887] Zabbix proxy can send only up to 2000 auto registered hosts per second Created: 2018 Sep 20  Updated: 2024 Apr 10  Resolved: 2018 Oct 21

Status: Closed
Project: ZABBIX BUGS AND ISSUES
Component/s: Proxy (P)
Affects Version/s: 3.4.14, 4.0.0rc1
Fix Version/s: 3.4.15rc1, 4.0.1rc1, 4.2.0alpha1, 4.2 (plan)

Type: Problem report Priority: Trivial
Reporter: Vladislavs Sokurenko Assignee: Michael Veksler
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Team: Team A
Sprint: Sprint 43, Sprint 44, Sprint 45
Story Points: 0.5

 Description   

Register 15000 hosts on Zabbix proxy
Start Zabbix server and see that only 2000 hosts are sent in at once, then datasender sleeps in my case for 5 seconds, due to datasender frequency.

Expected:
It sends 10000 values, then does not sleep and sends 5000 more

Following patch improves situation as it will not fail when *records_num grows to 2000

Index: src/libs/zbxdbhigh/proxy.c
===================================================================
--- src/libs/zbxdbhigh/proxy.c	(revision 85065)
+++ src/libs/zbxdbhigh/proxy.c	(working copy)
@@ -1985,7 +1985,7 @@
 	}
 	DBfree_result(result);
 
-	if (ZBX_MAX_HRECORDS == *records_num)
+	if (ZBX_MAX_HRECORDS == *records_num - records_num_last)
 		*more = ZBX_PROXY_DATA_MORE;
 
 	zabbix_log(LOG_LEVEL_DEBUG, "End of %s():%d lastid:" ZBX_FS_UI64 " more:%d size:" ZBX_FS_SIZE_T,

Later there is code that will never be executed because ZBX_MAX_HRECORDS_TOTAL equal 10000 and it will never be reached

	while (ZBX_DATA_JSON_BATCH_LIMIT > j->buffer_offset)
	{
		proxy_get_history_data_simple(j, ZBX_PROTO_TAG_AUTO_REGISTRATION, &areg, lastid, &id, &records_num,
				more);

		if (ZBX_PROXY_DATA_DONE == *more || ZBX_MAX_HRECORDS_TOTAL < records_num)
			break;
	}


 Comments   
Comment by Ingus Vilnis [ 2018 Oct 01 ]

This actually sounds like a good throttling mechanism to not kill Zabbix server (and/or) DB immediately after startup with forcing it to register 15k hosts at once. But on the other hand - how many real use cases with so many hosts will there be?

Comment by Vladislavs Sokurenko [ 2018 Oct 01 ]

As mentioned there is original limit set to 10000 and logic around it. Unfortunately some parts of code are logically dead, this should be fixed or logically dead code removed. But currently 10000 should be fine as Zabbix server will filter out duplicates and already registered hosts.

Comment by richlv [ 2018 Oct 01 ]

As for use cases, I suspect ZBXNEXT-2365 was related to discovering this slowdown.

Comment by Ingus Vilnis [ 2018 Oct 01 ]

Sure, fixing dead code is reasonable. Go for it. 

I was more referring to the problems discussed in ZBX-7702 and ZBXNEXT-3566 but that seems to not be the case here. 

As for ZBXNEXT-2365 - indeed could be this.

Comment by Michael Veksler [ 2018 Oct 16 ]

Available in:

  • 3.4.15rc1 r85822
  • 4.0.1rc1 r85823
  • 4.2.0alpha1 (trunk) r85824
Generated at Thu Apr 25 20:35:11 EEST 2024 using Jira 9.12.4#9120004-sha1:625303b708afdb767e17cb2838290c41888e9ff0.