[ZBX-5863] LLD and 64kB size limit Created: 2012 Nov 16  Updated: 2018 Feb 09  Resolved: 2013 Jan 24

Status: Closed
Project: ZABBIX BUGS AND ISSUES
Component/s: Agent (G)
Affects Version/s: 2.0.3
Fix Version/s: 2.0.5rc1, 2.1.0

Type: Incident report Priority: Major
Reporter: Ildefonso Camargo Assignee: Unassigned
Resolution: Fixed Votes: 3
Labels: discovery, json, lld
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

OS: Debian GNU/Linux Squeeze, 64 bits
Zabbix DB: PostgreSQL 8.4


Issue Links:
Duplicate
is duplicated by ZBX-5497 SNMP LLD fails if the {#SNMPINDEX} va... Closed

 Description   

Greetings,

I created a simple script for discovering DB objects, and it have been working just fine on several servers, recently I installed it on a server that has many DB objects: the discovered DB objects never showed up on Zabbix UI, and I saw nothing on the server's logs (nor agent's logs either), and on the UI I was getting "Value should be a JSON object", so, I tried to run zabbix_get :

zabbix_get -s SERVER_IP -k postgresql.tables.discovery

and I saw the output was truncated, so, I decided to see if it had a particular size:

Code:

zabbix_get -s SERVER_IP -k postgresql.tables.discovery|wc -c
65536

The total size is 82065 (from discovery script output)

So, I was wondering: is there a limit on the JSON answer size?

I found something similar, but for proxy, on bug ZBX-4780, but it says it was resolved.

Thanks in advance!

Ildefonso Camargo



 Comments   
Comment by Oleksii Zagorskyi [ 2012 Nov 17 ]

Here is some info about limitations http://www.zabbix.com/documentation/2.0/manual/installation/upgrade_notes_203, probably not related.

Comment by Ildefonso Camargo [ 2012 Nov 17 ]

Oleksiy, thanks for answering!

Yes, unfortunately, it is unrelated.... the limit I'm observing is just 64kB .

I'll try to dive into the code today, maybe it is something simple to fix, but I'm not familiar with Zabbix's code, so, it can take some time for me. Any help is welcome.

Ildefonso.

Comment by Ildefonso Camargo [ 2012 Nov 22 ]

sigh... I can't see a way of changing original issue description, this is not on Server, this is on Agent. On this thread, Palmertree gave a solution:

http://www.zabbix.com/forum/showthread.php?p=122179

changing include/common.h

#define MAX_BUFFER_LEN 65536

to

#define MAX_BUFFER_LEN 262144

And recompile Zabbix Agent, fixed the problem.

Please, remember this JSON object is coming from a UserParameter script.

I'm not sure if changing this value globally is the answer, as it could affect other components so, please, someone with more knowledge on the source code, can comment if is it safe to change this constant globally?

Thanks in advance,

Ildefonso Camargo

Comment by Oleksii Zagorskyi [ 2012 Nov 22 ]

Component has been changed from server to agent, as requested.

Comment by Marc [ 2012 Dec 04 ]

Just encounter the same issue today. I've tried to gather a lot of statistics from an IBM SAN Volume Controller 2145-CG8 (~50 Items/Virtual Disk, ~500 Virtual Disks per SVC) dynamically via LLD. That worked very well until I connected my implementation to one of our real SVCs. That results in a JSON file of ~900KB for a single SVC.

Would be nice to get MAX_BUFFER_LEN configurable.

Comment by Łukasz Jernaś [ 2012 Dec 13 ]

Maybe reading/writing to the buffer should block until the buffer is free again? This would also fix our issue with the F5 load balancers.

Comment by Alexei Vladishev [ 2013 Jan 23 ]

Possibly related to ZBX-6019.

Comment by Andris Zeila [ 2013 Jan 24 ]

Fixed in development branch svn://svn.zabbix.com/branches/dev/ZBX-5863

The UserParameter (zbx_execute) output buffer is now dynamically allocated and limited to 512KB.

The built-in LLD discovery services already used dynamically allocated buffer to assemble the resulting JSON data.

Comment by richlv [ 2013 Jan 27 ]

(1) (for documentation) - would this result in memory usage reduction when lld responses are less than the previous size limit ?

wiper It might reduce peak memory usage during script execution by 60KB. I don't think this is a significant reduction.

<richlv> added a basic note at https://www.zabbix.com/documentation/2.0/manual/introduction/whatsnew205#daemon_improvements - please, review

wiper reviewed, looks good.

Comment by dimir [ 2013 Jan 29 ]

Successfully tested. Please review my changes in r33234.

Comment by Andris Zeila [ 2013 Jan 30 ]

Fixed in:
pre-2.0.5rc1 r33238
pre-2.1.0 r33239

Comment by Jori Hämäläinen [ 2018 Feb 08 ]

> The UserParameter (zbx_execute) output buffer is now dynamically allocated and limited to 512KB.

But this does not solve this and like:
> That results in a JSON file of ~900KB for a single SVC.

I am also in a situation where LLD returns more that 512kB of JSON structure. And I really cannot splice it.

What is your recommendation? I am running 3.2.10..

Comment by Viktors Tjarve [ 2018 Feb 09 ]

If JSON is sent directly to (not through proxy) zabbix_server using zabbix_sender then there is no limit for JSON size.

Comment by Jori Hämäläinen [ 2018 Feb 09 ]

My application is using Zabbix Proxy to make LLD discovery to Zabbix-Agentd on server which produces a large JSON structure. And on zabbix-agent log from Userparameter I see

3623:20180206:205304.315 command output exceeded limit of 512 KB

And LLD is the only UserParameter I have..

./zabbix_autodiscovery.pl | wc
2582 25774 855035

So I've 2582 lines of JSON answer with 855035 bytes. Every line is relevant with multiple

{#TAGS}

per line which I need to setup to monitoring.

Comment by Viktors Tjarve [ 2018 Feb 09 ]

Sorry, this tracker is for bug reports only.
Please use IRC,forums or other channels for community support - see https://www.zabbix.org/wiki/Getting_help for more detail.
Alternatively contact [email protected] for commercial support.

Generated at Thu Mar 28 18:34:02 EET 2024 using Jira 9.12.4#9120004-sha1:625303b708afdb767e17cb2838290c41888e9ff0.