[ZBX-4780] vfs.fs.discovery fails with error 'Value should be a JSON object' Created: 2012 Mar 20 Updated: 2017 May 30 Resolved: 2012 Mar 27 |
|
Status: | Closed |
Project: | ZABBIX BUGS AND ISSUES |
Component/s: | Server (S) |
Affects Version/s: | 2.0.0rc1 |
Fix Version/s: | 2.0.0rc3 |
Type: | Incident report | Priority: | Major |
Reporter: | Andrew Howell | Assignee: | Unassigned |
Resolution: | Fixed | Votes: | 0 |
Labels: | None | ||
Remaining Estimate: | Not Specified | ||
Time Spent: | Not Specified | ||
Original Estimate: | Not Specified | ||
Environment: |
RHEL 6.2 on server and proxy |
Attachments: | ZBX-4780-lld-error.PNG ZBX-4780-lld-rule.PNG dr1lin028.discovery |
Description |
I have filesystem LLD setup using the vfs.fs.discovery key in a standard template (one for solaris, one for linux) This LLD rule fails on hosts monitored via a proxy, the same rule (using same template) works fine if monitored directly. The error shows up in the Error column of the hosts Discovery page. I can do a zabbix_get -s <host> -k vfs.fs.discovery from the proxy and it returns valid data. |
Comments |
Comment by richlv [ 2012 Mar 20 ] |
most likely the agent has not been updated - lld protocol was changed in |
Comment by Andrew Howell [ 2012 Mar 20 ] |
Please investigate this further. The server, proxy and agents involved are all running 2.0.0rc1, this was happening in 1.9.9 as well. I'm happy to collect more information to figure this out. |
Comment by richlv [ 2012 Mar 20 ] |
reopening then. when you do zabbix_get for that key, what exactly is the answer ? |
Comment by Andrew Howell [ 2012 Mar 20 ] |
output from zabbix_get -s dr1lin028 -k vfs.fs.discovery run from proxy |
Comment by Andrew Howell [ 2012 Mar 20 ] |
LLD rule and error |
Comment by richlv [ 2012 Mar 20 ] |
heh. looks like discovery json is trimmed |
Comment by richlv [ 2012 Mar 20 ] |
just to be sure, proxy database is a fresh one from the same version, right ? |
Comment by Andrew Howell [ 2012 Mar 20 ] |
Yes, I blew it away and it recreated it from scratch. |
Comment by Andrew Howell [ 2012 Mar 20 ] |
It's only affecting vfs.fs.discovery, on the same host I'm using net.if.discovery with no issues |
Comment by Andrew Howell [ 2012 Mar 20 ] |
Also it affects every host on this proxy (35) both linux and solaris agents |
Comment by richlv [ 2012 Mar 20 ] |
sasha suggested that this might be because of the length of the discovery response |
Comment by Andrew Howell [ 2012 Mar 20 ] |
That would explain why it doesn't happen on another proxy (those hosts don't have a thousand+ autofs maps) but itdoesn't explain why it works fine without a proxy. Is there some limit on the proxy that could be increased? |
Comment by richlv [ 2012 Mar 20 ] |
there might be some proxy specific limit, but nobody has investigated it more yet |
Comment by dimir [ 2012 Mar 22 ] |
It appeared to be the limit we have (64 kB) of what we put to history_text table. The response in this case is 96 kB. The reason it works on the server side is that server processes the response without storing it in the DB so basically there is no limit in that case. Removing that limit should fix that for SQLite and PostgreSQL (proxy). For MySQL we will change the db schema. There will be no fix for Oracle and IBM DB2 (we doubt anybody would use that for proxy anyways). For the record, which DB are you using for proxy? The docs will need to be updated and it should be mentioned that there will be limit for what lld returns to proxy in case of Oracle and IBM DB2. |
Comment by Andrew Howell [ 2012 Mar 22 ] |
Ah, great work figuring that out We use SQLite for our proxies. |
Comment by dimir [ 2012 Mar 27 ] |
Fixed in development branch svn://svn.zabbix.com/branches/dev/ZBX-4780 |
Comment by Alexander Vladishev [ 2012 Apr 03 ] |
Successfully tested! Please review my changes in r26563. |
Comment by dimir [ 2012 Apr 03 ] |
Fixed in pre-2.0.0 r26576. |
Comment by Andrew Howell [ 2012 Apr 11 ] |
Hi, I upgraded my proxy with the issue to pre-2.0.0rc3 r26725 and I still have the problem. (proxy uses SQLite DB) Do I also need to upgrade my server? If so, do I also need to apply the r26576 database patch? (server uses mysql DB) ALTER TABLE proxy_history |
Comment by dimir [ 2012 Apr 11 ] |
No, proxy db does not need to be patched. I have just tried it with your setup (proxy with SQLite, server using MySQL), got JSON error before r26576, then upgraded to the latest trunk and rebuild everything (didn't touch the databases), restarted everything and the error was gone. Can you confirm that after upgrade discovery rule gives exactly the same error ("Value should be a JSON object")? |
Comment by Andrew Howell [ 2012 Apr 11 ] |
Yes exact same error. I only upgraded my proxy. Do I need to upgrade the server as well? |
Comment by dimir [ 2012 Apr 11 ] |
Yep, the change affects the limit on both the server and proxy. You need to rebuild the server with MySQL support too. |
Comment by Andrew Howell [ 2012 Apr 11 ] |
Okay I'll update the server as well. Do I need to apply the schema update to the proxy_history table as well? |
Comment by dimir [ 2012 Apr 11 ] |
Nope. That would've been needed only if you'd run proxy with MySQL. |
Comment by richlv [ 2012 Apr 11 ] |
nope, server doesn't use that table. for consistency you can apply the schema change, but it's not required. |
Comment by richlv [ 2012 Apr 11 ] |
damn you. i was slower. |
Comment by dimir [ 2012 Apr 11 ] |
That'll teach you to interrupt a conversation like that! :-D |
Comment by Andrew Howell [ 2012 Apr 12 ] |
Updated the server this morning and it's working fine now. Thanks |