[ZBX-4991] zabbix[host,<type>,available] received values for agent/jmx does not change when it should Created: 2012 May 11  Updated: 2017 May 30  Resolved: 2013 Feb 25

Status: Closed
Project: ZABBIX BUGS AND ISSUES
Component/s: Server (S)
Affects Version/s: 2.0.0rc3
Fix Version/s: 2.0.6rc1, 2.1.0

Type: Incident report Priority: Critical
Reporter: Corey Shaw Assignee: Unassigned
Resolution: Fixed Votes: 4
Labels: internalmonitoring, server
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

RHEL 5.6 x64
MySQL 5.0.77


Attachments: File zabbix_patch     File zabbix_server.log    
Issue Links:
Duplicate
is duplicated by ZBX-4684 zabbix[host,<type>,available] doesn't... Closed

 Description   

I've run into a very odd situation through two different methods now where the results from the zabbix[host,<type>,available] key do not come back with the correct values. The problem this caused was that my triggers based on this particular check for agent/jmx availability were either going off or not going off when they should have.

Here's the first example:

1. Create a host that is monitored by the Zabbix server and make sure values are received correctly (especially zabbix[host,agent,available] should equal "1"). In my case, the host was created through auto-registration. I then added the appropriate templates to it.
2. Bring up a Zabbix proxy and make sure it works correctly with the Zabbix server.
3. Change that created host to be monitored by the newly created Zabbix proxy by editing it's "Server=" value in zabbix_agentd.conf to point to the Zabbix proxy. Restart the agent after making the change.
4. Verify that the host "auto-registers" itself again by changing it's already created profile in the GUI to be monitored by the proxy.
5. Verify that the "Z" icon next to the host in "Configuration"->"Hosts" in the GUI stays green.
6. Go to "Monitoring"->"Latest Data" and verify that information still comes as it should.
7. Watch the value from the zabbix[host,agent,available] key. It will possibly start returning "0" even though the "Configuration"->"Hosts" screen will show the "Z" icon as green.
8. After verifying that the zabbix[host,agent,available] key is returning the wrong value, restart the Zabbix server process.
9. Notice that the zabbix[host,agent,available] key starts returning the correct value.
10. I followed this same process on roughly 50+ servers. At least half of them had this problem occur. The only way to fix it was to restart the Zabbix server process.

The second way that I ran into this is described below:

1. Create a host that is monitored by the Zabbix server and make sure values are received correctly (especially zabbix[host,agent,available] should equal "1"). In my case, the host was created through auto-registration. I then added the appropriate templates to it.
2. Shut the monitored host completely off by doing a hard shut down (using the power button) or disconnect the host from the network.
3. Go to "Monitoring">"Latest Data" and watch the value from the zabbix[host,agent,available] key. It will possibly continue returning "1" even though the "Configuration">"Hosts" screen will show the "Z" icon as red.
4. After verifying that the zabbix[host,agent,available] key is returning the wrong value, restart the Zabbix server process.
5. Notice that the zabbix[host,agent,available] key starts returning the correct value.



 Comments   
Comment by Corey Shaw [ 2012 May 19 ]

I continue to see this problem constantly in 2.0.0rc4. I just had 6 servers go offline and none of them are showing up in Zabbix with problems for zabbix[host,agent,available] and zabbix[host,jmx,available]. The value still comes in as "1" for both keys even though "Configuration"->"Hosts" shows the Zabbix and JMX interfaces as being down.

Comment by Andrew Howell [ 2012 May 19 ]

I have the same problem. It's fine for agents monitored directly via the server, but via proxies doesn't work. I reported this a while ago in ZBX-4684

Comment by Corey Shaw [ 2012 May 19 ]

Nice to know I'm not alone in this. I assume this bug could be marked as a duplicate. This is definitely a show stopper for us. We were going to switch from Nagios to Zabbix for all our monitoring, but obviously can't do that if we can't even tell when a host goes offline.

Comment by Darryl Baker [ 2013 Jan 15 ]

I raised the logging to 4 and the agent (2.0.3) reports ZBX_NOTSUPPORTED back to the server(2.0.3). No proxies involved. Log excerpt below. This is from a Windows 2003 R2 Enterprise 32 bit server with PAE.

1156:20130114:152530.365 In collect_perfstat()
1156:20130114:152531.365 In collect_perfstat()
1156:20130114:152532.365 In collect_perfstat()
3676:20130114:152533.271 Processing request.
3676:20130114:152533.271 Requested [zabbix[host,agent,available]]
3676:20130114:152533.271 Sending back [ZBX_NOTSUPPORTED]
1156:20130114:152533.365 In collect_perfstat()
3676:20130114:152534.303 Processing request.
3676:20130114:152534.303 Requested [system.cpu.load[,avg15]]
3676:20130114:152534.303 Sending back [0.464286]
1156:20130114:152534.365 In collect_perfstat()

Comment by richlv [ 2013 Jan 15 ]

you are using incorrect item type. https://www.zabbix.com/documentation/2.0/manual/config/items/itemtypes/internal

Comment by Darryl Baker [ 2013 Jan 15 ]

I am just following the documentation:

zabbix[host,<type>,available]
Returns availability of a particular type of checks on the host. Value of this item corresponds to availability icons in the host list. Valid types are: agent, snmp, ipmi, jmx.

Return value: 0 (not available), 1 (available), 2 (unknown).

This item is supported starting from version 2.0.0.

Comment by richlv [ 2013 Jan 15 ]

sure, but if that ends up on the agent, your item type is "zabbix agent" (or active). it must be "zabbix internal"

Comment by Darryl Baker [ 2013 Jan 15 ]

I really don't understand what you mean by that statement.

The document says this is how you set up the test through the GUI. If it needs to be converted into a different form to query the agent shouldn't the server or the GUI do the translation?

Comment by richlv [ 2013 Feb 08 ]

what is selected in the "type" field for the item ?

Comment by Darryl Baker [ 2013 Feb 08 ]

The type field is agent.

zabbix[host,agent,available]

Comment by richlv [ 2013 Feb 08 ]

no, item properties, "type" dropdown

Comment by Darryl Baker [ 2013 Feb 08 ]

"Zabbix agent" does it need to be "Zabbix agent (active)"? Is this a case of IBM? Idiot behind machine. I never thought to look for another type than "Zabbix agent".

Comment by richlv [ 2013 Feb 08 ]

it should be "zabbix internal", as noted in my comment on january 15th.

also see https://www.zabbix.com/documentation/2.0/manual/config/items/itemtypes/internal

Comment by Maxim Tyukov [ 2013 Feb 12 ]

Have the same bug, item key and type are correct. Zabbix version 2.0.2.

Comment by richlv [ 2013 Feb 12 ]

could somebody who have this problem create a new database with only single enabled host, single zabbix agent passive item and zabbix[host,agent,available] item, then run zabbix server with debuglevel4 and attach the resulting logfile (once the problem shows up) ?

we just can't reproduce it so far...

Comment by Leonid [ 2013 Feb 13 ]

I have this problem. Some comments for log.

testing - zabbix server
rdq-test - zabbix proxy
tkernel - host

There are 2 items:
agent.ping and zabbix[host,agent,available]
And 1 trigger for them:
{tkernel:agent.ping.nodata({$MAX_UNREACHABLE_TIME})}=1 &

{tkernel:zabbix[host,agent,available].last(0)}

#1

{$MAX_UNREACHABLE_TIME} = 180

Comment by richlv [ 2013 Feb 13 ]

err, this issue is about a single item only, please do not add other items or triggers

Comment by Leonid [ 2013 Feb 14 ]

zabbix[host,agent,available] is based on host.available field (2 - host.available).
host.available is changed by agent.ping item. So you will see no changes to zabbix[host,agent,available] trigger without adding agent.ping value.
Macros and trigger may be redundant, but they don't affect the problem.

Comment by Leonid [ 2013 Feb 18 ]

zabbix patch

Comment by Leonid [ 2013 Feb 18 ]

When zabbix server process host availability (process_host_availability()), it doesn't update cache - only zabbix database. So all subsequent checks (zabbix[host,<type>,available]) get data from not updated cache. It's a reason why zabbix[host,<type>,available] return wrong value.

This patch forces zabbix server to update cache (DCconfig_deactivate_host()/DCconfig_activate_host()).
How to apply:
1. Change dir to zabbix-2.0.5 (it also should work with 2.0.4)
2. patch -p1 < /PATH_TO/zabbix_patch

Comment by Andris Zeila [ 2013 Feb 25 ]

Fixed in development branch svn://svn.zabbix.com/branches/dev/ZBX-4991

Comment by dimir [ 2013 Mar 11 ]

(1) Compilation warnings:

proxy.c: In function ‘process_host_availability’:
proxy.c:1192: warning: implicit declaration of function ‘DCconfig_update_host_availability’
[...]
poller.c: In function ‘activate_host’:
poller.c:309: warning: implicit declaration of function ‘DCconfig_update_host_availability’

dimir more after the fix:

proxy.c: In function ‘process_host_availability’:
proxy.c:1065: warning: ‘available’ may be used uninitialized in this function

dimir fixed in r34254, agreed with wiper, CLOSED

Comment by dimir [ 2013 Mar 11 ]

Successfully tested.

Comment by Andris Zeila [ 2013 Mar 12 ]

Fixed in:
pre-2.1.0 r34272
pre-2.0.6rc1 r34271

Comment by dimir [ 2013 Jun 05 ]

This fix introduced a regression ZBX-6610.

Comment by Cristian Mammoli [ 2013 Oct 24 ]

This bug still affects me from time to time, both with agent and snmp checks.
Agents, Proxies and Server are 2.0.8.

[host,agent/snmp,available] returns 0 even if I the server is up until I restart zabbix_server

Any hint on how to troubleshoot this?

Generated at Wed Apr 24 19:03:21 EEST 2024 using Jira 9.12.4#9120004-sha1:625303b708afdb767e17cb2838290c41888e9ff0.