[ZBX-4476] Nodata triggers immediately fire on newly created hosts Created: 2011 Dec 22  Updated: 2017 May 30  Resolved: 2014 Feb 11

Status: Closed
Project: ZABBIX BUGS AND ISSUES
Component/s: Server (S)
Affects Version/s: 1.8.9, 1.9.8 (beta), 2.0.6
Fix Version/s: 2.1.5

Type: Incident report Priority: Major
Reporter: Volker Fröhlich Assignee: Unassigned
Resolution: Fixed Votes: 11
Labels: nodata, triggers
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
is duplicated by ZBX-6134 Trigger included nodata func is evalu... Closed
is duplicated by ZBX-6691 False nodata() triggering for a histo... Closed
is duplicated by ZBX-6767 Host set into state "not monitored" ... Closed

 Description   

Steps to reproduce:

  • Create a template with some item and a nodata trigger on it
  • Create a new host and apply the template

If you're not lucky to get data for that item in the first couple of seconds the nodata trigger will fire.

Expected behaviour:

  • Only fire after the nodata timeout is reached

I tried to work around this kind of problem by extending the trigger expression with something like "... & {bla:min(#1)}" (can't remember right now).

This bug is somewhat related to ZBX-4256, which you can easily reproduce, contrary to what is the closing reason. Just in case you consider disabling a host bad practice, why is it there in the first place?



 Comments   
Comment by alix [ 2011 Dec 22 ]

the bug is there on the versions I run: 1.8.6-1.8.9. it's quite annoying to get false notifications on each added host.

Comment by richlv [ 2011 Dec 22 ]

note that disabling hosts is bad practice if you do that to simulate maintenance. but on new hosts that indeed can be a more serious issue

Comment by alix [ 2012 Jan 07 ]

now I've looked into possible solutions of this, it seems that the only reliable way to avoid false nodata() triggering is to have an item's last update (or creation) time, and compare server time against said time in evalfunc.c's evaluate_NODATA() function.

can mtime field of items table be used for this or it's reserved for some other purpose? it's not very clear, I found some references of mtime in distributed operation stuff, but I'm not quite certain.

if mtime can be used, then we just update it with current server time on item creation/modification in frontend, and add a trivial if block to evaluate_NODATA() on server side.

Comment by Oleksii Zagorskyi [ 2012 Jan 20 ]

connected issue ZBX-4565

Comment by mark fermor [ 2012 Sep 26 ]

We had exactly this problem as well. We added hosts and rely on Zabbix Agent ping in order to check the agent is OK. However on addition of a new host this always triggers, even if we disable the host first or try putting into an automated maintenance mode via API.

The work around for us is to change the trigger as follows (version 1.8.x):-
({TRIGGER.VALUE}=0&{Template:agent.ping.count(600,1,"eq")}>0 & {Template:agent.ping.nodata(300)}>0) | ({TRIGGER.VALUE}=1 & {Template:agent.ping.nodata(300)}>0)

This works as follows..
IF ("currently in OK state" && "at least one successful response in the last 600 seconds" && "last 300 seconds we've got no data")
THEN "Trigger go into PROBLEM state"
IF ("currently in PROBLEM state" && "last 300 seconds we've got no data")
THEN "Continue to stay in trigger state. Once we've had data back, then trigger state will revert to OK"

This works fine, the only obvious problem with this set-up to us was the host wouldn't trigger if there was a time that there was never any data (which you'd certainly want if the host came up but never captured any data because something was broken). For us this wasn't so much of a problem as we just created another item which was a simple tcp check for the Zabbix agent on port 10050 and trigger on that (but not triggering on nodata as the simple check always returns data one way or the other), not a concrete solution but we're feeling its pretty close and works for us. So you may need two items in order to get around the problem and I wouldn't want to have this as a permeant solution (needing to create two items and separate triggers for anything with a nodata check), would much rather it was solved as suggested.

Another work around for us was going to be to use the API to insert data into the history directly for this item, however its unfortunately not possible to falsely insert some data as if the check has run even though it hasn't with the API. This is something that's quite useful so might be worth thinking about why the API is being re-written

Best Regards,
Mark

Comment by Oleksii Zagorskyi [ 2012 Oct 09 ]

Related to ZBXNEXT-388.

Also, some reasons clarified in ZBX-3694

Comment by Oleksii Zagorskyi [ 2012 Oct 09 ]

Probably in version 2.0 (triggers have been added to internal cache) this problem is a bit less noticeable because now server first should refresh its cache and only then a timer process (responsible for time-based functions) will be able to notice this trigger.

In 1.8 the timer process was doing direct queries to the DB (each 30 seconds) since items cache updated more rarely - every 60 seconds (by default) or even more rarely in middle/big installations.

Comment by Tom M. [ 2013 Jul 05 ]

Having it on a nodata(180) trigger for an snmp item that gets polled every 20 seconds. When I add a new host using this templated items/triggers, the trigger fires.

Running zabbix 2.0.6.

Comment by Eino Mäkitalo [ 2013 Jul 05 ]

ZBX-6767 affects version 2.0.6

Comment by Dmitry Samsonov [ 2013 Sep 13 ]

Any new on that issue?
We have 2.0.3 and few nodata triggers, that fires up on every new host added.
And we add them often!

My version of workaround:
regular_item1.nodata(1800)=1 & regular_item2.sum(2400)>2
Both items are updated every 600s.
So IF regular_item1 doesn't collected any data for 1800s AND regular_item2 collected data at least 2 times for past 2400s THEN ALARM!
In that case trigger will never fire up until host is monitored for at least 1800s.

Comment by Andris Zeila [ 2013 Sep 18 ]

Fixed in development branch svn://svn.zabbix.com/branches/dev/ZBX-4476

Now if the item value was not updated since server startup the nodata() function will check the time item was added to configuration cache.

Comment by Dmitry Samsonov [ 2013 Sep 18 ]

Didn't understand the solution, but thank you anyway!Grate job!
Waiting for release.

Comment by Alexander Vladishev [ 2013 Sep 19 ]

(1) The algorithm of calculation of nodata() function shall be without changes.

wiper RESOLVED in r38619

sasha CLOSED

Comment by Alexander Vladishev [ 2013 Sep 19 ]

Successfully tested!

Please review my changes in r38635.

Comment by Andris Zeila [ 2013 Sep 19 ]

Released in:
pre-2.1.5 r38570

Comment by richlv [ 2014 Jan 27 ]

(2) as volter noted, this should be listed on whatsnew

wiper Please review - https://www.zabbix.com/documentation/2.2/manual/introduction/whatsnew220#fixed_nodata_function_calculation
RESOLVED

<richlv> thanks, description was understandable, but i feared it might confuse users - i tried to make it more user-oriented, please review

wiper looks good, CLOSED

Comment by Oleksii Zagorskyi [ 2014 Apr 07 ]

One of additional positive effect of this is:
In zabbix before 2.2 - for example our zabbix server and database server (mysql) located on the same host. After a "power failure/unexpected reboot" the database being recovered for example for 15+ minutes.
During this period zabbix server already running but cannot connect to the database, and trying reconnect every 10 seconds (ZBX-4611).

After the database recovery is complete, zabbix server successfully connected to database and took configuration part.
At this point the most of nodata(~180s-15m) triggers will fire to PROBLEM state because the function parameter is less than "CONFIG_SERVER_STARTUP_TIME + arg1 > now" (used before the change).

In 2.2 it works better! Say, CONFIG_SERVER_STARTUP_TIME will be counted already from a moment when server took its configuration from database (DCget_item_time_added).
So triggers do not fire before items supposedly will be polled according to update intervals (which are logically should be less than the function parameter).

Tested!

Generated at Sat May 03 06:48:37 EEST 2025 using Jira 9.12.4#9120004-sha1:625303b708afdb767e17cb2838290c41888e9ff0.