[ZBXNEXT-2972] Prediction validation and forecasting improvements Created: 2015 Sep 18 Updated: 2017 Sep 29 |
|
Status: | Open |
Project: | ZABBIX FEATURE REQUESTS |
Component/s: | Server (S) |
Affects Version/s: | None |
Fix Version/s: | None |
Type: | New Feature Request | Priority: | Trivial |
Reporter: | Glebs Ivanovskis (Inactive) | Assignee: | Unassigned |
Resolution: | Unresolved | Votes: | 4 |
Labels: | prediction, server, triggerfunctions | ||
Remaining Estimate: | Not Specified | ||
Time Spent: | Not Specified | ||
Original Estimate: | Not Specified |
Description |
Forecast validation trigger function would allow to choose best fit function or optimal interval "on the fly" or could be used to skip unreliable predictions and avoid "false positive" alerts. Here I'd like to summarize ideas and resources on the ways how prediction validation can be done and how forecast() and timeleft() trigger functions may be improved. Validation methods and criteria:
Book on forecasting: Best fit selection:
"Break" detection: More sophisticated forecasting algorithms:
A survey of online failure prediction methods: Predicting Resource Exhaustion with Double Exponential Smoothing |
Comments |
Comment by David Lang [ 2016 Jan 05 ] |
rrdtool implements the Holt-Winters Time Series Forecasting Algorithm for this purpose. It lets you define additional lines on your graph for expected value and expected value +- X standard deviations. It then lets you do comparisons of the actual value with these calculated expected values and take action based on the result This lets you do something like "Alert if the actual value is > 2SD away from the expected value" which can alert for 'traffic too high' at 3am on sunday for the same traffic level that generates a 'traffic too low' at 9am on monday. It 'learns' the patterns. As I understand it, after ~10 cycles of the pattern it will be pretty close to accurate. so after a couple weeks you can rely on it's daily pattern, after a couple months it will notice weekends with great accuracy, etc. some useful links for this the original usenix paper and slides info in it's implementation in rrdtool (also under GPLv2 so code can be copied directly) http://cricket.sourceforge.net/aberrant/rrd_hw.htm |
Comment by David Lang [ 2016 Jan 05 ] |
I think the most important thing is to introduce the concept of the forcast/trend prediction function and tracking. Once the concept is in Zabbix, then implementing additional forecast types is much easier. As I note in The other option is to calculate the forecast as needed (for display, for trigger evaluation, etc). I suspect that for all but the most trivial forecast algorithms, it's going to be better to sacrifice the space to store the pre-computed data instead of computing it each time it's referenced. I suspect that it's also probably less disruptive to the Zabbix codebase to pre-compute the data and have it available as 'just another item' than it is to do the computations on demand. |
Comment by richlv [ 2016 Jan 05 ] |
david@lang.hm, just to be sure, have you looked at |
Comment by David Lang [ 2016 Jan 06 ] |
I got to that after posting the comments above. I added comments to the short version (and this summary may be better than what I posted) there are going to be more types of calculations, the forecast and timeleft functions won't work if the calculation type needs different/additional parameters. we need the ability to graph the prediction over time, as such, I think it makes sense to have a way to define a new item as being calculated from an existing one so that it can be graphed. predictive values have both expected value and confidence level, so they are a compound value, not a single numeric value (for simple trends like forecast() and timeleft() currently support, confidence level isn't very meaningful, but for something like Holt-Winters the ability to plot (or trigger on) expected value +- 1SD is extremely valueable. |