[ZBX-9639] Negative chart width causes 100% CPU + DoS in frontend Created: 2015 Jun 15  Updated: 2017 May 30  Resolved: 2015 Aug 28

Status: Closed
Project: ZABBIX BUGS AND ISSUES
Component/s: Frontend (F)
Affects Version/s: 2.4.5
Fix Version/s: 2.0.16rc1, 2.2.11rc1, 2.4.7rc1, 3.0.0alpha2

Type: Incident report Priority: Blocker
Reporter: Vincent Huisman Assignee: Unassigned
Resolution: Fixed Votes: 1
Labels: graphs
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Debian and Gentoo, different versions of apache and PHP, different version of Zabbix, but still in 2.4.5


Issue Links:
Duplicate
is duplicated by ZBX-9697 HTTP server DoS, because of insuffici... Closed

 Description   

I have an old Linux-machine with Opera 12, which is the only browser with which I have observed this problem and actively tried to reproduce it. Sometimes when restoring an old browsing session in which graphs are opened, the graphs will not be displayed and the webserver will start to consume 100% CPU instead. Since the graph is re-requested every 30 seconds and Zabbix requires max_execution_time of 300, this will quickly cause problems on any server, when the user does not promptly notice that this issue is going on (for example when the tab with the graph was not in the foreground). I have found out that this is caused by the graph width calculation failing in certain circumstances and system loads, so the graph is requested with a negative graph.

A while ago, I traced the issue back to some piece of code, the frontend kept looping over the same loop indefinitely, trying to increase $y (or something like that) with no apparent progress, but at that moment I could not reproduce the exact input that caused it, so I forgot where I found this code. In any case, the input variable needs to be validated correctly but it might also be useful to modify the relevant loops such that this cannot happen again, even when some input variable was not properly validated.

Steps to reproduce:
Open a graph
Open the graph image itself
In the URL, change width=xxx to width=-1 (or any other negative value)

Expected result:
An image with "Error: Page received incorrect data. Incorrect value "-1" for "width" field", just like for width=0

Actual result:
No image is shown, page keeps on loading "indefinitely", while an apache process on the server is using 100% CPU.



 Comments   
Comment by richlv [ 2015 Jun 17 ]

ouch. reproducible in 2.4.5

Comment by richlv [ 2015 Jul 09 ]

(1) ZBX-9697 suggests such requests can be generated by the frontend, let's figure out how, and fix that as well

gunarspujats CLOSED

<richlv> judging by the comment-152202, we were unable to reproduce that so far. if anybody knows a way to reproduce frontend sending incorrect width parameter, please open a new issue.
still CLOSED

Comment by Gunars Pujats (Inactive) [ 2015 Aug 10 ]

(2) No translation strings changed.

sasha CLOSED

Comment by Gunars Pujats (Inactive) [ 2015 Aug 10 ]

RESOLVED in svn://svn.zabbix.com/branches/dev/ZBX-9639

Comment by Alexander Vladishev [ 2015 Aug 13 ]

(3) Please remove extra validation in the CChart class

gunarspujats RESOLVED in r54917

sasha CLOSED

Comment by Alexander Vladishev [ 2015 Aug 13 ]

(4) the code should be rewritten like this:

if (!check_fields($fields)) {
    exit();
}

gunarspujats RESOLVED in r54917

sasha CLOSED

Comment by Gunars Pujats (Inactive) [ 2015 Aug 13 ]

Updated validation rules for chart width.
RESOLVED in r54922.

sasha CLOSED

Comment by Alexander Vladishev [ 2015 Aug 27 ]

Successfully tested! Please take a look on my changes in r55185. I undid all formatting changes.

gunarspujats CLOSED

Comment by Gunars Pujats (Inactive) [ 2015 Aug 28 ]

Fixed in:

  • pre-2.0.16 r55210
  • pre-2.2.11 r55213
  • pre-2.4.7 r55217
  • pre-3.0.0alpha2 (trunk) r55218
Comment by Gunars Pujats (Inactive) [ 2015 Aug 28 ]

100% CPU load was caused by infinite loop in chart generation function. It happened when chart width was set less than 12 (some magic number used in loop).
As was decided, minimum chart width should be 20, and this number now is used in validations. So, if chart controller will receive width less then 20, error message will be generated.

This error was reproduced as it is described above - opening image itself and changing graph width. Getting graph width less than 12 using frontpage in "normal" way wasn't possible.

Comment by Alexander Vladishev [ 2015 Oct 13 ]

This fix caused a regression ZBX-9948.

Generated at Sat Apr 27 01:37:32 EEST 2024 using Jira 9.12.4#9120004-sha1:625303b708afdb767e17cb2838290c41888e9ff0.