Server: performance improvements
(ZBXNEXT-318)
[ZBXNEXT-325] Parrallel writing of historical data by N processes Created: 2010 Apr 23 Updated: 2017 May 31 Resolved: 2010 Apr 29 |
|
Status: | Closed |
Project: | ZABBIX FEATURE REQUESTS |
Component/s: | None |
Affects Version/s: | None |
Fix Version/s: | 1.8.3 |
Type: | Change Request (Sub-task) | Priority: | Major |
Reporter: | Alexei Vladishev | Assignee: | Unassigned |
Resolution: | Fixed | Votes: | 0 |
Labels: | None | ||
Remaining Estimate: | Not Specified | ||
Time Spent: | Not Specified | ||
Original Estimate: | Not Specified |
Comments |
Comment by Alexei Vladishev [ 2010 Apr 26 ] |
It will be implemented by introducing a new structure, which will keep list of items currently processed by DB Syncers. If an itemid is buing processed, other DBSyncer won't pick it up from the history buffer. |
Comment by Alexander Vladishev [ 2010 Apr 29 ] |
Added new configuration parameter StartDBSyncers. |
Comment by michael chan [ 2012 Oct 11 ] |
How will we know if this is working? Reason I ask, is that I have Zabbix monitoring with a small setup: And the issue is that 27 of these hosts I am actively logging log data (e.g. error.log, access.log, etc.), roughly 2 logs each host, at 10 lines per second. So the theoretical maximum is 540 lines/sec of log(text) data. Also I have a trigger assigned to all the logs - nothing special, just a simple REGEX on each, and performing an alert if it is matched. Now when I do a mysql processlist, I almost always see just 1 thread running, even if DBSyncers is 4, 16, 32. The DBSyncers value to be honest is irrelevant. Now when I increased my lines/sec to 20 lps on all my Items that gather log data, The Zabix text write cache & History write cache start decreasing slowly, basically unable to write the data out to the DB quick enough. I have tried increasing the server size, increase the database memory & # of cpus (I am running all this with an Amazon c1.medium EC2 instance and m1.small instance, and have increased the EC2 instance to a c1.xlarge and RDS to m1.large) and have not seen ANY improvement at all. The larger Amazon instances I have tried each have roughly 4x the memory and 4x the # of cores. Is there any way to see what the issue is? Why are there not more dbsyncers actively writing data and not all sleeping? The Amazon RDS mysql instance has all the defaults which are for the most part set it and forget it, and I have tried tweaking the Zabbix Server parameters and nothing makes any difference. Zabbix acts almost as if it is single-threaded, and is not scaling according to the # of cpus (neither server cpus nor database/RDS cpus). Thanks for any help you can provide. |