[ZBX-6729] Log files are slow processed by Agent on Windows Created: 2013 Jun 19  Updated: 2017 May 30  Resolved: 2014 Jan 27

Status: Closed
Project: ZABBIX BUGS AND ISSUES
Component/s: Agent (G)
Affects Version/s: 2.0.7rc1, 2.1.0
Fix Version/s: 2.0.12rc1, 2.2.3rc1, 2.3.0

Type: Incident report Priority: Major
Reporter: Alexey Pustovalov Assignee: Unassigned
Resolution: Fixed Votes: 3
Labels: logrt, performance, slow, windows
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate

 Description   

Sometimes Zabbix agent slow process log files because of some of used functions and logic (agent scans directory for each iteration and send only one line from log file) are not quite good.

Some of information which should help improve performance of log monitoring on Windows systems:

1. Windows _findnext:

Unless you have a specific reason to use the versions that specify the time size explicitly, use _findnext or _wfindnext or, if you need to support file sizes greater than 3 GB, use _findnexti64 or _wfindnexti64. All these functions use the 64-bit time type in Visual C++ 2005. In previous versions, these functions used a 32-bit time type. If this is a breaking change for an application, you might define _USE_32BIT_TIME_T to get the old behavior. If _USE_32BIT_TIME_T is defined, _findnext, _finnexti64 and their corresponding Unicode versions use a 32-bit time.

So http://msdn.microsoft.com/en-us/library/6tkkkc1y(v=vs.80).aspx recommend use _wfindnext instead of _findnext for files bigger than 3GB.
2. _wstat64 is used for getting information about stats. We should use _wstati64 instead of _wstat64:

Variations of these functions support 32- or 64-bit time types, and 32- or 64-bit file lengths. The first numerical suffix (32 or 64) indicates the size of the time type used; the second suffix is either i32 or i64, indicating whether the file size is represented as a 32-bit or 64-bit integer.

3. For Windows we can use notification ReadDirectoryChangesW from http://msdn.microsoft.com/en-us/library/aa365261(VS.85).aspx (it is similar inotify for Linux).

Most of interesting is third point. With it we can wait changes of specified directory.



 Comments   
Comment by Andris Mednis [ 2014 Jan 17 ]

I'm working to make log file processing faster. Notification ReadDirectoryChangesW is not yet planned to implement as there are simpler, non platform-specific optimizations to be done first.

Comment by Andris Mednis [ 2014 Jan 27 ]

For Zabbix 2.0 available in development branch svn://svn.zabbix.com/branches/dev/ZBX-6731.

See comments on ZBX-6731.

Comment by Andris Zeila [ 2014 Feb 03 ]

Successfully tested

Comment by Andris Mednis [ 2014 Feb 28 ]

Fixed in versions pre-2.0.12 r43102, pre-2.2.3 r43233, pre-2.3.0 r43239.

Comment by Andris Mednis [ 2014 Mar 06 ]

For 2.0 documented in https://www.zabbix.com/documentation/2.0/manual/introduction/whatsnew2012?&#daemon_improvements
For 2.2 documented in https://www.zabbix.com/documentation/2.2/manual/introduction/whatsnew223?&#daemon_improvements
Also see comments on ZBX-6731 for additional info.

Comment by Andris Mednis [ 2014 Mar 27 ]

Improved documentation at https://www.zabbix.com/documentation/2.2/manual/introduction/whatsnew223?&#daemon_improvements about conditions when log[] and logrt[] items become NOTSUPPORTED.

Comment by Andris Mednis [ 2014 May 07 ]

Improved documentation at https://www.zabbix.com/documentation/2.0/manual/introduction/whatsnew2012?&#daemon_improvements about conditions when logrt[] items become NOTSUPPORTED.

wiper reviewed.

martins-v I've reviewed all documentation related changes in this issue, so the documenting part could be CLOSED.

Comment by Andris Mednis [ 2014 May 22 ]

The fix caused a regression ZBX-8238.

Generated at Thu Apr 25 08:00:50 EEST 2024 using Jira 9.12.4#9120004-sha1:625303b708afdb767e17cb2838290c41888e9ff0.