See also ZBX-7301

There are two bugs that might/would happen if the last few log files have the same modification time:

• the last few lines from the old log file might be ignored
• the old file might get parsed again, resulting in duplicate item values

Fixed in development branch svn://svn.zabbix.com/branches/dev/ZBX-7098

(1) The fix has been developed for Zabbix 2.2 but 2.0 also must be fixed.

Before fixing ZBX-7098, it was decided to fix "log[]" and "logrt[]" performance issue ZBX-6731, ZBX-6729 and also ZBX-7740. Currently a fix for ZBX-6731 and ZBX-6729 is developed for Zabbix 2.0 (available in development branch svn://svn.zabbix.com/branches/dev/ZBX-6731) and is being ported to Zabbix 2.2. The performance fix is rather large change in log file processing in Zabbix agent. When completed, the work on ZBX-7098 will continue.

The solution turned out to be more complex, than expected.
Currently it is being tested.

Until now we used only file 'mtime' and 'lastlogsize' for following log file changes. We have added using inode numbers and MD5 sums of the first 512-bytes for deciding whether a log[] or logrt[] file checked during the previous check can be counted as the same file and can be checked from 'lastlogsize' position, or it is a different file and should be analyzed from the start.

One consequence of the coming solution - do not modify 'mtime' of log files with 'touch', do not copy the log file with later restoration of the original name (it will change file's inode number). In both cases the file will be counted as different and will be analyzed from the start, which may result in duplicated alerts.

On Microsoft Windows, if log files reside on NTFS or ReFS file systems, we have added 64-bit file indexes or 128-bit file IDs to track log file rotation. On file systems where file indexes change (e.g. FAT32) we have developed a fall-back algorithm to take a sensible approach in uncertain conditions.

For Zabbix 2.0 available in development branch svn://svn.zabbix.com/branches/dev/ZBX-7098-20-1

It is a significant change, need to be thoroughly tested on UNIX, GNU/Linux (with various file systems) and Microsoft Windows (including log files on NTFS, FAT32 (or eXFAT) and ReFS), including complicated log file rotations, truncations, multiple log files with the same modification time, even with identical log files... Try to break it in various ways.

wiper successfully tested

For Zabbix 2.2 available in development branch svn://svn.zabbix.com/branches/dev/ZBX-7098-22-1

wiper successfully tested

For trunk available in development branch svn://svn.zabbix.com/branches/dev/ZBX-7098-23

wiper successfully tested

Successfully tested!

Fixed in versions pre-2.0.13 r45702, pre-2.2.4 r45706, pre-2.3.0 r45718.

Documented at
https://www.zabbix.com/documentation/2.0/manual/config/items/itemtypes/log_items?&#important_notes
https://www.zabbix.com/documentation/2.2/manual/config/items/itemtypes/log_items?&#important_notes
https://www.zabbix.com/documentation/2.4/manual/config/items/itemtypes/log_items?&#important_notes
https://www.zabbix.com/documentation/2.0/manual/introduction/whatsnew2013
https://www.zabbix.com/documentation/2.2/manual/introduction/whatsnew224

wiper technical review done.

martins-v Reviewed as well. Some small grammar points improved. CLOSED.

Solution for ZBX-7098 caused an issue ZBX-8269.