[ZBX-25543] Agent2 Crash During Log Monitoring Due to Regular Expression Processing (regexp_sub_ex2) Created: 2024 Nov 14  Updated: 2025 Feb 07  Resolved: 2025 Feb 07

Status: Closed
Project: ZABBIX BUGS AND ISSUES
Component/s: Agent (G)
Affects Version/s: 6.0.35
Fix Version/s: 6.0.39rc1, 7.0.10rc1, 7.2.4rc1, 7.4.0alpha1

Type: Problem report Priority: Trivial
Reporter: Kim Jongkwon Assignee: Vladislavs Sokurenko
Resolution: Fixed Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: 11.5h
Original Estimate: Not Specified

Attachments: Text File 0001-.G.-ZBX-25543-added-sanitizing-of-invalid-UTF-8-when.patch     Text File 0001-.G.-ZBX-25543-test-added-logging.patch     Text File agent2_patched.log    
Issue Links:
Duplicate
Sub-task
part of ZBX-25805 UTF-8 Error: Isolated Byte (0x80) in ... Confirmed
Team: Team A
Sprint: S24-W50/51/52/1, S25-W2/3, S25-W4/5, S25-W6/7
Story Points: 1

 Description   

I’m currently investigating this crash issue. I received a report of Zabbix Agent2 crashing with backtrace records, as shown below.

[2024-11-06 10:01:20.750] 2024/11/05 19:22:51.133195 process_logrt() old file list:
[2024-11-06 10:01:20.750] 2024/11/05 19:22:51.133200    nr:0 filename:'/var/log/xxx/log_file.log' mtime:1730802120 size:1292897198 processed_size:1478396 seq:1 copy_of:-1 incomplete:0 dev:64768 ino_hi:0 ino_lo:134912620 md5_block_size:512 first_block_md5:9b3958b59a53233463e8099a801f6b1c last_block_offset:1292896686 last_block_md5:f7c07268b383f942a7cf32d89a1551b1
[2024-11-06 10:01:20.750] 2024/11/05 19:22:51.133206 process_logrt() new file list: (mtime:1730802120 lastlogsize:1478396 start_idx:0)
[2024-11-06 10:01:20.750] 2024/11/05 19:22:51.133212    nr:0 filename:'/var/log/xxx/log_file.log' mtime:1730802120 size:1292897198 processed_size:1478396 seq:0 copy_of:-1 incomplete:0 dev:64768 ino_hi:0 ino_lo:134912620 md5_block_size:512 first_block_md5:9b3958b59a53233463e8099a801f6b1c last_block_offset:1292896686 last_block_md5:f7c07268b383f942a7cf32d89a1551b1
[2024-11-06 10:01:20.761] 2024/11/05 19:22:51.133216 In process_log() filename:'/var/log/xxx/log_file.log' lastlogsize:1478396 mtime:1730802120 seek_offset:1478396
[2024-11-06 10:01:20.761] 2024/11/05 19:22:51.133358 itemid 115537: regexp runtime error: Error occurred while matching regular expression: UTF-8 error: isolated byte with 0x80 bit set
[2024-11-06 10:01:20.761] 2024/11/05 19:22:51.134464 9: /usr/sbin/zabbix_agent2(zbx_backtrace+0x3c) [0xb22eec]
[2024-11-06 10:01:20.761] 2024/11/05 19:22:51.134478 8: /usr/sbin/zabbix_agent2(zbx_log_fatal_info+0x475) [0xb23485]
[2024-11-06 10:01:20.761] 2024/11/05 19:22:51.134482 7: /usr/sbin/zabbix_agent2() [0x9c6ad6]
[2024-11-06 10:01:20.761] 2024/11/05 19:22:51.134489 6: /lib64/libpthread.so.0(+0x12ce0) [0x7f5150516ce0]
[2024-11-06 10:01:20.761] 2024/11/05 19:22:51.134495 5: /lib64/libc.so.6(+0xc818e) [0x7f514fa8a18e]
[2024-11-06 10:01:20.761] 2024/11/05 19:22:51.134498 4: /usr/sbin/zabbix_agent2(regexp_sub_ex2+0x86) [0xb29e56]
[2024-11-06 10:01:20.761] 2024/11/05 19:22:51.134501 3: /usr/sbin/zabbix_agent2(process_log_check+0x17ad) [0xb3c7bd]
[2024-11-06 10:01:20.773] 2024/11/05 19:22:51.134504 2: /usr/sbin/zabbix_agent2() [0x9c6bdc]
[2024-11-06 10:01:20.773] 2024/11/05 19:22:51.134508 1: /usr/sbin/zabbix_agent2(_cgo_6e671d72c461_Cfunc_invoke_process_log_check+0x6c) [0x9c6d1e]
[2024-11-06 10:01:20.773] 2024/11/05 19:22:51.134511 0: /usr/sbin/zabbix_agent2() [0x48e0e8]

The information below indicates that the issue occurred during log and regular expression matching:

regexp runtime error: Error occurred while matching regular expression: UTF-8 error: isolated byte with 0x80 bit set

ZBX-25805 has been created for it.

Zabbix Agent2 version was 6.0.27

$ zabbix_agent2 -V
zabbix_agent2 (Zabbix) 6.0.27

I expected ZBX-22871 to fix this issue, but 6.0.27 was expected to be a fixed version. and the crash still occurred even after upgrading from version 6.0.27 to 6.0.35.

We are investigating further, including the log content and regular expressions.



 Comments   
Comment by Vladislavs Sokurenko [ 2025 Jan 29 ]

Fixed in pull request feature/ZBX-25543-6.0

Comment by Vladislavs Sokurenko [ 2025 Feb 06 ]

Fixed in:

Generated at Sun Apr 20 20:59:12 EEST 2025 using Jira 9.12.4#9120004-sha1:625303b708afdb767e17cb2838290c41888e9ff0.