[ZBX-23815] Zabbix server crashing in alerter Created: 2023 Dec 10  Updated: 2024 Jan 10  Resolved: 2023 Dec 11

Status: Closed
Project: ZABBIX BUGS AND ISSUES
Component/s: Server (S)
Affects Version/s: 6.4.9
Fix Version/s: None

Type: Problem report Priority: Trivial
Reporter: Audrius Sadlauskas Assignee: Zabbix Support Team
Resolution: Duplicate Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Zabbix server,DB.frontend and proxy'ies on separate AlmaLinux 8 servers.


Issue Links:
Duplicate
duplicates ZBX-23787 processes are crashing in "zbx_replac... Closed
duplicates ZBX-23799 Crash zabbix server after upgrade 6.4.9 Closed

 Description   

Zabbix server started to crashing in alerter  again.

163515:20231210:105735.421 === Backtrace: ===
163515:20231210:105735.422 22: /usr/sbin/zabbix_server: alerter #1 started(zbx_backtrace+0x41) [0x55e1a856f4c1]
163515:20231210:105735.422 21: /usr/sbin/zabbix_server: alerter #1 started(zbx_log_fatal_info+0x4a5) [0x55e1a856fa95]
163515:20231210:105735.422 20: /usr/sbin/zabbix_server: alerter #1 started(+0xc0ad6) [0x55e1a856fad6]
163515:20231210:105735.422 19: /lib64/libpthread.so.0(+0x12cf0) [0x7fe0ee355cf0]
163515:20231210:105735.422 18: /usr/sbin/zabbix_server: alerter #1 started(zbx_replace_invalid_utf8+0x3) [0x55e1a85507f3]
163515:20231210:105735.422 17: /usr/sbin/zabbix_server: alerter #1 started(zbx_http_convert_to_utf8+0xcd) [0x55e1a85f583d]
163515:20231210:105735.422 16: /usr/sbin/zabbix_server: alerter #1 started(+0x289543) [0x55e1a8738543]
163515:20231210:105735.422 15: /usr/sbin/zabbix_server: alerter #1 started(+0x2698b5) [0x55e1a87188b5]
163515:20231210:105735.422 14: /usr/sbin/zabbix_server: alerter #1 started(+0x859c3) [0x55e1a85349c3]
163515:20231210:105735.422 13: /usr/sbin/zabbix_server: alerter #1 started(+0x26aac6) [0x55e1a8719ac6]
163515:20231210:105735.422 12: /usr/sbin/zabbix_server: alerter #1 started(+0x269a85) [0x55e1a8718a85]
163515:20231210:105735.422 11: /usr/sbin/zabbix_server: alerter #1 started(+0x2819e7) [0x55e1a87309e7]
163515:20231210:105735.422 10: /usr/sbin/zabbix_server: alerter #1 started(+0x265fc6) [0x55e1a8714fc6]
163515:20231210:105735.422 9: /usr/sbin/zabbix_server: alerter #1 started(duk_pcall+0x32) [0x55e1a8732782]
163515:20231210:105735.422 8: /usr/sbin/zabbix_server: alerter #1 started(zbx_es_execute+0x211) [0x55e1a8739cf1]
163515:20231210:105735.422 7: /usr/sbin/zabbix_server: alerter #1 started(zbx_alerter_thread+0x266) [0x55e1a87b3646]
163515:20231210:105735.423 6: /usr/sbin/zabbix_server: alerter #1 started(zbx_thread_start+0x27) [0x55e1a86151a7]
163515:20231210:105735.423 5: /usr/sbin/zabbix_server: alerter #1 started(+0x951da) [0x55e1a85441da]
163515:20231210:105735.423 4: /usr/sbin/zabbix_server: alerter #1 started(MAIN_ZABBIX_ENTRY+0xc62) [0x55e1a87bbad2]
163515:20231210:105735.423 3: /usr/sbin/zabbix_server: alerter #1 started(zbx_daemon_start+0xef) [0x55e1a85700ef]
163515:20231210:105735.423 2: /usr/sbin/zabbix_server: alerter #1 started(main+0x234) [0x55e1a8530234]
163515:20231210:105735.423 1: /lib64/libc.so.6(__libc_start_main+0xe5) [0x7fe0ec2ced85]
163515:20231210:105735.423 0: /usr/sbin/zabbix_server: alerter #1 started(_start+0x2e) [0x55e1a8536d2e]
163515:20231210:105735.423 === Memory map: ===

You closed my previous issue

https://support.zabbix.com/browse/ZBX-23785

With what issue I was lucky and identified/found workaround.

Now unable to identify what to disable/remove to stop server from crashing.

Could you please provide instructions how to disable/remove crashing alerts from Database.

As server now is not functional and do not know when next version v6.4.10 will be released in which I hope fix will be provided.

 Comments   
Comment by Audrius Sadlauskas [ 2023 Dec 10 ]

Found how to disable crashing alert myself.

You should put this information to known issues as I guess I'm not alone who is faced this kind of issue with alerter crash.

My workaround:

Connect to zabbix database.

use zabbix;

select count * from alerts WHERE status=0 AND alerttype=0;

Identify first alert - in my case it was again Jira cloud media webhook (alertid=907299).

Set status to failed and ( error I set to 'CANCELEDfromDB' ) :

UPDATE alerts SET status=2,error='CANCELEDfromDB' WHERE alertid=907299 AND status=0 AND alerttype=0;

Disable crashing media.

I hope  version 6.4.10  will include fix for alerter crashes.

And I think should include some loging on which alert crashes.

 

 

Comment by dimir [ 2023 Dec 11 ]

I understand your concern but still, it's a duplicate of ZBX-23787. You can put your instructions as a comment there.

6.4.10 will be released in a couple of days.

Alternatively you can build your own version of 6.4.10rc1 that contains the fix: https://cdn.zabbix.com/zabbix/sources/release_candidates/6.4/zabbix-6.4.10rc1.tar.gz

Generated at Tue May 06 07:29:20 EEST 2025 using Jira 9.12.4#9120004-sha1:625303b708afdb767e17cb2838290c41888e9ff0.