|
Before upgrade to zabbix 2.21, the server was very stable and reliable.
After we performed upgrade from 2.0.9 -> 2.2.1:
Several times a day, zabbix-server crashes with following notification in its logfile:
Can you help me with this issue?
Kind regards
Andreas Niedermann
Armacom AG
|
|
maybe related to https://support.zabbix.com/browse/ZBX-7521
|
|
senfuu, the ZBX-7521 shows a bit different backtrace.
Andreas, you have now to re-compile "Pre-2.2.2rc1 (stable)" form here http://www.zabbix.com/developers.php and confirm is the crash still happens.
Every time please show the crash report.
|
|
Oleksiy, thank you for the reply.
I will compile "Pre-2.2.2rc1" as soon as possible and let you know the result.
Andreas
|
|
Hi Oleksiy, I've tried to launch a compiled pre-2.2.2rc1 version of zabbix-server (compiled using the following configure string):
./configure --enable-server --with-mysql --with-jabber --with-libxml2 --with-net-snmp --with-ssh2 --with-libcurl --with-unixodbc --with-openipmi --with-iconv
The difference between the previous version 2.2.1 (installed by deb package management system) and the newer 2.2.2rc1 (compiled from source) is:
2.2.1 -> crashing after some hours
2.2.2rc1 -> crashing subsequently and directly after starting the binary
The crash report will be attached to this doc. the objdump is split into 2 parts to allow uploading a size over 10m.
Do you suggest more testing or anything we can do?
|
|
newest log from 2.2.2rc1 binary
|
|
part one of objdump using v. 2.2.2.rc1
|
|
part two of objdump using v. 2.2.2.rc1
|
|
From the last Andreas' attached zabbix_server.log:
Indeed, it's right after daemon start.
But currently it's in an escalator process !
One possibly interesting thing which is the same - it happens after calling zbx_strdup2() function.
|
|
Andreas, if it's possible, you could truncate the escalations table and try to start 2.2.2rc1 once again.
But remember that truncating the table will stop all running escalations, so some users may not get expecting alerts.
But in most cases is not a problem.
|
|
The crash happens when a log (or logrt) item triggers an event which is processed by action with
{ITEM.LOG.SOURCE} macro used in subject/message. The {ITEM.LOG.SOURCE}
field is eventlog specific and will be NULL for log, logrt items, resulting in crash.
|
|
The current idea is to substitute eventlog specific fields (eventid, source, timestamp) with * UNKNOWN * value for log,logrt items.
sasha Value for {ITEM.LOG.SEVERITY}, {ITEM.LOG.NSEVERITY}, {ITEM.LOG.EVENTID} and {ITEM.LOG.SOURCE} should be *UNKNOWN* for non eventlog[] items.
Value for {ITEM.LOG.DATE}, {ITEM.LOG.TIME} and {ITEM.LOG.AGE} should be processed for all items with "Type of information" = "Log".
|
|
Fixed in development branch svn://svn.zabbix.com/branches/dev/ZBX-7653
|
|
Here two supposedly different crashes are listed.
Both have been fixed, or only escalator related one?
wiper I believe the history syncer (value cache related) crash has been already fixed in ZBX-7521 (it contained 2 fixes actually).
zalex_ua Thanks. I've just updated issue summary.
Interesting thing why Andreas (issue reporter) experienced the escalator related crash (last one) after the attempt to use pre-2.2.2
Could it be that it was a regression after 2.2.1 release ?
wiper No, it was introduced in 2.2.0
|
|
(1) It is incorrect check for eventlog[] items. eventlog[] values can be without eventid.
Related issue: ZBX-6851
wiper Then only option left is to check item key (i.e. read it directly from the database)
wiper Added check for item key by using configuration cache. RESOLVED in r41776
sasha CLOSED Please review my changes in r41787.
|
|
Released in:
pre-2.2.2rc1 r41806
pre-2.3.0 r41807
|
|
(2) comment for r41776 said :
added that to https://www.zabbix.com/documentation/2.2/manual/introduction/whatsnew222#daemon_improvements - please, review
sasha Thanks! CLOSED.
|
Generated at Fri Apr 26 19:02:28 EEST 2024 using Jira 9.12.4#9120004-sha1:625303b708afdb767e17cb2838290c41888e9ff0.
objdump_zbx_221.tar.bz2
zabbix_server.log