[ZBX-14898] Zabbix server stops with error Created: 2018 Sep 24  Updated: 2018 Sep 26  Resolved: 2018 Sep 24

Status: Closed
Project: ZABBIX BUGS AND ISSUES
Component/s: Server (S)
Affects Version/s: 3.4.14
Fix Version/s: None

Type: Incident report Priority: Critical
Reporter: SuNova Assignee: Unassigned
Resolution: Duplicate Votes: 0
Labels: crash
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

3.10.0-862.11.6.el7.x86_64 #1 SMP Tue Aug 14 21:49:04 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

Attachments: Zip Archive zabbix_log.zip    
Issue Links:
Duplicate
duplicates ZBX-7665 Zabbix 2.2.1 ODBC monitor crash - in ... Closed

 Description   

Steps to reproduce:

  1. Just upgraded to 3.4.14_1.el7

Result:
Logs and memory dump is attached

 

 Comments   
Comment by SuNova [ 2018 Sep 24 ]

It is resolved as an Environment variable "LD_LIBRARY_PATH" required for Oracle ODBC client lib files where deleted from zabbix-server.service after the update, and ODBC driver was failing somehow. Please consider retouching this file instead of rewriting it in update packages. Issue can be closed.

Comment by Vladislavs Sokurenko [ 2018 Sep 24 ]

Thank you for your report, does moving ODBC checks to Zabbix proxy help ? Are you using postresql both as backend and as a driver for unixODBC ?

Comment by SuNova [ 2018 Sep 24 ]

Thanks for caring. We were using some sort of proxy called DBforBIX before, but because it requires adding one extra host for each database it wasn't fit for our goal. But adding a proxy later is in our plans for sure...

And we use Postgres for backend, but use UnixODBC for monitoring our Oracle databases (zabbix database itself is not being monitored)

Comment by Alexander Vladishev [ 2018 Sep 24 ]

Backtrace for easier searching:

 17359:20180924:163140.272 === Backtrace: ===
 17359:20180924:163140.297 20: /usr/sbin/zabbix_server: poller #2 [got 49 values in 0.517615 sec, getting values](zbx_log_fatal_info+0x13c) [0x55555797a530]
 17359:20180924:163140.297 19: /usr/sbin/zabbix_server: poller #2 [got 49 values in 0.517615 sec, getting values](+0xe9945) [0x55555797a945]
 17359:20180924:163140.297 18: /lib64/libc.so.6(+0x362f0) [0x7f82df4b72f0]
 17359:20180924:163140.297 17: /lib64/libclntsh.so.18.1(kpuhhaloc+0x3b8) [0x7f827c4692a8]
 17359:20180924:163140.297 16: /lib64/libclntsh.so.18.1(kpughndl0+0x204) [0x7f827c4c5ec4]
 17359:20180924:163140.297 15: /usr/lib/oracle/18.3/client64/lib/libsqora.so.18.1(bcoSQLAllocEnv+0x136) [0x7f82863e33f6]
 17359:20180924:163140.297 14: /usr/lib/oracle/18.3/client64/lib/libsqora.so.18.1(+0x7593b) [0x7f828644193b]
 17359:20180924:163140.297 13: /lib64/libodbc.so.2(+0xd8dc) [0x7f82e1c058dc]
 17359:20180924:163140.297 12: /lib64/libodbc.so.2(SQLConnect+0x1a7) [0x7f82e1c07af7]
 17359:20180924:163140.297 11: /usr/sbin/zabbix_server: poller #2 [got 49 values in 0.517615 sec, getting values](odbc_DBconnect+0x24d) [0x5555579d2575]
 17359:20180924:163140.297 10: /usr/sbin/zabbix_server: poller #2 [got 49 values in 0.517615 sec, getting values](+0x60c40) [0x5555578f1c40]
 17359:20180924:163140.297 9: /usr/sbin/zabbix_server: poller #2 [got 49 values in 0.517615 sec, getting values](get_value_db+0xa9) [0x5555578f1ed1]
 17359:20180924:163140.297 8: /usr/sbin/zabbix_server: poller #2 [got 49 values in 0.517615 sec, getting values](+0x52054) [0x5555578e3054]
 17359:20180924:163140.297 7: /usr/sbin/zabbix_server: poller #2 [got 49 values in 0.517615 sec, getting values](+0x534f4) [0x5555578e44f4]
 17359:20180924:163140.297 6: /usr/sbin/zabbix_server: poller #2 [got 49 values in 0.517615 sec, getting values](poller_thread+0x1aa) [0x5555578e51f5]
 17359:20180924:163140.297 5: /usr/sbin/zabbix_server: poller #2 [got 49 values in 0.517615 sec, getting values](zbx_thread_start+0x37) [0x555557987937]
 17359:20180924:163140.297 4: /usr/sbin/zabbix_server: poller #2 [got 49 values in 0.517615 sec, getting values](MAIN_ZABBIX_ENTRY+0x898) [0x5555578d1685]
 17359:20180924:163140.297 3: /usr/sbin/zabbix_server: poller #2 [got 49 values in 0.517615 sec, getting values](daemon_start+0x325) [0x555557979c76]
 17359:20180924:163140.297 2: /usr/sbin/zabbix_server: poller #2 [got 49 values in 0.517615 sec, getting values](main+0x312) [0x5555578d0deb]
 17359:20180924:163140.297 1: /lib64/libc.so.6(__libc_start_main+0xf5) [0x7f82df4a3445]
 17359:20180924:163140.297 0: /usr/sbin/zabbix_server: poller #2 [got 49 values in 0.517615 sec, getting values](+0x344b9) [0x5555578c54b9]
Comment by Alexander Vladishev [ 2018 Sep 24 ]

Seems like it is an old and known problem. It is also documented here

Closing as duplicate of .ZBX-7665.

Comment by Glebs Ivanovskis [ 2018 Sep 26 ]

This is neither a duplicate of ZBX-7665 nor is it documented as known issue. This is a "normal" behaviour of Oracle's ODBC driver (or should I say Oracle's DB client library) when environment variables are not set properly – it simply crashes. Same will happen with isql utility or any other application. There is no link between this issue and library conflict causing ZBX-7665.

Comment by Vladislavs Sokurenko [ 2018 Sep 26 ]

I think it's duplicate of ZBX-10928, though there could be many reasons for unixODBC library crashing and we should probably not attempt to categorize them as duplicates and simply close as this is not zabbix issue.

Comment by Glebs Ivanovskis [ 2018 Sep 26 ]

I agree that it is more likely a duplicate of ZBX-10928 and an older ZBX-9009. Same invalid address is being dereferenced:

 17359:20180924:163140.272 Got signal [signal:11(SIGSEGV),reason:1,refaddr:0x68]. Crashing ...

vso, I believe all ODBC issues that are not Zabbix issues should be mentioned in Known issues.

Generated at Fri Apr 19 15:33:23 EEST 2024 using Jira 9.12.4#9120004-sha1:625303b708afdb767e17cb2838290c41888e9ff0.