|
Different version, but backtrace is very similar to one from ZBX-12898:
|
|
Just happened one more time:
|
|
Please try monitoring odbc through proxy as a workaround, on top of that if possible compile proxy without openldap and curl and use different database type than you are monitoring, for example SQLite, see if issue is gone then.
|
|
Is it possible to try Release 12.2.0.1.0 and unixodbc 2.3.1
https://docs.oracle.com/database/122/ODBCR/toc.htm#ODBCR101
Which version did you use?
|
|
hello,
currently we are using Instant Client 12.1:
and unixODBC 2.3.1:
I can try and upgrade the Instant Client and re-enable the query if it helps, right now we disabled the two queries and we haven't got any additional server crash.
thanks
Gabriele
|
|
Hi, gabriele.armao! Nice to hear from you!
Were you able to understand how those two queries were different from the rest? Did they use different driver? Had they been querying different database? Did they use specific data types in the query itself?
At this point everything suggest that this is a problem on Oracle's side, therefore updating the driver may be a good thing to do.
|
|
Hello Glebs,
not really sure about the difference, they were working before Zabbix upgrade (2.4.8 --> 3.4.2) as the crashes started shortly after the update.
They were querying the same database as many other queries on the same Zabbix Server, same driver, I think the only notable differences are that the items are set with a "text" information type and that sometimes, the queries returned an ORA-01489 error because we were using the LISTAGG() function to concatenate multiple-row output into a single row, and sometimes the number of rows to concatenate was too big and the LISTAGG() function returned an error, the database monitor item went into an unsupported state with an ODBC error returned by the LISTAGG() function (ORA-01489). All other items are setup with float or unsigned information type.
|
|
And as far as I understand executing same query does not always cause crashing. My best guess is that it is a result of some destructive interplay between ODBC drivers and libraries they and Zabbix depend upon (linked library versions could have changed with 2.4->3.4 upgrade). Workaround for this situation is isolating problematic queries and monitor them via Zabbix proxy.
|
|
After some time of intensive investigation and patching Zabbix we got this information:
For some reason Oracle's driver decides to write 2862 or 2258 bytes to the buffer we provide despite we are asking for just 2047 (MAX_STRING_LEN minus one) bytes. Interesting thing is that what Oracle writes starts with spaces and the number of spaces equals amount of bytes written by the second call to SQLGetData().
Crashes are due to buggy Oracle ODBC driver. Case closed.
|
|
In 3.4.10 the crash can be fixed with patch
|
|
Oracle ODBC does not corrupt buffer, it returns number of available data bytes in 'len' value in a call
For example, we provide 2048 byte buffer in the call mentioned above, but result is 2245 characters large.
The 1st call sets len=2245, but respects 2048 byte buffer size, too. So, 2047 characters and terminating '\0' is returned in buffer.
The 2nd call sets len=198 and returns remaining characters. <--- That is 2245 - 2047 = 198.
|
|
https://docs.microsoft.com/en-us/sql/odbc/reference/syntax/sqlgetdata-function?view=sql-server-2017 says about SQLGetData() function's last parameter StrLen_or_IndPtr:
So, Oracle ODBC returns "length of the data available" every time. Seems reasonable.
|
|
Similar info in https://www.ibm.com/support/knowledgecenter/SSEPEK_11.0.0/odbc/src/tpc/db2z_fngetdata.html
About 'pcbValue' parameter it says:
Points to the value that indicates the amount of space that the data you are retrieving requires. If the data is retrieved in pieces, this contains the number of bytes still remaining.
|
|
Fixed in development branches:
- svn://svn.zabbix.com/branches/dev/ZBX-13010-22
- svn://svn.zabbix.com/branches/dev/ZBX-13010-30
- svn://svn.zabbix.com/branches/dev/ZBX-13010-40
|
|
Improved after code review patch for 3.4.10 is in attachment
"patch_ZBX-13010_for3_4_10.txt".
|
|
Fixed in versions:
- pre-2.2.23rc1 r82646
- pre-3.0.20rc1 r82649
- pre-3.4.12rc1 r82652
- pre-4.0.0alpha9 (trunk) r82653
|
|
Thanks to glebs.ivanovskis for all investigation and preparation work to resolve this !
|
|
Hm, an unexpected finale... andris, do you have an explanation for this observation:
Interesting thing is that what Oracle writes starts with spaces and the number of spaces equals amount of bytes written by the second call to SQLGetData().
|
|
It could be that this observation comes from using patch:
In this patch the contents of 'buffer' is printed with specification '%*s' where field width is specified at runtime by 'len'.
The 1st call to SQLGetData() sets 'len' to, for example, 2862. So we get up to 2047-bytes string printed as 2862-byte field, padded with spaces on the left.
|
|
Invisible patch, hehe 
Indeed, your explanation makes sense! I was assuming that buffer wasn't null-terminated and it was my mistake.
Thank you for the response!
Looks like + 1 is not needed here:
char buffer[MAX_STRING_LEN + 1];
|
|
Thanks, Gleb ! Your 'odbcmin.c' greatly helped to resolve this case.
Indeed, + 1 is not needed there.
|
Generated at Thu Apr 25 16:29:33 EEST 2024 using Jira 9.12.4#9120004-sha1:625303b708afdb767e17cb2838290c41888e9ff0.
57673_zabbix_proxy_valgrind.zip
bindings_out_MSSQL.txt
Team A