[ZBX-15788] crash in poller in substitute_lld_macros Created: 2019 Mar 07  Updated: 2024 Apr 10  Resolved: 2019 Mar 17

Status: Closed
Project: ZABBIX BUGS AND ISSUES
Component/s: Server (S)
Affects Version/s: 3.0.25, 4.0.5, 4.2.0beta2, 4.2 (plan)
Fix Version/s: 3.0.26rc1, 4.0.6rc1, 4.2.0rc1, 4.2 (plan)

Type: Problem report Priority: Blocker
Reporter: Oleksii Zagorskyi Assignee: Vladislavs Sokurenko
Resolution: Fixed Votes: 0
Labels: crash
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: PNG File Problematic_prototype_item_2019-03-07_15-22-37.png     File zabbix_server.objdump_output.txt.gz     File zabbix_server.tar.gz     XML File zbx_export_hosts.xml    
Issue Links:
Causes
caused by ZBX-9458 discovery macros are not quoted corre... Closed
Duplicate
Team: Team A
Sprint: Sprint 50 (Mar 2019)
Story Points: 0.125

 Description   

Steps to reproduce:

Create calculated item for network interfaces on sub-template that is connected to 100 templates that are connect with 1300+ hosts.

Everything works when I make "soft" unlink of template on one host and then create the same item on hosts LLD. But when I create the same item on production template - Zabbix crashes.

Please check screenshot of problematic item prototype:

Cleaned debug with level 4:

  3729:20190307:151923.143 End of substitute_lld_macros():SUCCEED data:'GigabitEthernet0/1: Speed (user input)'
  3729:20190307:151923.143 In substitute_key_macros() data:'ifHighSpeed_calc[{#MVSNMPVALUE1}_index{#SNMPINDEX}]'
  3729:20190307:151923.143 In substitute_lld_macros() data:'{#MVSNMPVALUE1}_index{#SNMPINDEX}'
  3729:20190307:151923.143 End of substitute_lld_macros():SUCCEED data:'GigabitEthernet0/1_index10101'
  3729:20190307:151923.143 End of substitute_key_macros():SUCCEED data:'ifHighSpeed_calc[GigabitEthernet0/1_index10101]'
  3729:20190307:151923.143 In substitute_lld_macros() data:'6630'
  3729:20190307:151923.143 End of substitute_lld_macros():SUCCEED data:'6630'
  3729:20190307:151923.143 In substitute_lld_macros() data:'7d'
  3729:20190307:151923.144 End of substitute_lld_macros():SUCCEED data:'7d'
  3729:20190307:151923.144 In substitute_lld_macros() data:'0'
  3729:20190307:151923.144 End of substitute_lld_macros():SUCCEED data:'0'
  3729:20190307:151923.144 In substitute_lld_macros() data:'bps'
  3729:20190307:151923.144 End of substitute_lld_macros():SUCCEED data:'bps'
  3729:20190307:151923.144 In substitute_formula_macros()
  3729:20190307:151923.144 In substitute_lld_macros() data:'(
({$PORT_SPEED_MB:"{#MVSNMPVALUE1}"}=0)

*

last('
  3729:20190307:151923.144 In substitute_lld_macros() data:'{#MVSNMPVALUE1}'
  3729:20190307:151923.144 End of substitute_lld_macros():SUCCEED data:'GigabitEthernet0/1'
  3729:20190307:151923.144 End of substitute_lld_macros():SUCCEED data:'(
({$PORT_SPEED_MB:"GigabitEthernet0/1"}=0)

*

last('
  3729:20190307:151923.144 In substitute_function_lld_param()
  3729:20190307:151923.144 In substitute_key_macros() data:'ifHighSpeed[{#MVSNMPVALUE1}_index{#SNMPINDEX}]'
  3729:20190307:151923.144 In substitute_lld_macros() data:'{#MVSNMPVALUE1}_index{#SNMPINDEX}'
  3729:20190307:151923.144 End of substitute_lld_macros():SUCCEED data:'GigabitEthernet0/1_index10101'
  3729:20190307:151923.144 End of substitute_key_macros():SUCCEED data:'ifHighSpeed[GigabitEthernet0/1_index10101]'
  3729:20190307:151923.144 In substitute_lld_macros() data:'0'
  3729:20190307:151923.144 End of substitute_lld_macros():SUCCEED data:'0'
  3729:20190307:151923.144 End of substitute_function_lld_param()
  3729:20190307:151923.144 In substitute_lld_macros() data:'
)

+

(
({$PORT_SPEED_MB:"{#MVSNMPVALUE1}"}<>0)

*

({$PORT_SPEED_MB:"{#MVSNMPVALUE1}"}*1000000)
)'
  3729:20190307:151923.144 In substitute_lld_macros() data:'{#MVSNMPVALUE1}'
  3729:20190307:151923.144 End of substitute_lld_macros():SUCCEED data:'GigabitEthernet0/1'
*** Error in `/usr/local/sbin/zabbix_server: poller #9 [got 0 values in 0.000000 sec, getting values]': realloc(): invalid next size: 0x0000000001345240 ***
======= Backtrace: =========
/lib64/libc.so.6(+0x7f5e4)[0x7f5f8cb7c5e4]
/lib64/libc.so.6(+0x84c39)[0x7f5f8cb81c39]
/lib64/libc.so.6(realloc+0x1b2)[0x7f5f8cb82eb2]
/usr/local/sbin/zabbix_server: poller #9 [got 0 values in 0.000000 sec, getting values](zbx_realloc2+0x3a)[0x4ac34a]
/usr/local/sbin/zabbix_server: poller #9 [got 0 values in 0.000000 sec, getting values](zbx_replace_string+0xcf)[0x4b2fdf]
/usr/local/sbin/zabbix_server: poller #9 [got 0 values in 0.000000 sec, getting values](substitute_lld_macros+0x4b2)[0x473df2]
/usr/local/sbin/zabbix_server: poller #9 [got 0 values in 0.000000 sec, getting values][0x4e8efe]
/usr/local/sbin/zabbix_server: poller #9 [got 0 values in 0.000000 sec, getting values][0x4ea56c]
/usr/local/sbin/zabbix_server: poller #9 [got 0 values in 0.000000 sec, getting values](lld_update_items+0xd9e)[0x4eb38e]
/usr/local/sbin/zabbix_server: poller #9 [got 0 values in 0.000000 sec, getting values](lld_process_discovery_rule+0xe9f)[0x4e3b4f]
/usr/local/sbin/zabbix_server: poller #9 [got 0 values in 0.000000 sec, getting values](zbx_preprocess_item_value+0x73)[0x45b363]
/usr/local/sbin/zabbix_server: poller #9 [got 0 values in 0.000000 sec, getting values][0x42e847]
/usr/local/sbin/zabbix_server: poller #9 [got 0 values in 0.000000 sec, getting values](poller_thread+0xd5)[0x42ec85]
/usr/local/sbin/zabbix_server: poller #9 [got 0 values in 0.000000 sec, getting values](zbx_thread_start+0x3e)[0x4a791e]
/usr/local/sbin/zabbix_server: poller #9 [got 0 values in 0.000000 sec, getting values](MAIN_ZABBIX_ENTRY+0x6a0)[0x420ef0]
/usr/local/sbin/zabbix_server: poller #9 [got 0 values in 0.000000 sec, getting values](daemon_start+0x1a5)[0x49e495]
/usr/local/sbin/zabbix_server: poller #9 [got 0 values in 0.000000 sec, getting values](main+0x3f3)[0x420063]
/lib64/libc.so.6(__libc_start_main+0xf5)[0x7f5f8cb1f445]
/usr/local/sbin/zabbix_server: poller #9 [got 0 values in 0.000000 sec, getting values][0x4203cb]
======= Memory map: ========
...
  3646:20190307:151923.195 One child process died (PID:3729,exitcode/signal:6). Exiting ...
  3646:20190307:151923.195 zbx_on_exit() called
...
  3646:20190307:151924.336 Zabbix Server stopped. Zabbix 4.0.5 (revision 90164).


 Comments   
Comment by Andris Mednis [ 2019 Mar 08 ]

Update interval: 6630  - is that ok ?

zalex_ua let's skip this point (if it's not related) for now and focus on the crash.

Comment by Vladislavs Sokurenko [ 2019 Mar 09 ]

Does fixing this expression help ?

'
)

+

(
({$PORT_SPEED_MB:"{#MVSNMPVALUE1}"}<>0)

*

({$PORT_SPEED_MB:"{#MVSNMPVALUE1}"}*1000000)
)'

zalex_ua It looks like identical to the debug log. What is the idea? Check please.

Comment by Andris Mednis [ 2019 Mar 11 ]

From crash dump investigation:
Starting from lld_update_items():

int     lld_update_items(zbx_uint64_t hostid, zbx_uint64_t lld_ruleid, const zbx_vector_ptr_t *lld_rows, char **error,
                int lifetime, int lastcheck)
{
        const char              *__function_name = "lld_update_items";

        zbx_vector_ptr_t        applications, application_prototypes, items, item_prototypes, item_dependencies;
        zbx_hashset_t           applications_index, items_index, items_applications;
        int                     ret = SUCCEED, host_record_is_locked = 0;

        zabbix_log(LOG_LEVEL_DEBUG, "In %s()", __function_name);

        zbx_vector_ptr_create(&item_prototypes);

        lld_item_prototypes_get(lld_ruleid, &item_prototypes);

        if (0 == item_prototypes.values_num)
                goto out;

        zbx_vector_ptr_create(&application_prototypes);

        lld_application_prototypes_get(lld_ruleid, &application_prototypes);

        zbx_vector_ptr_create(&applications);
        zbx_hashset_create(&applications_index, application_prototypes.values_num * lld_rows->values_num,
                        lld_application_index_hash_func, lld_application_index_compare_func);

        zbx_vector_ptr_create(&items);
        zbx_hashset_create(&items_index, item_prototypes.values_num * lld_rows->values_num, lld_item_index_hash_func,
                        lld_item_index_compare_func);

        zbx_hashset_create(&items_applications, 100, lld_item_application_hash_func, lld_item_application_compare_func);

        lld_applications_get(lld_ruleid, &applications);
        lld_applications_make(&application_prototypes, lld_rows, &applications, &applications_index);
        lld_applications_validate(hostid, lld_ruleid, &applications, &applications_index, error);

        lld_item_application_prototypes_get(&item_prototypes, &application_prototypes);

        lld_items_get(&item_prototypes, &items);
        lld_items_make(&item_prototypes, lld_rows, &items, &items_index, error);   <---- It goes here.
...	

lld_items_make():

static void     lld_items_make(const zbx_vector_ptr_t *item_prototypes, const zbx_vector_ptr_t *lld_rows,
                zbx_vector_ptr_t *items, zbx_hashset_t *items_index, char **error)
{
        const char                      *__function_name = "lld_items_make";
        int                             i, j, index;
        zbx_lld_item_prototype_t        *item_prototype;
        zbx_lld_item_t                  *item;
        zbx_lld_row_t                   *lld_row;
        zbx_lld_item_index_t            *item_index, item_index_local;
        char                            *buffer = NULL;

        zabbix_log(LOG_LEVEL_DEBUG, "In %s()", __function_name);

        /* create the items index */
        for (i = 0; i < item_prototypes->values_num; i++)
        {
                item_prototype = (zbx_lld_item_prototype_t *)item_prototypes->values[i];

                for (j = 0; j < lld_rows->values_num; j++)
                        zbx_vector_ptr_append(&item_prototype->lld_rows, lld_rows->values[j]);
        }

        /* Iterate in reverse order because usually the items are created in the same order as     */
        /* incoming lld rows. Iterating in reverse optimizes lld_row removal from item prototypes. */
        for (i = items->values_num - 1; i >= 0; i--)
        {
                item = (zbx_lld_item_t *)items->values[i];

                if (FAIL == (index = zbx_vector_ptr_bsearch(item_prototypes, &item->parent_itemid,
                                ZBX_DEFAULT_UINT64_PTR_COMPARE_FUNC)))
                {
                        THIS_SHOULD_NEVER_HAPPEN;
                        continue;
                }

                item_prototype = (zbx_lld_item_prototype_t *)item_prototypes->values[index];

                for (j = item_prototype->lld_rows.values_num - 1; j >= 0; j--)
                {
                        lld_row = (zbx_lld_row_t *)item_prototype->lld_rows.values[j];

                        buffer = zbx_strdup(buffer, item->key_proto);

                        if (SUCCEED != substitute_key_macros(&buffer, NULL, NULL, &lld_row->jp_row, MACRO_TYPE_ITEM_KEY,
                                        NULL, 0))
                        {
                                continue;
                        }

                        if (0 == strcmp(item->key, buffer))
                        {
                                item_index_local.parent_itemid = item->parent_itemid;
                                item_index_local.lld_row = lld_row;
                                item_index_local.item = item;
                                zbx_hashset_insert(items_index, &item_index_local, sizeof(item_index_local));

                                zbx_vector_ptr_remove_noorder(&item_prototype->lld_rows, j);
                                break;
                        }
                }
        }

        zbx_free(buffer);

        /* update/create discovered items */
        for (i = 0; i < item_prototypes->values_num; i++)
        {
                item_prototype = (zbx_lld_item_prototype_t *)item_prototypes->values[i];
                item_index_local.parent_itemid = item_prototype->itemid;

                for (j = 0; j < lld_rows->values_num; j++)
                {
                        item_index_local.lld_row = (zbx_lld_row_t *)lld_rows->values[j];

                        if (NULL == (item_index = (zbx_lld_item_index_t *)zbx_hashset_search(items_index, &item_index_local)))
                        {
                                if (NULL != (item = lld_item_make(item_prototype, item_index_local.lld_row, error)))	       <---- It goes here.
...

lld_item_make():

static zbx_lld_item_t   *lld_item_make(const zbx_lld_item_prototype_t *item_prototype, const zbx_lld_row_t *lld_row,
                char **error)
{
        const char                      *__function_name = "lld_item_make";

        zbx_lld_item_t                  *item;
        const struct zbx_json_parse     *jp_row = (struct zbx_json_parse *)&lld_row->jp_row;
        char                            err[MAX_STRING_LEN];
        int                             ret;

        zabbix_log(LOG_LEVEL_DEBUG, "In %s()", __function_name);

        item = (zbx_lld_item_t *)zbx_malloc(NULL, sizeof(zbx_lld_item_t));

        item->itemid = 0;
        item->parent_itemid = item_prototype->itemid;
        item->lastcheck = 0;
        item->ts_delete = 0;
        item->type = item_prototype->type;
        item->key_proto = NULL;
        item->master_itemid = item_prototype->master_itemid;

        item->name = zbx_strdup(NULL, item_prototype->name);
        item->name_proto = NULL;
        substitute_lld_macros(&item->name, jp_row, ZBX_MACRO_ANY, NULL, 0);
        zbx_lrtrim(item->name, ZBX_WHITESPACE);

        item->key = zbx_strdup(NULL, item_prototype->key);
        item->key_orig = NULL;
        ret = substitute_key_macros(&item->key, NULL, NULL, jp_row, MACRO_TYPE_ITEM_KEY, err, sizeof(err));

        item->delay = zbx_strdup(NULL, item_prototype->delay);
        item->delay_orig = NULL;
        substitute_lld_macros(&item->delay, jp_row, ZBX_MACRO_ANY, NULL, 0);
        zbx_lrtrim(item->delay, ZBX_WHITESPACE);

        item->history = zbx_strdup(NULL, item_prototype->history);
        item->history_orig = NULL;
        substitute_lld_macros(&item->history, jp_row, ZBX_MACRO_ANY, NULL, 0);
        zbx_lrtrim(item->history, ZBX_WHITESPACE);

        item->trends = zbx_strdup(NULL, item_prototype->trends);
        item->trends_orig = NULL;
        substitute_lld_macros(&item->trends, jp_row, ZBX_MACRO_ANY, NULL, 0);
        zbx_lrtrim(item->trends, ZBX_WHITESPACE);

        item->units = zbx_strdup(NULL, item_prototype->units);
        item->units_orig = NULL;
        substitute_lld_macros(&item->units, jp_row, ZBX_MACRO_ANY, NULL, 0);
        zbx_lrtrim(item->units, ZBX_WHITESPACE);

        item->params = zbx_strdup(NULL, item_prototype->params);
        item->params_orig = NULL;

        if (ITEM_TYPE_CALCULATED == item_prototype->type)
        {
                if (SUCCEED == ret)
                        ret = substitute_formula_macros(&item->params, jp_row, err, sizeof(err));     <---- It goes here.
...

substitute_formula_macros():

static int      substitute_formula_macros(char **data, const struct zbx_json_parse *jp_row,
                char *error, size_t max_error_len)
{
        const char      *__function_name = "substitute_formula_macros";

        char            *exp, *tmp, *e;
        size_t          exp_alloc = 128, exp_offset = 0, tmp_alloc = 128, tmp_offset = 0, f_pos, par_l, par_r;
        int             ret = FAIL;

        zabbix_log(LOG_LEVEL_DEBUG, "In %s()", __function_name);

        exp = (char *)zbx_malloc(NULL, exp_alloc);
        tmp = (char *)zbx_malloc(NULL, tmp_alloc);

        for (e = *data; SUCCEED == zbx_function_find(e, &f_pos, &par_l, &par_r, error, max_error_len); e += par_r + 1)
        {
                /* substitute LLD macros in the part of the string preceding function parameters */

                zbx_strncpy_alloc(&tmp, &tmp_alloc, &tmp_offset, e, par_l + 1);
                if (SUCCEED != substitute_lld_macros(&tmp, jp_row, ZBX_MACRO_NUMERIC, error, max_error_len))
                        goto out;

                tmp_offset = strlen(tmp);
                zbx_strncpy_alloc(&exp, &exp_alloc, &exp_offset, tmp, tmp_offset);

                if (++tmp_offset > tmp_alloc)
                        tmp_alloc = tmp_offset;

                tmp_offset = 0;

                /* substitute LLD macros in function parameters */

                if (SUCCEED != substitute_function_lld_param(e + par_l + 1, par_r - (par_l + 1), 1,
                                &exp, &exp_alloc, &exp_offset, jp_row, error, max_error_len))
                {
                        goto out;
                }

                zbx_strcpy_alloc(&exp, &exp_alloc, &exp_offset, ")");
        }

        if (par_l > par_r)
                goto out;

        /* substitute LLD macros in the remaining part */

        zbx_strcpy_alloc(&tmp, &tmp_alloc, &tmp_offset, e);
        if (SUCCEED != substitute_lld_macros(&tmp, jp_row, ZBX_MACRO_NUMERIC, error, max_error_len))		<---- It goes here.
...

substitute_lld_macros():

int     substitute_lld_macros(char **data, const struct zbx_json_parse *jp_row, int flags, char *error,
                size_t max_error_len)
{
        const char      *__function_name = "substitute_lld_macros";

        int             ret = SUCCEED, pos = 0;
        zbx_token_t     token;

        zabbix_log(LOG_LEVEL_DEBUG, "In %s() data:'%s'", __function_name, *data);

        while (SUCCEED == ret && SUCCEED == zbx_token_find(*data, pos, &token, ZBX_TOKEN_SEARCH_BASIC))
        {
                if (0 != (token.type & flags))
                {
                        switch (token.type)
                        {
                                case ZBX_TOKEN_LLD_MACRO:
                                case ZBX_TOKEN_LLD_FUNC_MACRO:
                                        ret = process_lld_macro_token(data, &token, flags, jp_row, error,
                                                        max_error_len);
                                        pos = token.loc.r;
                                        break;
                                case ZBX_TOKEN_USER_MACRO:
                                        process_user_macro_token(data, &token, jp_row);	    <---- It goes here.
...

process_user_macro_token():

static void     process_user_macro_token(char **data, zbx_token_t *token, const struct zbx_json_parse *jp_row)
{
        int                     force_quote;
        size_t                  context_r;
        char                    *context, *context_esc;
        zbx_token_user_macro_t  *macro = &token->data.user_macro;

        /* user macro without context, nothing to replace */
        if (0 == token->data.user_macro.context.l)
                return;

        force_quote = ('"' == (*data)[macro->context.l]);
        context = zbx_user_macro_unquote_context_dyn(*data + macro->context.l, macro->context.r - macro->context.l + 1);

        /* substitute_lld_macros() can't fail with ZBX_TOKEN_LLD_MACRO or ZBX_TOKEN_LLD_FUNC_MACRO flags set */
        substitute_lld_macros(&context, jp_row, ZBX_TOKEN_LLD_MACRO | ZBX_TOKEN_LLD_FUNC_MACRO, NULL, 0);

        context_esc = zbx_user_macro_quote_context_dyn(context, force_quote);

        context_r = macro->context.r;
        zbx_replace_string(data, macro->context.l, &context_r, context_esc);	<---- It goes here.
...

zbx_replace_string():

void    zbx_replace_string(char **data, size_t l, size_t *r, const char *value)
{
        size_t  sz_data, sz_block, sz_value;
        char    *src, *dst;

        sz_value = strlen(value);	<---- length of replacing string
        sz_block = *r - l + 1;		<---- length of block where replacement goes into

        if (sz_value != sz_block)	<---- Replacement block does not fit.
        {
                sz_data = *r + strlen(*data + *r);	<---- Calculate the new lenght after replacement.
                sz_data += sz_value - sz_block;

                if (sz_value > sz_block)	   <---- sz_value > sz_block
                        *data = (char *)zbx_realloc(*data, sz_data + 1);   <---- zbx_realloc() is called.
...
}

zbx_realloc2():

void    *zbx_realloc2(const char *filename, int line, void *old, size_t size)
{
        int     max_attempts;
        void    *ptr = NULL;

        for (
                max_attempts = 10, size = MAX(size, 1);
                0 < max_attempts && NULL == ptr;
                ptr = realloc(old, size), max_attempts--	<---- Crash in realloc() with "invalid next size: 0x0000000001345240".
        );
....
Comment by Vladislavs Sokurenko [ 2019 Mar 11 ]

Reproduced:
Create calculated item with the following formula, see zbx_export_hosts.xml

(
({$PORT_SPEED_MB:"{#MVSNMPVALUE1}"}=0)

*

last("ifHighSpeed[{#MVSNMPVALUE1}_index{#SNMPINDEX}]",0)
)

+

(
({$PORT_SPEED_MB:"{#MVSNMPVALUE1}"}<>0)

*

({$PORT_SPEED_MB:"{#MVSNMPVALUE1}"}*1000000)
)

Send

./bin/zabbix_sender -v -z 127.0.0.1 -p 10051 -s "Zabbix server" -k "lld_rule" -o '{"data":[{"{#MVSNMPVALUE1}":"GigabitEthernet0/1", "{#SNMPINDEX}":"10101", "{#FOO}":"b"}]}'

See valgrind complaining:

==338175== Invalid write of size 1
==338175==    at 0x85042A: zbx_strncpy_alloc (str.c:324)
==338175==    by 0x850A47: zbx_strcpy_alloc (str.c:353)
==338175==    by 0x672783: substitute_formula_macros (lld_item.c:1815)
==338175==    by 0x67770C: lld_item_update (lld_item.c:2131)
==338175==    by 0x67C0B7: lld_items_make (lld_item.c:2464)
==338175==    by 0x6908C7: lld_update_items (lld_item.c:5191)
==338175==    by 0x63B2CA: lld_process_discovery_rule (lld.c:679)
==338175==    by 0x636AA5: lld_process_task (lld_worker.c:89)
==338175==    by 0x637287: lld_worker_thread (lld_worker.c:243)
==338175==    by 0x83413D: zbx_thread_start (threads.c:132)
==338175==    by 0x423F3A: MAIN_ZABBIX_ENTRY (server.c:1223)
==338175==    by 0x8036F2: daemon_start (daemon.c:392)
==338175==  Address 0x6e1a03b is 0 bytes after a block of size 59 alloc'd
==338175==    at 0x483AD19: realloc (vg_replace_malloc.c:836)
==338175==    by 0x840A15: zbx_realloc2 (misc.c:550)
==338175==    by 0x85663B: zbx_replace_string (str.c:2407)
==338175==    by 0x72ACC5: process_user_macro_token (expression.c:5375)
==338175==    by 0x72B2AA: substitute_lld_macros (expression.c:5475)
==338175==    by 0x6725C5: substitute_formula_macros (lld_item.c:1785)
==338175==    by 0x67770C: lld_item_update (lld_item.c:2131)
==338175==    by 0x67C0B7: lld_items_make (lld_item.c:2464)
==338175==    by 0x6908C7: lld_update_items (lld_item.c:5191)
==338175==    by 0x63B2CA: lld_process_discovery_rule (lld.c:679)
==338175==    by 0x636AA5: lld_process_task (lld_worker.c:89)
==338175==    by 0x637287: lld_worker_thread (lld_worker.c:243)
==338175== 
==338175== Invalid write of size 1
==338175==    at 0x8504FB: zbx_strncpy_alloc (str.c:328)
==338175==    by 0x850A47: zbx_strcpy_alloc (str.c:353)
==338175==    by 0x672783: substitute_formula_macros (lld_item.c:1815)
==338175==    by 0x67770C: lld_item_update (lld_item.c:2131)
==338175==    by 0x67C0B7: lld_items_make (lld_item.c:2464)
==338175==    by 0x6908C7: lld_update_items (lld_item.c:5191)
==338175==    by 0x63B2CA: lld_process_discovery_rule (lld.c:679)
==338175==    by 0x636AA5: lld_process_task (lld_worker.c:89)
==338175==    by 0x637287: lld_worker_thread (lld_worker.c:243)
==338175==    by 0x83413D: zbx_thread_start (threads.c:132)
==338175==    by 0x423F3A: MAIN_ZABBIX_ENTRY (server.c:1223)
==338175==    by 0x8036F2: daemon_start (daemon.c:392)
==338175==  Address 0x6e1a06e is 18 bytes before a block of size 49 free'd
==338175==    at 0x4839A0C: free (vg_replace_malloc.c:540)
==338175==    by 0x72BC76: substitute_function_lld_param (expression.c:5698)
==338175==    by 0x6726C3: substitute_formula_macros (lld_item.c:1801)
==338175==    by 0x67770C: lld_item_update (lld_item.c:2131)
==338175==    by 0x67C0B7: lld_items_make (lld_item.c:2464)
==338175==    by 0x6908C7: lld_update_items (lld_item.c:5191)
==338175==    by 0x63B2CA: lld_process_discovery_rule (lld.c:679)
==338175==    by 0x636AA5: lld_process_task (lld_worker.c:89)
==338175==    by 0x637287: lld_worker_thread (lld_worker.c:243)
==338175==    by 0x83413D: zbx_thread_start (threads.c:132)
==338175==    by 0x423F3A: MAIN_ZABBIX_ENTRY (server.c:1223)
==338175==    by 0x8036F2: daemon_start (daemon.c:392)
==338175==  Block was alloc'd at
==338175==    at 0x483880B: malloc (vg_replace_malloc.c:309)
==338175==    by 0x84093B: zbx_malloc2 (misc.c:518)
==338175==    by 0x8587A9: zbx_function_param_unquote_dyn (str.c:2887)
==338175==    by 0x72BB71: substitute_function_lld_param (expression.c:5681)
==338175==    by 0x6726C3: substitute_formula_macros (lld_item.c:1801)
==338175==    by 0x67770C: lld_item_update (lld_item.c:2131)
==338175==    by 0x67C0B7: lld_items_make (lld_item.c:2464)
==338175==    by 0x6908C7: lld_update_items (lld_item.c:5191)
==338175==    by 0x63B2CA: lld_process_discovery_rule (lld.c:679)
==338175==    by 0x636AA5: lld_process_task (lld_worker.c:89)
==338175==    by 0x637287: lld_worker_thread (lld_worker.c:243)
==338175==    by 0x83413D: zbx_thread_start (threads.c:132)
==338175== 
==338175== Invalid read of size 1
==338175==    at 0x483BC34: strlen (vg_replace_strmem.c:461)
==338175==    by 0x8565D7: zbx_replace_string (str.c:2403)
==338175==    by 0x72ACC5: process_user_macro_token (expression.c:5375)
==338175==    by 0x72B2AA: substitute_lld_macros (expression.c:5475)
==338175==    by 0x6727B6: substitute_formula_macros (lld_item.c:1816)
==338175==    by 0x67770C: lld_item_update (lld_item.c:2131)
==338175==    by 0x67C0B7: lld_items_make (lld_item.c:2464)
==338175==    by 0x6908C7: lld_update_items (lld_item.c:5191)
==338175==    by 0x63B2CA: lld_process_discovery_rule (lld.c:679)
==338175==    by 0x636AA5: lld_process_task (lld_worker.c:89)
==338175==    by 0x637287: lld_worker_thread (lld_worker.c:243)
==338175==    by 0x83413D: zbx_thread_start (threads.c:132)
==338175==  Address 0x6e1a03b is 0 bytes after a block of size 59 alloc'd
==338175==    at 0x483AD19: realloc (vg_replace_malloc.c:836)
==338175==    by 0x840A15: zbx_realloc2 (misc.c:550)
==338175==    by 0x85663B: zbx_replace_string (str.c:2407)
==338175==    by 0x72ACC5: process_user_macro_token (expression.c:5375)
==338175==    by 0x72B2AA: substitute_lld_macros (expression.c:5475)
==338175==    by 0x6725C5: substitute_formula_macros (lld_item.c:1785)
==338175==    by 0x67770C: lld_item_update (lld_item.c:2131)
==338175==    by 0x67C0B7: lld_items_make (lld_item.c:2464)
==338175==    by 0x6908C7: lld_update_items (lld_item.c:5191)
==338175==    by 0x63B2CA: lld_process_discovery_rule (lld.c:679)
==338175==    by 0x636AA5: lld_process_task (lld_worker.c:89)
==338175==    by 0x637287: lld_worker_thread (lld_worker.c:243)
==338175== 
==338175== Conditional jump or move depends on uninitialised value(s)
==338175==    at 0x483B8C9: index (vg_replace_strmem.c:252)
==338175==    by 0x85C207: zbx_token_find (str.c:3795)
==338175==    by 0x72B400: substitute_lld_macros (expression.c:5462)
==338175==    by 0x6727B6: substitute_formula_macros (lld_item.c:1816)
==338175==    by 0x67770C: lld_item_update (lld_item.c:2131)
==338175==    by 0x67C0B7: lld_items_make (lld_item.c:2464)
==338175==    by 0x6908C7: lld_update_items (lld_item.c:5191)
==338175==    by 0x63B2CA: lld_process_discovery_rule (lld.c:679)
==338175==    by 0x636AA5: lld_process_task (lld_worker.c:89)
==338175==    by 0x637287: lld_worker_thread (lld_worker.c:243)
==338175==    by 0x83413D: zbx_thread_start (threads.c:132)
==338175==    by 0x423F3A: MAIN_ZABBIX_ENTRY (server.c:1223)
==338175==  Uninitialised value was created by a heap allocation
==338175==    at 0x483AD19: realloc (vg_replace_malloc.c:836)
==338175==    by 0x840A15: zbx_realloc2 (misc.c:550)
==338175==    by 0x85663B: zbx_replace_string (str.c:2407)
==338175==    by 0x72ACC5: process_user_macro_token (expression.c:5375)
==338175==    by 0x72B2AA: substitute_lld_macros (expression.c:5475)
==338175==    by 0x6727B6: substitute_formula_macros (lld_item.c:1816)
==338175==    by 0x67770C: lld_item_update (lld_item.c:2131)
==338175==    by 0x67C0B7: lld_items_make (lld_item.c:2464)
==338175==    by 0x6908C7: lld_update_items (lld_item.c:5191)
==338175==    by 0x63B2CA: lld_process_discovery_rule (lld.c:679)
==338175==    by 0x636AA5: lld_process_task (lld_worker.c:89)
==338175==    by 0x637287: lld_worker_thread (lld_worker.c:243)
==338175== 
==338175== Conditional jump or move depends on uninitialised value(s)
==338175==    at 0x483B8CD: index (vg_replace_strmem.c:252)
==338175==    by 0x85C207: zbx_token_find (str.c:3795)
==338175==    by 0x72B400: substitute_lld_macros (expression.c:5462)
==338175==    by 0x6727B6: substitute_formula_macros (lld_item.c:1816)
==338175==    by 0x67770C: lld_item_update (lld_item.c:2131)
==338175==    by 0x67C0B7: lld_items_make (lld_item.c:2464)
==338175==    by 0x6908C7: lld_update_items (lld_item.c:5191)
==338175==    by 0x63B2CA: lld_process_discovery_rule (lld.c:679)
==338175==    by 0x636AA5: lld_process_task (lld_worker.c:89)
==338175==    by 0x637287: lld_worker_thread (lld_worker.c:243)
==338175==    by 0x83413D: zbx_thread_start (threads.c:132)
==338175==    by 0x423F3A: MAIN_ZABBIX_ENTRY (server.c:1223)
==338175==  Uninitialised value was created by a heap allocation
==338175==    at 0x483AD19: realloc (vg_replace_malloc.c:836)
==338175==    by 0x840A15: zbx_realloc2 (misc.c:550)
==338175==    by 0x85663B: zbx_replace_string (str.c:2407)
==338175==    by 0x72ACC5: process_user_macro_token (expression.c:5375)
==338175==    by 0x72B2AA: substitute_lld_macros (expression.c:5475)
==338175==    by 0x6727B6: substitute_formula_macros (lld_item.c:1816)
==338175==    by 0x67770C: lld_item_update (lld_item.c:2131)
==338175==    by 0x67C0B7: lld_items_make (lld_item.c:2464)
==338175==    by 0x6908C7: lld_update_items (lld_item.c:5191)
==338175==    by 0x63B2CA: lld_process_discovery_rule (lld.c:679)
==338175==    by 0x636AA5: lld_process_task (lld_worker.c:89)
==338175==    by 0x637287: lld_worker_thread (lld_worker.c:243)
==338175== 
==338175== Conditional jump or move depends on uninitialised value(s)
==338175==    at 0x483BC38: strlen (vg_replace_strmem.c:461)
==338175==    by 0x850A29: zbx_strcpy_alloc (str.c:353)
==338175==    by 0x6727D2: substitute_formula_macros (lld_item.c:1819)
==338175==    by 0x67770C: lld_item_update (lld_item.c:2131)
==338175==    by 0x67C0B7: lld_items_make (lld_item.c:2464)
==338175==    by 0x6908C7: lld_update_items (lld_item.c:5191)
==338175==    by 0x63B2CA: lld_process_discovery_rule (lld.c:679)
==338175==    by 0x636AA5: lld_process_task (lld_worker.c:89)
==338175==    by 0x637287: lld_worker_thread (lld_worker.c:243)
==338175==    by 0x83413D: zbx_thread_start (threads.c:132)
==338175==    by 0x423F3A: MAIN_ZABBIX_ENTRY (server.c:1223)
==338175==    by 0x8036F2: daemon_start (daemon.c:392)
==338175==  Uninitialised value was created by a heap allocation
==338175==    at 0x483AD19: realloc (vg_replace_malloc.c:836)
==338175==    by 0x840A15: zbx_realloc2 (misc.c:550)
==338175==    by 0x85663B: zbx_replace_string (str.c:2407)
==338175==    by 0x72ACC5: process_user_macro_token (expression.c:5375)
==338175==    by 0x72B2AA: substitute_lld_macros (expression.c:5475)
==338175==    by 0x6727B6: substitute_formula_macros (lld_item.c:1816)
==338175==    by 0x67770C: lld_item_update (lld_item.c:2131)
==338175==    by 0x67C0B7: lld_items_make (lld_item.c:2464)
==338175==    by 0x6908C7: lld_update_items (lld_item.c:5191)
==338175==    by 0x63B2CA: lld_process_discovery_rule (lld.c:679)
==338175==    by 0x636AA5: lld_process_task (lld_worker.c:89)
==338175==    by 0x637287: lld_worker_thread (lld_worker.c:243)
==338175== 

Suspicious code:

		tmp_offset = strlen(tmp);
		zbx_strncpy_alloc(&exp, &exp_alloc, &exp_offset, tmp, tmp_offset);

		if (++tmp_offset > tmp_alloc)
			tmp_alloc = tmp_offset;

Possible fix:

Index: src/zabbix_server/lld/lld_item.c
===================================================================
--- src/zabbix_server/lld/lld_item.c	(revision 90837)
+++ src/zabbix_server/lld/lld_item.c	(working copy)
@@ -1791,9 +1791,7 @@
 		tmp_offset = strlen(tmp);
 		zbx_strncpy_alloc(&exp, &exp_alloc, &exp_offset, tmp, tmp_offset);
 
-		if (++tmp_offset > tmp_alloc)
-			tmp_alloc = tmp_offset;
-
+		tmp_alloc = tmp_offset + 1;
 		tmp_offset = 0;
 
 		/* substitute LLD macros in function parameters */
Comment by Glebs Ivanovskis [ 2019 Mar 11 ]

Some say that this error is not about invalid realloc() arguments, but about corrupted heap.

Comment by Andris Mednis [ 2019 Mar 11 ]

Thanks, Gleb, for pointing to that

 

Comment by Vladislavs Sokurenko [ 2019 Mar 11 ]

Fixed in:

  • pre-3.0.26rc1 r90867
  • pre-4.0.6rc1 r90868
  • pre-4.2.0rc1 (trunk) r90870
Generated at Thu Apr 25 19:14:10 EEST 2024 using Jira 9.12.4#9120004-sha1:625303b708afdb767e17cb2838290c41888e9ff0.