Recently we discussed it on #zabconf2012 that LUA library (thanks to nelsonab for a proof of concept and even more) could be effectively used for such cases.

I'm not sure about external scripting languages I mentioned (BASH, PHP) - just like as examples, with not very good performance if compare to the LUA library, but still could be useful.

For example we could "replace" multiplier to such preprocessing (just as an example).

Also it could help or even solve issues like ZBXNEXT-300 -> ZBXNEXT-1329

i had a feeling this was already reported, but couldn't find it

somewhat similar issue - using usermacros in the processing/multiplier : ZBXNEXT-873

Lua Master ticket: ZBXNEXT-1444

Ok, I will collect real cases where it could be useful:
http://www.zabbix.com/forum/showthread.php?t=32379

Another real world case where this would be useful is for data returned from SNMP. We have devices that return values like "3.04 dB" and "2.0 dB". The ability to extract just the numeric value (strip off the "dB" portion) and then store that as a numeric value in Zabbix would be great. This would make it possible to define useful Triggers based on thresholds, graphing, and storing long term trend data.

Maybe distantly related to ZBXNEXT-1812

How about implementing this via ZBXNEXT-2651?
When choosing this e.g. in the 'Store value' combo-box, then an additional input field could appear to define the function (and parameters?) that is applied before storing a value.
I could also think of a function list that gets subsequently applied.

Ok, I will collect real cases where it could be useful:

I will too, some more real cases:
1) One more QNAP NAS in forum: https://www.zabbix.com/forum/showthread.php?t=44943
2) Electrolink Orchestra FM-transmitter returns values like "105.00 MHz", "6 KHz", etc

additional input field could appear to define the function

Good idea. Is the same regexp-vehicle already works in "Value extracting from logfiles and more" (http://blog.zabbix.com/zabbix-2-2-features-part-7-value-extracting-from-logfiles-and-more/2272/)

Another use case would be to transform some HEX Date values:

ie: for Cisco License Expiration Date (SNMP VAR: clmgmtLicenseEndDate),

HEX Value returned: 07 E0 01 17 12 00 00 00

Where Year 07 E0 --> 2016
and Month 01 --> 01
and Date: 17 --> 23
etc...

So we could add License Expiration triggers...

G.

Another use case is datetime values returned as non-unix timestamps. Like it happens with some (or maybe all) WMI classes and *Time properties

One more use case is a not-ASCII snmp value which is not UTF8 encoded.
For example, network interfaces description in Russian Windows is CP866 encoded when requested by snmpget.

Would be nice to be able to decode a value correctly for an item.

Use case - aggregated status, represent like summ of powers of two (ommon status of Blankom digital receivers). Example:
Power supply #1 failure = 1 (2^0)
Power supply #2 failure = 2 (2^1)
CAM-slot failure = 4 (2^2)

In case of unplugged power cable (PS1) and hunged CA-module I have Common status, equal to 5 (= 1 + 4 = 2^0 + 2^2).

Use case - representation of IP-addresses in Octal, Hexadecimal or Integer notation (https://en.wikipedia.org/wiki/IPv4#Address_representations), I have waited this usecase in real life for a long time and found it in some identical satellite receivers.

3 use ceses found in jjmartres's github:
https://github.com/jjmartres/Zabbix/blob/master/zbx-scripts/ift.ldmode/ift.ldmode # Script converts binary to decimal
https://github.com/jjmartres/Zabbix/blob/master/zbx-scripts/ift.ldstatus/ift.ldstatus # Script converts binary to decimal
https://github.com/jjmartres/Zabbix/blob/master/zbx-scripts/ift.ldsize/ift.ldsize # Seems, that script converts hex to decimal

Author's topic in zabbix forum: Great template collection

Related request for LLD JSON: ZBXNEXT-3341.

Server side ready for testing in development branch svn://svn.zabbix.com/branches/dev/ZBXNEXT-1443

(1) Unsuccessfull import of data.sql

Error in query (1054): Unknown column 'multiplier' in 'field list'

wiper RESOLVED in r64374

iivs Thanks!
CLOSED

(2) Missing default item preprocessing data on fresh install. The item_preproc table is empty.

wiper RESOLVED in r64385

iivs CLOSED

(3) [S] Server must unescape trim function charater list paramater.

wiper RESOLVED in r64390

wiper CLOSED

(4) [S] item_preproc_multiplier_variant can cause uncontrolled numeric overflow when working with big numbers or multiple steps of multiplication. I suggest to introduce check.

wiper Moved to a separate issue - ZBXNEXT-3633
CLOSED

(5) [I] Database patch must reset unused formula field for non lld rules (flags<>1) to ''.

wiper RESOLVED in r64895

vjaceslavs CLOSED

Frontend ready for testing.

(6) [S] The <type> to decimal conversions works only with string data types. While there might not be real use cases, it would be less confusing if we dropped this requirement and simply converted to string data type before applying <type> to decimal conversion.

wiper RESOLVED in r64690

wiper CLOSED

(6) [S] Results with type AR_TEXT are treated as if they have type AR_STRING. This causes data truncation in dc_local_add_history_str.

wiper the truncation will be moved to dc_history_set_value() function.

wiper For now I simply removed the history cache truncation in r64894. The values are truncated when being inserted with zbx_db_insert_. However I remember sasha was in favour of explicit truncation of database fields, so we will need to discuss it.

wiper It was decided:

1. cut all incoming data to 64K characters
2. explicitly cut all historical data to their corresponding field sizes after preprocessing

RESOLVED in r64960, r64962, r64963

vjaceslavs Thanks!
CLOSED

(7) [F] There is a 20 character limit (maxlength attribute) for regexp pattern in frontend (this limit changes after item is saved - limit is set to 255).
The fact that limit changes after saving makes me think that it's a bug and not a feature, but just in case, consider following scenario: Log file containing multiple lines of text.

...
Active item count: 3 (estimated)
Total item count: 1234 (estimated)
...
Active item count: 12 (counted)
...

Value needed to be extracted is 12. Pattern would be something like that "^Active item count: ([0-9]+) (counted)$" (41 character).

iivs RESOLVED in r64701

vjaceslavs In specification validation rules for Regular expression type is defined as "Two non empty parameters are required". In frontend only second one is required. Empty first param will cause "match all" behavior and replacement will be made for all values.
Also, as far as I know, no trim should be applied to pattern as it may produce incorrect results ("([0-9]+)" and ("([0-9]+) " are not the same patterns).

REOPENED.

iivs RESOLVED in r64722, r64724

vjaceslavs Thanks! For better usability I would suggest to warn user about non-visible characters at the beginning or at the end of regexp pattern, but as far as I know, this will not be introduced now, so
CLOSED

(8) [F] Numeric validator allows setting invalid numeric values:

Value is valid, but as server have a upper limit for numbers defined by data types used, this value will not be used correctly. This causes invalid results with no error from server or frontend.

iivs This will be handled by the new API validator.

Moved to ZBX-3783 (63)

CLOSED

(9) [F] Regular expression params are being concatenated and the result string is checked to be not longer than 255 characters. As new line is used in concatenation, this produces an invalid error message (as if there are multiple errors):

Both fields are limited to 255 characters, but there are no sign of a total limit until I submit the form. I suggest to introduce client-side validation with JS for those fields so user input is limited as he types and not after submitting the form.

iivs No such JS will be introduced for now. This error comes from DB validation and should come from API. The error messages will be handled by the new API validator.

Moved to ZBX-3783 (64)

CLOSED

(10) [F] Unit tests fails.

iivs RESOLVED in r64744

gunarspujats Minor code style changes in r64754

iivs Thanks!
CLOSED

(11) [F] Can't delete last preprocessing if there are more than one

iivs RESOLVED in r64745

gunarspujats CLOSED

(12) [F] There is no limit for preprocessing step count, so max_input_vars limit can be hit resulting invalid results (odd behavior with data truncation and errors while trying to save data). If there is no limit for step count and 300+ steps are considered a valid input, then we should check for larger values of max_input_vars when performing a setup (check of pre-requisites) to make it possible.

iivs This problem exists in other places as well. There already is a separate issue on this topic ZBX-4785

WON'T FIX

(13) [F] In item and item prototype forms field Units is missing if item value type is Numeric (unsigned)

iivs RESOLVED in r64757

gunarspujats CLOSED

(14) [F] Coding style
disc_prototypes.php:

• L334 - break; for default in switch statement could be omitted
• L381 - API output option has obsolete fields multiplier, delta, formula and data_type

iivs RESOLVED in r64746

gunarspujats Thanks, looks OK.

Small code style fix in r64778, r64794

As of jQuery 3.0, .delegate() has been deprecated. It was superseded by the .on() method since jQuery 1.7, so its use was already discouraged.

• include/views/js/configuration.item.massupdate.js.php
• include/views/js/common.item.edit.js.php

REOPENED

iivs RESOLVED in r64804

gunarspujats CLOSED

Extra spaces after define function name in include/defines.inc.php:444

REOPENED

RESOLVED in r64840

iivs Thanks!
CLOSED

(15) [F] Custom multiplier accepts only integers. Previous solution (Use custom multiplier) allowed to use fractions as multiplier value.

iivs RESOLVED in r64758

vjaceslavs CLOSED

(16) [F] Looks like captions are invalid for delta preprocessing types:

ZBX_PREPROC_DELTA_VALUE should have caption "Delta" and ZBX_PREPROC_DELTA_SPEED should have caption "Delta per second".

iivs RESOLVED in r64759

vjaceslavs Thanks! CLOSED

(17) [S] item_preproc_delta_uint64 and item_preproc_delta_float have the same definition (if we ignore names of variables). This causes errors with large numbers as uint64 is treated as double.

wiper RESOLVED in r64908

vjaceslavs Thanks! CLOSED

(18) [S] There is a weird limitation for delta options - item type should be numeric. It is weird because there is no such limitation for multiplication (numeric operation) and that makes it even more confusing.

Error messages make it even worse:

So no string allowed as a value for preprocessing

So no uint64 is allowed as a value for preprocessing

It fact error message is trying to say: "I will not work with your data until you change type of information to numeric". I think there should be casting like in multiplication operation or a different error message that will explain the limitation.

wiper RESOLVED in r64908

vjaceslavs CLOSED

(19) [S] Server forces REG_NEWLINE flag to regular expressions so there are no way to get multiple values from different lines:

...
Active item count: 3 (estimated)
Total item count: 1234 (estimated)
...
Active item count: 12 (counted)
...

Values needed to be extracted are 3 and 12. Pattern could be something like that:

Active item count: ([0-9]+) \(estimated\).*Active item count: ([0-9]+) \(counted\)

And the output could be:

\1 / \2

Currently there are no way to bypass this limitation without altering server code. If this is an intended limitation, then it should be mentioned in specification.

zalex_ua there is a workaround described at ZBXNEXT-968

vjaceslavs as far as I understand this is not the same thing. If i need data from the first line (only if line starts with "Active") and from the line N, then I need to use ^ marker to specify beginning of the string (not the beginning of any inner line). There is meta-character "\A" but it does not work for me. So this is a solution for some cases, but not for all of them.

wiper Right, it should be multiline matching.
RESOLVED in r64966

vjaceslavs CLOSED

(20) Translation string changes.
Strings added:

• Boolean to decimal
• Delta
• Delta per second
• Hexadecimal to decimal
• Item pre-processing is missing parameters: %1$s
• Item pre-processing is not allowed for discovery rules.
• Left trim
• Octal to decimal
• Only one "Delta" step is allowed.
• Preprocessing
• Right trim
• Trim
• first parameter is expected
• second parameter is expected
• list of characters
• number
• output
• pattern

Strings deleted:

• Boolean
• Data type
• Decimal
• Delta (simple change)
• Delta (speed per second)
• Hexadecimal
• Octal
• Store value
• Use custom multiplier

gunarspujats CLOSED

(21) Creation of item prototype without preprocessing throws error

Incorrect value for field "preprocessing": cannot be empty.

iivs RESOLVED in r64776

gunarspujats CLOSED

(22) [S] There should be more consistency on how items are processed - item_preproc_2dec performs extra trimming / data slicing that causes weird behavior:

FF.00

This is a valid hex value that can be converted to decimal 255 with "Hexadecimal to decimal" preprocessing step

FF.01

This is an invalid hex value for "Hexadecimal to decimal" preprocessing step

+""+ " ++"+"-1.0000000000000000000" """" "" "

This is a valid boolean value that can be converted to decimal 1 "Boolean to decimal" preprocessing step

Result of a command vfs.file.contents[/tmp/demo.txt] is being cut off after first 255 bytes (because of (6), but it still would be cut at 2047 bytes in item_preproc_2dec) so demo.txt contents like "+[253 or 2045 (when (6) is fixed) spaces here]FF any junk here" is a valid hex value that can be converted to decimal 255.

Specification mentions trimming when preprocessing is done and item type is numeric, but the same trimmings (and even more) are silently applied within item_preproc_2dec.

wiper Trimming (and decimal removal) occurs whenever string value is converted to numeric. This was done for backwards compatibility. As *2dec perprocessing in essence is value conversion to uint64, the same trimming was applied there. However trying to apply data type conversion for +1 or 1.00 doesn't really make sense, so it was removed.

wiper RESOLVED in r64909, r64913

vjaceslavs Fixed. CLOSED

(23) [A] item.get with random string as parameter for option selectPreprocessing returns unexpected result with extra array

{
    "output": ["name"],
    "itemids": 25587,
    "selectPreprocessing": "asd"
}

"result": [
        {
            "itemid": "25587",
            "name": "Available memory 2",
            "preprocessing": []
        },
        {
            "preprocessing": [
                [],
                [],
                [],
                []
            ]
        }
    ]

iivs RESOLVED in r64810

I noticed same problem in value mappings. Should we fix it separately?

gunarspujats Thanks! Same problem in other methods should be fixed under ZBX-3783 (67)

CLOSED

[A][F] API and Frontend tested

(24) [A] Items must be created with empty formula field. Currently all non-discovery items are created with formula '1'.

sasha RESOLVED in r65011

gunarspujats CLOSED

[S] Server tested

(25) [I] formula field must be set to '' for all items and item prototypes in data.tmpl file.

wiper RESOLVED in r65033

sasha CLOSED

(26) [A] item.get() and itemprototype.get() methods shall not return formula field.

sasha MOVED to ZBX-3783

CLOSED

Released in:

• pre-3.3.0 r65073

(27) [S] Incorrect value conversion from floating to unsigned int.

Steps to repeat:

1. create a trapper item with value type unsigned
2. add custom multiplier preprocessing option
3. send -1 to the item

Preprocessing will convert the received "-1" string to floating value, apply multiplier and then convert to unsigned by a simply type cast, resulting in a huge unsigned int value.

Item preprocessing should return error when trying to convert negative values to unsigned.

RESOLVED in development branch svn://svn.zabbix.com/branches/dev/ZBXNEXT-1443

vjaceslavs CLOSED

[S] Server tested

Released in:

• pre-3.3.0 r65087

You test it before pushing?

in src/libs/zbxsysinfo/sysinfo.c:683

if (SUCCEED != is_uint64(c, &value_uint64)) {
                                SET_UI64_RESULT(result, value_uint64);
                                ret = SUCCEED;
                        }

This set result only when value not uint64.
And in next switch branch

                        if (SUCCEED == is_double(c))
                        {
                                SET_DBL_RESULT(result, atof(c));
                        }

missing ret = SUCCEED;

wiper The set_result_type() function with numeric value type is now only used by vfs.fs.* checks, so unfortunately it did slip past testing. Thank you very much for noticing!

(28) [S] fixed text to numeric value conversions used by threaded metrics.

RESOLVED in development branch svn://svn.zabbix.com/branches/dev/ZBXNEXT-1443

vjaceslavs CLOSED

(29) [F] when copy the items to other hosts or templates, preprocessing not copied
Steps to reproduce:
1. Go to Configuration->Hosts
2. Click on Items for the respective host
3. Select some item (with preprocessing) and press on 'copy' button
4. Choose target

sasha RESOLVED in development branch svn://svn.zabbix.com/branches/dev/ZBXNEXT-1443 r65139

natalja.zabbix ui tested

oleg.egorov CLOSED

Fixed in pre-3.3.0 (trunk) r65150.

Released in:

• pre-3.3.0 r65119

(30) [F] import failed, when importing new items without preprocessing
Steps to reproduce:
1. Create item without preprocessing for host (or template)
2. Export this host
3. Delete this host
4. Import this host and item
Result: Incorrect value for field "preprocessing": cannot be empty.

sasha RESOLVED in development branch svn://svn.zabbix.com/branches/dev/ZBXNEXT-1443 r65138

natalja.zabbix ui tested

oleg.egorov CLOSED

Fixed in pre-3.3.0 (trunk) r65150.

(31) I will be your Coverity, Luke
I believe these warnings are preprocessing-related:

$ CFLAGS='-Wall -Wextra -g -O2' ./configure --enable-server --enable-proxy --enable-agent --with-postgresql

dbupgrade_3030.c: In function ‘DBpatch_3030018_add_numeric_preproc_steps’:
dbupgrade_3030.c:187:17: warning: unused parameter ‘value_type’ [-Wunused-parameter]
   unsigned char value_type, unsigned char data_type, const char *formula, unsigned char delta)
                 ^

item_preproc.c: In function ‘item_preproc_convert_value_to_numeric’:
item_preproc.c:64:65: warning: unused parameter ‘item’ [-Wunused-parameter]
 static int item_preproc_convert_value_to_numeric(const DC_ITEM *item, zbx_variant_t *value_num,
                                                                 ^

dbcache.c: In function ‘dc_add_history’:
dbcache.c:2576:56: warning: unused parameter ‘value_type’ [-Wunused-parameter]
 void dc_add_history(zbx_uint64_t itemid, unsigned char value_type, unsigned char item_flags, AGENT_RESULT *result,
                                                        ^

Plus an outdated comment of dc_add_history().

wiper RESOLVED in r65271. Also fixed unrelated signed/unsigned comparison warnings in r65276.

vjaceslavs CLOSED

(32) Updated documentation:

• Item configuration
• What's new
• Upgrade notes
• XML export examples/tag value description for templates and hosts

RESOLVED

iivs CLOSED

(33) [S] Coverity CID 155653: Uninitialized pointer read

** CID 155653:  Memory - illegal accesses  (UNINIT)
/src/libs/zbxdbcache/dbconfig.c: 4425 in DCsync_item_preproc()
________________________________________________________________________________________________________
    1. var_decl: Declaring variable item without initializer.
4402        ZBX_DC_ITEM             *item;
4403        zbx_dc_item_preproc_t   *preproc;
4404
4405        zabbix_log(LOG_LEVEL_DEBUG, "In %s()", __function_name);
4406
4407        zbx_vector_uint64_create(&syncids);
4408
    2. Condition NULL != (row = DBfetch(result)), taking true branch.
4409        while (NULL != (row = DBfetch(result)))
4410        {
4411                ZBX_STR2UINT64(itemid, row[1]);
4412
    3. Condition itemid != lastitemid, taking false branch.
4413                if (itemid != lastitemid && NULL == (item = zbx_hashset_search(&config->items, &itemid)))
4414                        continue;
4415
4416                ZBX_STR2UINT64(item_preprocid, row[0]);
4417                zbx_vector_uint64_append(&syncids, item_preprocid);
4418
4419                preproc = DCfind_id(&config->item_preproc, item_preprocid, sizeof(zbx_dc_item_preproc_t), &found);
4420
4421                ZBX_STR2UCHAR(preproc->type, row[2]);
4422                DCstrpool_replace(found, &preproc->params, row[3]);
4423
4424                /* cleared in DCsync_items() */
    CID 155648: Dereference after null check (FORWARD_NULL) [select issue]
    CID 155653 (#1 of 1): Uninitialized pointer read (UNINIT)4. uninit_use: Using uninitialized value item.
4425                zbx_vector_ptr_append(&item->preproc_ops, preproc);
4426
4427                lastitemid = itemid;
4428        }
4429

It's false positive, as itemid can't be 0, so item will be initialized further in the loop. However we can rearrange code a bit to avoid relying on the 'hidden' logic.

wiper RESOLVED in r65249

vjaceslavs CLOSED

(34) [S] Coverity CID 155652
Variable value_ui64 can stay uninitialized if operation type none of these values - ZBX_PREPROC_BOOL2DEC, ZBX_PREPROC_OCT2DEC, ZBX_PREPROC_HEX2DEC

In src/libs/zbxdbcache/item_preproc.c:

Declaring variable value_ui64 without initializer.

static int      item_preproc_2dec(zbx_variant_t *value, unsigned char op_type, char **errmsg)
{
       zbx_uint64_t    value_ui64;

Uninitialized scalar variable (UNINIT)4. uninit_use_in_call: Using uninitialized value value_ui64 when calling zbx_variant_set_ui64.

zbx_variant_set_ui64(value, value_ui64);

RESOLVED in r65251

vjaceslavs CLOSED

Well, but where is divide operation ?

Multiplication by a decimals can be used as a divide operation.

Provide equal multiplication for "123.12 / 3.14", or "100/3.14"

I understand what you are trying to tell, but there always will be some data loss if we are talking about decimals with some precision.
I can't tell what precision do you need for your results, but I would say that you can use multiplication by 0.31847134 as a substitute for division by 3.14.

I'm say about consistency too. Triggers have in operations division. preprocessing - not. Totally misleading. And remember about magic numbers where 0.31847134 - is equal 3.14 ^-1, too.

Given that 1/1024 factor can be quite common it's a pity that it must be written in a form of 0.000976562.

The initial set of preprocessing operations was created mostly based on backwards compatibility. There would be no problem in adding more operations (well, except cluttered UI). However it would be better to create a new request instead of cramming too many features into one issue.

Coverity issues/warnings fixed and released in:

• pre-3.3.0 r65564

(35) [S] Trunk does not compile with OpenIPMI. Fix:

Index: src/zabbix_server/ipmi/ipmi_manager.c
===================================================================
--- src/zabbix_server/ipmi/ipmi_manager.c	(revision 65567)
+++ src/zabbix_server/ipmi/ipmi_manager.c	(working copy)
@@ -747,7 +747,7 @@
 				init_result(&result);
 				SET_TEXT_RESULT(&result, value);
 				value = NULL;
-				dc_add_history(itemid, ITEM_VALUE_TYPE_TEXT, 0, &result, &ts, state, NULL);
+				dc_add_history(itemid, 0, &result, &ts, state, NULL);
 				free_result(&result);
 			}
 			break;
@@ -756,7 +756,7 @@
 		case AGENT_ERROR:
 		case CONFIG_ERROR:
 			state = ITEM_STATE_NOTSUPPORTED;
-			dc_add_history(itemid, ITEM_VALUE_TYPE_TEXT, 0, NULL, &ts, state, value);
+			dc_add_history(itemid, 0, NULL, &ts, state, value);
 	}
 
 	dc_flush_history();
@@ -847,7 +847,7 @@
 			int		errcode = CONFIG_ERROR;
 
 			zbx_timespec(&ts);
-			dc_add_history(items[i].itemid, ITEM_VALUE_TYPE_TEXT, 0, NULL, &ts, state, error);
+			dc_add_history(items[i].itemid, 0, NULL, &ts, state, error);
 			DCrequeue_items(&items[i].itemid, &state, &ts.sec, NULL, NULL, &errcode, 1);
 			zbx_free(error);
 			continue;

wiper Moved to ZBXNEXT-3386.
CLOSED

(36) [F] One and the only preprocessing option can be dragged as it has been just created. If I reenter Preprocessing tab or add and then delete preprocessing options leaving just one, it can't be dragged.

iivs RESOLVED in r65713

sasha CLOSED

(37) [F] The only section in Preprocessing tab is called Preprocessing. Aren't there better naming options?

iivs What is your proposal?

iivs Discussed with wiper and martins-v and other name options could be options, operations, rules. And I understood that it's not up to us to decide.

wiper I believe preprocessing will stay.. what preprocessing - options, operations, rules - that's not quite clear.

PavelA it's obvious that these are duplicates and an obvious solution would be to remove duplicates: remove the "Preprocessing" form label

<dimir> "Preprocessing" -> "Preprocessing" is silly. Please do not keep it.

iivs Renamed label to "Preprocessing steps".
RESOLVED in r65983

sasha In Mass update form "Preprocessing" field also should be renamed. REOPENED

iivs RESOLVED in r67323

sasha CLOSED

(38) [F] Consider reordering available preprocessing options in dropdown according to their usage frequency. Ideally it would be nice to have some statistics for this. But even without stats it is clear that Trim is a more common case than more specific Left trim and Right trim.

iivs Statistics? Sorry I don't follow. Are you willing to change order of the list to Trim, Left trim, Right trim?

iivs Currently they are sorted by defines 0, 1, 2, 3 ... Maybe better sort them alphabetically?

glebs.ivanovskis Maybe try to imagine how you would use them? Alphabetic sorting makes little sense but at least it has some.

glebs.ivanovskis In item configuration we have a nice dropdown for interfaces subdivided into interface types. Would be great to have something similar for preprocessing options. Some of them are clearly more useful for numbers, other - for text, etc. Adds a bit of structure and simplifies navigation.

<dimir> Something like:

• Textual
  • Trim
  • Left trim
  • Right trim
• Numeric
  • Delta
  • Delta per second
  • Hexadecimal to decimal
  • Octal to decimal
  • Boolean to decimal

?

PavelA Looks like we have to make consistent and future proof UI decisions vs. exclusive. I think that providing a live filtering/search option inside dropdowns would help users to deal with long dropdown lists in general.

vjaceslavs I suggest to add preprocessing step groups (optgroup) and to reorder items as follows:

iivs RESOLVED in r66044, (r66047, r66051, r66057 - bugfixes)

sasha

1. Have a look at my changes in r67120
2. Under this commit has been refactored setSelected() method. That's wonderful! But calls of this method are not fixed. For example:
   1. frontends/php/include/classes/html/CComboItem.php:24: default value for $selected parameter is not changed to boolean
   2. frontends/php/include/views/configuration.item.edit.php:122, frontends/php/include/views/configuration.item.prototype.edit.php:96, frontends/php/include/views/configuration.host.discovery.edit.php:82, frontends/php/include/views/configuration.item.massupdate.php:49,62: the third parameter should be boolean
   3. Delta group must be renamed into Change

REOPENED

iivs RESOLVED in r67319

sasha CLOSED

Originally came to the surface when finally documenting ZBXNEXT-3353. Value handling by server vs. value handling by proxy has changed, this affects history export via modules functionality. Probably a note in here needs and update. Probably something should be said about it in upgrade notes. But probably not, I haven't heard that someone actually uses this functionality, especially on proxies. This does not qualify as a subissue, I leave it here just for history, in case something breaks for someone it will be easier to find the reason.

(39) [DF] While we are at it, let's rename Speed per second to something more appropriate. It is one of many examples of bad wording and misuse of commonly accepted terminology.

1. Speed (according to Wikipedia) is the magnitude of object's velocity (the rate of change of its position). Rate of change or simply Rate has wider meaning and is more appropriate here.
2. ... per second is redundant because there are no other options (like ... per hour, ... per week, etc.), plus the Speed/Rate itself is the same, no matter in which units it is expressed.
3. Speed per second can be confused with acceleration.

My suggestions are Rate of change, Rate, Change per second. Of course, it should be documented, that the new name in Preprocessing tab does the same as the old one. And that rate is calculated in units per second.

martins-v It would be nice to remove reference to speed. It's not intuitively understood. We use 'change' in trigger functions. Could we use 'Change' and 'Change per second' here, too?

vjaceslavs I agree with martins-v that 'Change per second' would be the best option. Updated screenshot in (38)

glebs.ivanovskis Hm, are there many Greeks among users? How about Delta -> Change?

• Change
  • Simple
  • Per second

vjaceslavs This is not the first time when this option (preprocessing that is called "delta") is present in Zabbix, so I think that we should stick to the previous name as it will be less confusing for existing users.

martins-v If we really want to improve the naming, then I'd support the suggestion by glebs.ivanovskis.

glebs.ivanovskis Also, we can have delta() trigger function which is very different from this Delta.

vjaceslavs Ok, I agree, that 'Change' is better. But option names should stay as they should be valid even if dropdown is not expanded so we can't change names from "Simple change" to "Simple" as after selection user will see the following preprocessing step:

sasha May be should be renamed only "Delta"

Moved to (38) and CLOSED

(40) Additional translation string changes.
Strings added:

• Arithmetic
• Change per second
• Left trim
• Numeral systems
• Preprocessing steps
• Simple change

Strings deleted:

• Delta
• Delta per second
• Left trim 

sasha CLOSED

Has anyone experienced this issue?

(41) [AF] Mass updating something (e.g. update interval) in Templates/Hosts->Items drops all preprocessing options of updated items.

gcalenko RESOLVED in r66810

sasha frontends/php/items.php:823:842: this code should be moved into new if statement

REOPENED

gcalenko RESOLVED in r66923

sasha CLOSED

(42) the contributors in the changelog entry are not alphabetical

sasha Thanks! RESOLVED in r66918

iivs CLOSED

looks like ZBXNEXT-1722 was included in (38)/(39)

(46) [D] API documentation updated:

• Item object, item.create, item.get, item.update
• Item prototype object, itemprototype.create, itemprototype.get, itemprototype.update
• Changes from 3.2 to 3.4

sasha

• formula should be removed from item and itemprototype objects
• formula, delta, data_type and multiplier should be removed from item.get and itemprototype.get examples
• an information about removed fields should be added into "Backward incompatible changes" section

REOPENED

iivs I removed from formula from item and itemprototype objects, but I couldn't find the mentioned fields in item.get and itemprototype.get examples. And seems they have already been removed and backwards incompatible changes were too already resolved by gcalenko.

RESOLVED

sasha Thanks! CLOSED

(47) Additional translation string changes.
Strings added:

• Only one change step is allowed.

Strings deleted:

• Only one "Delta" step is allowed.

iivs CLOSED

Additional issues fixed in pre-3.4.0alpha1 (trunk) r67980

(48) Caused ZBX-12201.

Would be great increase the preprocessing options adding transformation from ºF to ºC and vice versa.
https://support.zabbix.com/browse/ZBXNEXT-4040

I want to convert from mWatt to dBm.
Formula is very complex for preprocessing.
P(dBm) = 10 ⋅ log10( P(mW) / 1mW)

Please advice which is option i can do.

Dear theptais, you may be interested in ZBXNEXT-552 and ZBXNEXT-3884 or ZBXNEXT-3508. Currently such conversion is not possible.

Thanks @Glebs Ivanovskis

Would be nice to add in preprocessing simple conversion for words to numbers.

Example, like: alarm -> 1 or normal -> 0

Consider creating a ZBXNEXT for that (if there is no such).

Hey, I see this is closed, BUT. 

Why wasn't the ability to convert strings to numbers (integers) kept? there are loads of requests for help on this issue all over the forum?

Particularly, people want to evaluate responses from hosts, and since its not possible to do this with strings, people are asking to convert the string or substring to a number value for comparison. 

as an example,  localtime which returns a string in the case of [local] may need to convert to numbers because you can't compare strings from multiple items in triggers. 

My scenario, converting the timezone which comes in as 2018-05-24.124+01:00, stripping down all but the 01 of +01:00 with regex then converting it to an int to be able to compare it with a reference server check? how is this not a real use case? 

ikaxs, convertion is done based on type selected in item configuration.

You can set type to unsigned int and convertion will be done after all preprocessing steps.

Thank you, but given the type of system.localtime(local) which gets a string, with number (unassigned) it returns nothing. 

You should not take initial type into account. Look at it this way:

1. You get a string from an item (let it be system.localtime(local))
2. You apply preprocessing steps (could be any set, but regex in this case). This is the essential part as you will not be able to save date string as unsigned int.

At this point you don't have initial value (date), but you have preprocessed value (01 in your case).

So at this point Zabbix just checks the value type provided for item and performs the transformation.

I may be missing something then, what pre processing do I need to apply to get that value? 

with only the system.localtime(local) and number (unsigned) I get no result at all?

You just said yourself - 'with regex then converting it to an int'. Or more precisely it should be a regex extracting the integer portion out of the original value. So create numeric (unsigned) item system.localtime[local], add regex preprocessing step to extract timezone and you will have the desired result.

If you need to keep the original value then you need to keep system.localtime[local] without preprocessing and add a dependent item with the required type and preprocessing steps.

When a preprocessing step is done the value is converted to the required data type. String for trim, regex functions. Integer for multiplication, delta steps. When all steps are done the resulting value is converted to item's value type. Data conversion failure will result in unsupported item.

Thanks Andris, 

The documentation isn't clear about the order and what happens.

With only the system.localtime(local) and type set as number (unsigned) the result is unsupported, it returns a greyed out entry. 

We've managed to make it work by adding a custom multiplier of 1

That's quite strange, I'm getting the following (tested on latest 3.4 branch):

Ok, thank you, that seems to be working ok now, I had to recreate it. and my regex was different but returned that value. 

If I create a calculated item like this, how do I refer to it in a trigger expression? given that I'm also referencing a template?

I'm trying to reference all the local times of servers in a template, against a particular host, for which I've created a calculated item. 

Can you advise? I keep getting errors when creating a trigger