(1) [I] Database patch created in development branch svn://svn.zabbix.com/branches/dev/ZBXNEXT-3277
(and simplified by merging old/new tables in r60846)
RESOLVED

sandis.neilands CLOSED.

Server side is being implemented in development branch svn://svn.zabbix.com/branches/dev/ZBXNEXT-3277-2 (and mostly done). This branch is based on ZBXNEXT-3277 with trigger level correlation changes ported from ZBXNEXT-3274

(2) [I] The dbversion in branch svn://svn.zabbix.com/branches/dev/ZBXNEXT-3277-2 seemed incorrect. I got an error after clean install. Please see changes in r61028

sandis.neilands CLOSED.

(3) [F] Translation string changes:
Strings added:

• Cannot add correlation
• Cannot add correlation condition
• Cannot delete correlation
• Cannot delete selected correlations
• Cannot disable correlation
• Cannot disable correlations
• Cannot enable correlation
• Cannot enable correlations
• Cannot update correlation
• Close new event
• Close old events
• Condition "%2$s" is not used in formula "%3$s" for correlation "%1$s".
• Condition "%2$s" used in formula "%3$s" for correlation "%1$s" is not defined.
• Correlation
• Correlation "%1$s" already exists.
• Correlation added
• Correlation deleted
• Correlation disabled
• Correlations disabled
• Correlation enabled
• Correlations enabled
• Correlation is missing parameters: %1$s
• Correlation name cannot be empty.
• Correlation updated
• Create correlation
• Delete current correlation?
• Delete selected correlations?
• Disable selected correlations?
• Duplicate "%1$s" value "%2$s" for correlation "%3$s".
• Duplicate "%1$s" value "%2$s" for correlation.
• Empty correlation ID.
• Enable selected correlations?
• Event correlation
• Event correlation rules
• Event tag pair
• Generated by
• Group "%1$s" cannot be deleted, because it is used in a correlation condition.
• Incorrect correlation ID.
• Incorrect custom expression "%2$s" for correlation "%1$s": %3$s.
• Incorrect filter condition formula ID given for correlation "%1$s".
• Incorrect type of calculation for correlation "%1$s".
• Incorrect value "%1$s" in field "%2$s" for correlation "%3$s".
• New event host group
• New event tag
• New event tag value
• New operation
• No "%1$s" given for correlation "%2$s".
• No "%1$s" given for correlation.
• No condition type given for correlation "%1$s".
• No operation type given for correlation "%1$s".
• Old event tag
• Old event tag value
• new event tag
• Only super admins can create correlations.
• Only super admins can delete correlations.
• Only super admins can update correlations.
• Selected correlations deleted
• new event tag
• old event tag
• Conditions duplicates for correlation "%1$s".

gunarspujats CLOSED

Frontend and API ready for testing.
Changes implemented in both branches svn://svn.zabbix.com/branches/dev/ZBXNEXT-3277 and svn://svn.zabbix.com/branches/dev/ZBXNEXT-3277-2

(4) [S] process_events() returns garbage value when there are no events (events_num == 0). Similar problem in process_trigger_events().

sandis.neilands RESOLVED in r61050.

wiper CLOSED

Available in pre-3.1.0 (trunk) r61054.

(6) [F] Filter state (open/closed) is not remembered in Configuration -> Event correlation

iivs RESOLVED in r61090

oleg.egorov CLOSED

wiper and merged into trunk r61100

(7) [D] API documenatation updated:

• Correlation
• Correlation object
• correlation.create method
• correlation.delete method
• correlation.get method
• correlation.update method
• Event object new fields in output
• hostgroup.delete description updated
• API changes from 3.0 to 3.2

gunarspujats CLOSED

(8) [F] Event correlation status update issue
Not possible to disable event correlation via update form or to create disabled event correlation.

iivs RESOLVED in r61121

oleg.egorov CLOSED

(9) [F] Create condition "New event host group" and select all host groups and add it, then again select "New event host group" and select all host groups and try again to add it.

Undefined variable: new_condition [ in correlation.php:176]

1. Remove tag
2. Add new
3. Try to save

iivs RESOLVED in r61122

oleg.egorov CLOSED

Created new development branch for bug fixing - ZBXNEXT-3277-4.

(10) [S] Warning when compiling with CFLAGS='-Wall -g -O2':

events.c: In function ‘correlate_events_by_global_rules’:
events.c:1023:39: warning: ‘tag’ may be used uninitialized in this function [-Wmaybe-uninitialized]
    value_esc = DBdyn_escape_string(tag->value);
                                       ^
events.c:982:13: note: ‘tag’ was declared here
  zbx_tag_t *tag;
             ^

Compiler:

$ gcc --version
gcc (Ubuntu 5.4.0-6ubuntu1~16.04.1) 5.4.0 20160609
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

wiper This should have been fixed in ZBXNEXT-3274-2 (merged in ZBXNEXT-3277-4 r61157)
RESOLVED

sandis.neilands CLOSED.

(11) Cosmetic: merge into trunk in r61100 has some extra blank lines in include/dbcache.h and the function name in the comment for zbx_strmatch_condition() is incorrect.

wiper RESOLVED in ZBXNEXT-3274-3 r61226

sandis.neilands CLOSED.

(12) [S] One of the added OK events has a tag, the other one does not (see event_tags.png).

Configuration

dummy_host_0

• dummy_item_0: zabbix_trapper, text;
• dummy_trigger_0:
  • problem expression: {dummy_host_0:dummy_item_0.strlen(#1)}

    >5;

  • recovery expression: {dummy_host_0:dummy_item_0.strlen(#1)}

    <=3;

  • single problem generation mode;
  • ok events closes all problems;
  • tags - dummy_tag_0, no value.

dummy_host_1

• dummy_item_1: zabbix_trapper, text;
• dummy_trigger_1:
  • problem expression: {dummy_host_1:dummy_item_1.strlen(#1)}

    >5;

  • recovery expression: {dummy_host_1:dummy_item_1.strlen(#1)}

    <=3;

  • single problem generation mode;
  • ok events closes all problems;
  • tags - dummy_tag_1, no value.

dummy_correlation_0: And/Or.

Conditions:

• Old event tag = dummy_tag_0;
• New event tag = dummy_tag_1.

Operations:

• Close old events;
• Close new event;

wiper Just clarifying - the OK events generated by event correllation should not have tags.
wiper RESOLVED in ZBXNEXT-3274-3 r61227
sandis.neilands CLOSED.

(13) [S] After some testing I noticed that some events in have ''Unknown' status (see event_in_unknown_state.png). Their 'value' in 'events' table is 3 which corresponds to TRIGGER_VALUE_NONE in server. Supposedly this value should never be written to DB.

sandis.neilands CLOSED. This issue will be fixed in ZBXNEXT-3274 (10).

wiper the fix was merged in, should be working starting with r61157

(14) [A] Event correlation with custom expressions return Undefined offset

–Undefined offset: 54 [correlation.php:471 → CFrontendApiWrapper->get() → CApiWrapper->__call() → CFrontendApiWrapper->callMethod() → CApiWrapper->callMethod() → CFrontendApiWrapper->callClientMethod() → CLocalApiClient->callMethod() → call_user_func_array() → CCorrelation->get() → CCorrelation->addRelatedObjects() in include\classes\api\services\CCorrelation.php:1438]
–Undefined offset: 55 [correlation.php:471 → CFrontendApiWrapper->get() → CApiWrapper->__call() → CFrontendApiWrapper->callMethod() → CApiWrapper->callMethod() → CFrontendApiWrapper->callClientMethod() → CLocalApiClient->callMethod() → call_user_func_array() → CCorrelation->get() → CCorrelation->addRelatedObjects() in include\classes\api\services\CCorrelation.php:1438]
–Undefined offset: 56 [correlation.php:471 → CFrontendApiWrapper->get() → CApiWrapper->__call() → CFrontendApiWrapper->callMethod() → CApiWrapper->callMethod() → CFrontendApiWrapper->callClientMethod() → CLocalApiClient->callMethod() → call_user_func_array() → CCorrelation->get() → CCorrelation->addRelatedObjects() in include\classes\api\services\CCorrelation.php:1438]
–Undefined offset: 57 [correlation.php:471 → CFrontendApiWrapper->get() → CApiWrapper->__call() → CFrontendApiWrapper->callMethod() → CApiWrapper->callMethod() → CFrontendApiWrapper->callClientMethod() → CLocalApiClient->callMethod() → call_user_func_array() → CCorrelation->get() → CCorrelation->addRelatedObjects() in include\classes\api\services\CCorrelation.php:1438]
–Undefined offset: 58 [correlation.php:471 → CFrontendApiWrapper->get() → CApiWrapper->__call() → CFrontendApiWrapper->callMethod() → CApiWrapper->callMethod() → CFrontendApiWrapper->callClientMethod() → CLocalApiClient->callMethod() → call_user_func_array() → CCorrelation->get() → CCorrelation->addRelatedObjects() in include\classes\api\services\CCorrelation.php:1438]
–Undefined offset: 59 [correlation.php:471 → CFrontendApiWrapper->get() → CApiWrapper->__call() → CFrontendApiWrapper->callMethod() → CApiWrapper->callMethod() → CFrontendApiWrapper->callClientMethod() → CLocalApiClient->callMethod() → call_user_func_array() → CCorrelation->get() → CCorrelation->addRelatedObjects() in include\classes\api\services\CCorrelation.php:1438]
–Undefined offset: 60 [correlation.php:471 → CFrontendApiWrapper->get() → CApiWrapper->__call() → CFrontendApiWrapper->callMethod() → CApiWrapper->callMethod() → CFrontendApiWrapper->callClientMethod() → CLocalApiClient->callMethod() → call_user_func_array() → CCorrelation->get() → CCorrelation->addRelatedObjects() in include\classes\api\services\CCorrelation.php:1438]
–Undefined offset: 61 [correlation.php:471 → CFrontendApiWrapper->get() → CApiWrapper->__call() → CFrontendApiWrapper->callMethod() → CApiWrapper->callMethod() → CFrontendApiWrapper->callClientMethod() → CLocalApiClient->callMethod() → call_user_func_array() → CCorrelation->get() → CCorrelation->addRelatedObjects() in include\classes\api\services\CCorrelation.php:1438]
–Undefined offset: 62 [correlation.php:471 → CFrontendApiWrapper->get() → CApiWrapper->__call() → CFrontendApiWrapper->callMethod() → CApiWrapper->callMethod() → CFrontendApiWrapper->callClientMethod() → CLocalApiClient->callMethod() → call_user_func_array() → CCorrelation->get() → CCorrelation->addRelatedObjects() in include\classes\api\services\CCorrelation.php:1438]
–Undefined offset: 63 [correlation.php:471 → CFrontendApiWrapper->get() → CApiWrapper->__call() → CFrontendApiWrapper->callMethod() → CApiWrapper->callMethod() → CFrontendApiWrapper->callClientMethod() → CLocalApiClient->callMethod() → call_user_func_array() → CCorrelation->get() → CCorrelation->addRelatedObjects() in include\classes\api\services\CCorrelation.php:1438]
–Undefined offset: 64 [correlation.php:471 → CFrontendApiWrapper->get() → CApiWrapper->__call() → CFrontendApiWrapper->callMethod() → CApiWrapper->callMethod() → CFrontendApiWrapper->callClientMethod() → CLocalApiClient->callMethod() → call_user_func_array() → CCorrelation->get() → CCorrelation->addRelatedObjects() in include\classes\api\services\CCorrelation.php:1438]

1. Create event correlation with custom expression
2. Update custom expression, removing element and removing tag
3. Open it again

iivs RESOLVED in r61125

oleg.egorov CLOSED

(15) [F] Try to create New condition->New event tag value without tag, but with value
Result is correct error message "Incorrect value for field "tag": cannot be empty."
But in the form missed value for this condition.

iivs RESOLVED in r61124

oleg.egorov CLOSED

(16) [F] Error if host group was removed

Undefined offset: 3 [correlation.php:452 → CView->render() → include() in include\views\configuration.correlation.edit.php:71]
Argument 2 passed to getCorrConditionDescription() must be of the type array, null given, called in C:\xampp\htdocs\ZBXNEXT-3277-4\frontends\php\include\views\configuration.correlation.edit.php on line 71 and defined [correlation.php:452 → CView->render() → include() → getCorrConditionDescription() in include\correlation.inc.php:190]

1. Create host group: test
2. Select it in New event host group
3. Remove host group
4. Add condition

iivs RESOLVED in r61162

oleg.egorov CLOSED

(17) [S] When processing correlation rule 'old event tag xxx = new event tag yyy' if the new event has multiple yyy tags only the first one is checked.

RESOLVED in r61144 (and r61228 in ZBXNEXT-3274-3 branch)

sandis.neilands CLOSED.

Event recovery generated duplicate records.

[Z3005] query failed: [1062] Duplicate entry '5633178' for key 'PRIMARY' [insert into event_recovery (eventid,r_eventid,correlationid,c_eventid) values (5633226,5633313,null,null),(5633178,5633310,null,null),(5633285,5633310,null,null),(5633243,5633309,null,null),(5633227,5633314,null,null),(5633225,5633312,null,null),(5633224,5633311,null,null);]

Relevant parts of log attached.

wiper Yes, event recovery is currently broken (also in trunk). It has been fixed and is currently being tested in svn://svn.zabbix.com/branches/dev/ZBXNEXT-3274-2

How long wait working trunk?

The fix in ZBXNEXT-3274-2 should be in trunk by tomorrow morning.

(18) [S] Event correlation does not work when mutiple conditions involving old event tags are used.
RESOLVED in ZBXNEXT-3274-3 r61183

sandis.neilands CLOSED.

(19) [S] Event correlation might not work when a large batch of events is processed.
RESOLVED in ZBXNEXT-3274-3 r61203

sandis.neilands CLOSED.

(20) [A] Possible create tag duplicates

{
    "name": "long_test",
    "description": "",
    "status": 0,
    "operations": {
        "0": {
            "type": 0
        }
    },
    "filter": {
        "conditions": {
            "0": {
                "type": 0,
                "tag": "1",
                "formulaid": "A",
                "operator": 0
            },
            "1": {
                "type": 1,
                "operator": 0,
                "tag": "1",
                "formulaid": "B"
            },
            "2": {
                "type": 1,
                "operator": 0,
                "tag": "1",
                "formulaid": "C"
            }
        },
        "evaltype": 0
    },
    "correlationid": 7
}

In frontend:
A Old event tag = 1
B New event tag = 1
C New event tag = 1

RESOLVED IN r61278

gunarspujats CLOSED

(21) [A] FormulaId duplicates

{
    "name": "1",
    "description": "",
    "status": 0,
    "operations": {
        "0": {
            "type": 0
        }
    },
    "filter": {
        "conditions": {
            "1": {
                "type": 1,
                "operator": 0,
                "tag": "1",
                "formulaid": "B"
            },
            "2": {
                "type": 1,
                "operator": 0,
                "tag": "1",
                "formulaid": "B"
            }
        },
        "evaltype": 3,
        "formula": "B and B"
    },
    "correlationid": 1
}

Result in frontend:

Undefined offset: 180 [correlation.php:406 → CFrontendApiWrapper->get() → CApiWrapper->__call() → CFrontendApiWrapper->callMethod() → CApiWrapper->callMethod() → CFrontendApiWrapper->callClientMethod() → CLocalApiClient->callMethod() → call_user_func_array() → CCorrelation->get() → CCorrelation->addRelatedObjects() in include\classes\api\services\CCorrelation.php:1449]

RESOLVED IN r61278

gunarspujats CLOSED

If testing with Valgrind then be aware of issue (3) from ZBXNEXT-3195 which seems to be a false positive.

Hmm. I'm observe crashes when server update lld rules. In lld_triggers_validate() missed initialization of new fields?

@@ -1950,6 +1950,8 @@ static void       lld_triggers_validate(zbx_ui
                        db_trigger->comments_orig = NULL;
                        db_trigger->url = NULL;
                        db_trigger->url_orig = NULL;
+                       db_trigger->correlation_tag = NULL;
+                       db_trigger->correlation_tag_orig = NULL;
                        db_trigger->flags = ZBX_FLAG_LLD_TRIGGER_UNSET;

wiper Thanks, added sub-issue ZBXNEXT-3274 (2)

(22) Any example regarding correlation conditions where "And" expression is useful and "And/Or" is not applicable?

wiper I can give an artificial example, but not sure how useful it would be: event value like ABC and event value like 123.

sandis.neilands If we do not have immediate plans to use it then shouldn't we remove it to avoid confusion? For the corner case you mentioned a custom expression is always available.

wiper I think it's less confusing to have the same evaluation type options as in other places.

sandis.neilands Ok, CLOSED.

(23) [S] ZBXNEXT-3274-3 branch: ZBX_CORR_CONDITION_NEW_EVENT_HOSTGROUP not handled in correlation_condition_get_event_filter() thus THIS_SHOULD_NEVER_HAPPEN happens in correlation_add_event_filter().

zabbix_server [596]: ERROR [file:events.c,line:1182] Something impossible has just happened.

wiper RESOLVED in r61234 (ZBXNEXT-3274-3 branch)

sandis.neilands CLOSED.

(24) [F] When configuring event tag pair it is not clear which text fields is for old tag and which one is for the new tag (both have 'tag' in light gray).

oleg.egorov RESOLVED in r61256

sasha Placeholders are renamed to old event tag and new event tag accordingly. CLOSED

(25) [F] What is the maximum number of problems that the 'Problems' page can handle? I have around 50 thousand problems. 'Events' view shows them (21 page that is) but 'Problems' view does not load (it displays white page and that's it).

oleg.egorov Check your memory limit in php.ini and max execution time

sandis.neilands 128M and 300s. The point is that events page still works with these settings.

oleg.egorov Confirmed performance issue

Moved to ZBXNEXT-3201 (16). CLOSED

(26) [F] What does the 'Action' column in 'Problems' page is supposed to show? Just the actions taken for problem event, problem and ok event?

oleg.egorov Moved to ZBXNEXT-3201 (17). CLOSED

(27) [S] IT services are calculated from event value. With correlation the event value might differ from the trigger value. For example when trigger has multiple open problems OK event might close only one of them, leaving trigger in problem state.

To fix it we should use the trigger changeset (trigger_diff vector) for IT service calculations instead of events cache.

wiper RESOLVED in r61258 (ZBXNEXT-3274-3 branch)

sandis.neilands CLOSED.

on trunk r61206 - all triggers with dependencies in locked state, used size in history cache constantly growing up, all items with such triggers never update in db.

All of these triggers - timer based, locked inside process_time_functions() by DCconfig_get_time_based_triggers() and never be unlocked. DCconfig_lock_triggers_by_history_items() can't lock it after new value received and unsynced entries in history buffer growing.

I confirm that in current trunk, seems like time-based triggers are unlocked only if they generate events. Thank you, lynxchaus, for analysis!

wiper Added to ZBXNEXT-3274

(28) [A] Coding style

include/classes/api/services/CCorrelation.php
:97,1398 - deprecated function isset() is used
:920, 1528 - mistakes in PHPDoc syntax
:1360, 1381 - missing PHPDoc
:1364 - comment syntax doesn't match coding guideline
:1512 - definition of unused variable $operationids
:1536 - used deprecated function empty()

RESOLVED in dev branch svn://svn.zabbix.com/branches/dev/ZBXNEXT-3277-4, r61352

sasha CLOSED

(29) [F] Coding style

correlation.php
:122, 503 - comment syntax doesn't match coding guideline

include/correlation.inc.php
:89, 96 - redundant break; statement

RESOLVED in dev branch svn://svn.zabbix.com/branches/dev/ZBXNEXT-3277-4, r61365

sasha CLOSED

Server side merged to ZBXNEXT-3274-3 --> trunk (r61393).

Frontend and API merged to trunk r61443 (ZBXNEXT-3277-4)